Skip to content

1.5.9

⚙️ Patch 1.5.9 — Otto monitor, Scrapling migration, Docker patchright, Cron safety & FBA fix

🚀 Summary

  • Small release with stability and reliability improvements plus a migration of the Otto scraping layer from Selenium to Scrapling.

✨ New Features

  • Scrapling evaluation script: added a small test harness eval_scrapling.py to sample-scrape pages with and without proxies using Scrapling's StealthyFetcher/StealthySession.
  • Docker: include patchright install in the service image so the Patchright engine required by Scrapling is available at runtime.

🔧 Improvements

  • Scrapling migration: replaced Selenium / selenium-wire scraping with Scrapling's StealthySession and StealthyFetcher in otto_service.py. This simplifies driver lifecycle, reduces orphaned WebDriver sessions, and improves anti-bot handling.
  • Proxy/session handling: proxies are now bound to Scrapling sessions via _build_proxy_string() and rotated by re-creating sessions, avoiding per-request proxy injection that Scrapling doesn't support.
  • Removed Selenium helpers: setup_driver/scroll_page were removed or stubbed; cleanup_driver / kill_driver / cleanup_all_selenium_sessions are now no-ops.
  • Additional-offers detection and offer parsing: more robust HTML access (use page.body / page.content), safer CSS parsing via page.css(), and a rewritten extract_offer_prices() that includes shipping parsing.

🐛 Bug Fixes

  • Fixed: do_sync_articles_from_otto() returned early inside its loop — now all SKUs are processed (was causing only the first SKU to be synced).
  • Fixed: extract_offer_prices() rewritten for Scrapling (parses .js_of_offers-price__inner, retail price selectors and shipping costs correctly).
  • Fixed: additional-offers check crashed with "'Response' object has no attribute 'html'" — now uses page.body/page.content (bytes-safe) instead.
  • Plenty integration: price changes detected by the Otto monitor are now collected and forwarded to Plenty.
  • Cron robustness and FBA export fixes (retain earlier stability changes): scheduler guard rails and treat missing ignore as False.

📁 Files changed

  • backend/service_multi-repricer/app/api/v1/endpoints/otto_articles.py — fix return/notification placement so sync processes all SKUs and webhooks are sent appropriately.
  • backend/service_multi-repricer/app/services/Otto/otto_service.py — full Scrapling migration: session/proxy refactor, fetch_prices_from_skus rewrite, _scrape_product_page rewrite, extract_offer_prices rewrite, and Selenium cleanup stubs.
  • backend/service_multi-repricer/eval_scrapling.py — evaluation/test script for Scrapling (proxy/no-proxy sampling).
  • docker/service_multi-repricer/Dockerfile — add RUN patchright install to install the Patchright engine used by Scrapling.
  • backend/service_multi-repricer/requirements.txt — pin pyOpenSSL<24.0.0 (mitmproxy/selenium-wire compatibility note retained).
  • backend/service_multi-repricer/app/api/v1/endpoints/fba_articles.py — FBA export: treat missing ignore flag as False to avoid skipping exports.