Pricewatch: competitor price intelligence that runs every night

Pricewatch is a demo Bravko built to answer one question a prospect always asks: what does the output actually look like? It runs every night against real public retail sites, so the answer is a real spreadsheet and a real dashboard, not a slide.

The problem. A team that sells online wants to know when a competitor moves. The manual version is a person opening the same forty product pages each morning, copying prices into a spreadsheet, and guessing at stock. It is slow, it is wrong by lunchtime, and it stops the moment that person is on holiday. The day a rival drops a bestseller by two pounds is the day nobody checked.

What we built. A configuration file lists a brand store and its competitors. Each night a collector visits their product pages, reads the structured data the pages already carry (JSON-LD first, the store platform’s own product endpoints for variant barcodes, a selector fallback last), and stores one observation per product per run with the full history kept. A tiered matcher then pairs the same product across stores: exact barcode sets first, then title plus size, then “closest” proposals that a reviewer accepts or rejects from the command line. Accepted pairs come back as confirmed matches on every later run; rejected pairs are never proposed again.

Two things read from that history. The artifact exporter writes an .xlsx with a Summary tab for outreach, which lists only claim-grade matches, headline counts, the brand’s own price, one short flag per competitor, and rows sorted by price gap, and a Data tab with everything including match confidence. The dashboard is a read-only page with prices, changes, stock, and alerts, and a stats strip computed live from the database.

The decisions that mattered. Product pages on the usual store platforms carry one barcode per variant, so a product stores the whole set and the exact tier matches on any overlap; without that, two stores’ pages for the same product rarely share the “first” barcode. A shared brand name is never enough for a match: after the brand tokens are stripped, two titles must still share a product-specific word, which removed a chain of junk matches on the first real run. Some stores strip their structured data from the page after it loads, so the collector reads the raw server response before the browser touches it. And uncertain matches never drive a claim: they appear in grey for context, get no cheaper or pricier flag, and are excluded from the prospect spreadsheet even inside rows that qualify through another competitor.

Politeness rules. robots.txt is fetched once per host with a descriptive user agent and obeyed; a disallowed page is skipped and recorded. Every two requests to the same host are separated by two to three seconds of jittered delay, and a larger crawl delay in robots.txt wins. Listing pages are paginated only while each new page adds a product not seen before. Sites that block bots, put a password on their catalogue, or serve wholesale prices were rejected during feasibility rather than worked around.

Drift detection. A run that extracts no product URLs from a listing, a product count that halves against the previous run, or a robots.txt that changes its answer all raise an alert on the dashboard. Automation that silently reports wrong numbers is worse than none, so the system says when it is unsure.

How it is built. Python 3.12 with Playwright for collection, SQLite with the complete observation history, FastAPI and Jinja for the dashboard (no JavaScript on the page), openpyxl for the artifact. It runs as a systemd timer on a small server, one process, with retries and backoff. 115 unit tests and one live smoke test cover it.

Where it stands. Pricewatch is running nightly against four retailers. The numbers on this page come from the collector’s own stats, updated automatically once the demo publishes them; nothing here is estimated.