Commit graph

7 commits

Author SHA1 Message Date
2efb35dc48
Stop scrape chain from dying on a single bad product
A scraped product with nil price_per_round_cents made Prices.create_snapshot
return {:error, changeset}, which blew up the {:ok, _} match in the runner,
crashed the whole perform, and after 3 retries discarded the job. Nothing
reseeds the chain once the job is discarded, so prod scrapes silently died
on Mar 17 and stayed dead until the app was restarted.

- Runner handles snapshot insert errors by logging and continuing instead
  of raising.
- ScrapeJob wraps each Runner.run in try/rescue so one retailer crashing
  doesn't take down the whole cycle.
- ScrapeJob moves schedule_next into an after block so the chain is always
  reseeded, even if the scrape loop crashes.
2026-04-22 15:51:15 -05:00
ae7e117b23
Prune listings that disappear from retailer scrapes
When a product no longer appears in a successful scrape for its
retailer+caliber, mark it out of stock. After each scrape cycle, hard
delete any product whose last_seen_at is older than 30 days (snapshots
cascade via FK). A zero-result scrape leaves existing listings alone so
a parse failure or transient site glitch can't nuke the catalog.
2026-04-22 15:41:40 -05:00
0171ef1570
Separate 5.56x45 NATO and .223 Remington into distinct calibers
Previously combined as a single "556-223" caliber. Now tracked separately
with dedicated scraper URLs per retailer. Also fixes broken URLs for
Lucky Gunner, True Shot Ammo, and Palmetto State Armory.
2026-03-12 12:27:59 -05:00
9eb85faebd
Add 4 new retailer scrapers for broader price coverage
Add True Shot Ammo (Shopify JSON), Palmetto State Armory (Magento HTML),
Bulk Ammo (Magento HTML), and Natchez Shooters Supply (GraphQL API).

Extends scraper infrastructure with HttpClient.post_json/3 and an
optional fetch/2 callback on the Scraper behaviour for scrapers that
need custom HTTP flows (e.g. GraphQL POST requests).
2026-03-12 10:00:59 -05:00
cd8c080846
Seed retailers and calibers via database migration 2026-03-12 08:55:12 -05:00
16a28ac01c
Add Target Sports USA scraper, product filtering, and subsonic detection
- Add subsonic boolean field to products with composite filter indexes
- Add TextDetector utility for detecting subsonic/casing from product titles
- Wire TextDetector into SgAmmo (casing + subsonic) and LuckyGunner (subsonic + fallback casing)
- Extend latest_prices_for_caliber with casing, grain_weight, and subsonic filter options
- Add distinct_grain_weights_for_caliber and distinct_casings_for_caliber queries
- Add filter UI with toggleable casing, grain weight, and subsonic buttons
- Add grain weight column and subsonic badge to product table
- Refresh filter options on PubSub price updates for real-time UI
- Implement Target Sports USA scraper with 18 caliber mappings
- Display all prices in dollar format ($0.38 instead of 38¢)
2026-03-11 16:28:04 -05:00
bbe5bde145
Initial implementation of ammo price tracker
Phoenix 1.8 app that scrapes ammunition retailers (Lucky Gunner,
SGAmmo) for price data and displays historical price trends.

- Data model: retailers, calibers, products, price snapshots
- Scraper infrastructure with Req, Floki, realistic browser headers
- Oban-scheduled scrape jobs (every 4h with randomized delays)
- LiveView pages: homepage with category cards, caliber detail with
  price table, Chart.js price history, and price stats banner
- 18 seeded calibers across handgun/rifle/rimfire/shotgun categories
- 77 tests
2026-03-11 15:58:12 -05:00