Commit graph

25 commits

Author SHA1 Message Date
c56bc12e88
chore: update dependencies 2026-07-08 14:30:50 -05:00
872c94bea1
Fix bugs and performance: Oban chain, 0-cent prices, duplicate queries, regex fragility
- Fix Oban self-scheduling chain silently dying (remove unique constraint so after block can schedule next job)
- Fix Natchez and TrueShotAmmo extract_price defaulting to 0 for missing price data
- Fix caliber_matcher redundant String.downcase inside Enum.find (pre-downcase outside loop)
- Fix runner.ex double iteration (Enum.map + Enum.count merged into single Enum.reduce)
- Fix price_stats_for_caliber firing 3 queries instead of 2 (merge all_time + thirty_day via FILTER)
- Fix PSA duplicate Floki.find on same selector (merge extract_title + extract_url)
- Fix PubSub broadcast carrying no caliber IDs (now includes IDs; clients skip irrelevant reloads)
- Fix BulkAmmo ^ anchor skipping non-leading round counts
- Fix TargetSportsUSA fragile exact text match 'Add To Cart' (use case-insensitive contains)
- Fix SgAmmo dead destructure {_, _, _} = row
- Fix text_detector brass/nickel detection order
- Fix Natchez GraphQL string interpolation (add escape_gql_string)
- Fix chart data triple Enum.map merged into single reduce
- Change Oban scraping queue workers from 2 to 1 (only one needed with Process.sleep)
2026-06-21 18:14:49 -05:00
a52a73ee09
Update dependencies, add Plausible analytics, fix Oban unique states warning 2026-06-06 12:43:19 -05:00
66b39d8ad3
Update dependencies
Includes Oban 2.22 upgrade which requires migration to v14 (adds suspended
state) and bumps decimal to 3.x.
2026-05-11 10:12:53 -05:00
53b04b9090
chore: pin erlang 28.5 and elixir 1.19.5-otp-28 2026-05-05 16:38:15 -05:00
2efb35dc48
Stop scrape chain from dying on a single bad product
A scraped product with nil price_per_round_cents made Prices.create_snapshot
return {:error, changeset}, which blew up the {:ok, _} match in the runner,
crashed the whole perform, and after 3 retries discarded the job. Nothing
reseeds the chain once the job is discarded, so prod scrapes silently died
on Mar 17 and stayed dead until the app was restarted.

- Runner handles snapshot insert errors by logging and continuing instead
  of raising.
- ScrapeJob wraps each Runner.run in try/rescue so one retailer crashing
  doesn't take down the whole cycle.
- ScrapeJob moves schedule_next into an after block so the chain is always
  reseeded, even if the scrape loop crashes.
2026-04-22 15:51:15 -05:00
ae7e117b23
Prune listings that disappear from retailer scrapes
When a product no longer appears in a successful scrape for its
retailer+caliber, mark it out of stock. After each scrape cycle, hard
delete any product whose last_seen_at is older than 30 days (snapshots
cascade via FK). A zero-result scrape leaves existing listings alone so
a parse failure or transient site glitch can't nuke the catalog.
2026-04-22 15:41:40 -05:00
200b6dd9cf
Update dependencies
- bandit 1.10.3 -> 1.10.4
- credo 1.7.17 -> 1.7.18
- floki 0.38.0 -> 0.38.1
- lazy_html 0.1.10 -> 0.1.11
- oban 2.20.3 -> 2.21.1
- phoenix_live_view 1.1.27 -> 1.1.28
- swoosh 1.23.0 -> 1.25.0
- Chart.js 4.4.7 -> 4.5.1 (vendored)
2026-04-22 15:28:07 -05:00
525692fe05
Add chart empty state and home-page real-time price refresh
- Show "No price history available for this range yet." overlay when
  the caliber chart range contains no daily averages (previously the
  chart rendered as an unlabeled empty grid).
- Subscribe HomeLive to the "prices:updated" PubSub topic so the
  cheapest-per-caliber grid refreshes without a full page reload.
- Drop the unused textColor computation and unused max series from the
  chart hook/data.
2026-04-22 15:28:02 -05:00
f075ab55fb
Fix price chart destroyed by LiveView DOM patches on filter change
LiveView's morphdom was replacing the canvas element inside the chart
container whenever filters were toggled, destroying the Chart.js
rendering context. Add phx-update="ignore" to the chart container so
LiveView preserves JS-managed children after initial render.
2026-03-13 13:38:45 -05:00
fd907c40cb
Add credo with --strict mode and fix all issues 2026-03-12 13:35:12 -05:00
77ff082ebb
Switch to self-scheduling scrapes with longer intervals
Replace cron-based scheduling (every 4h) with self-scheduling jobs
that randomly pick 7-12 hours between runs. Also increase the
inter-request delay from 5-15 minutes to 10-20 minutes.
2026-03-12 13:26:46 -05:00
0171ef1570
Separate 5.56x45 NATO and .223 Remington into distinct calibers
Previously combined as a single "556-223" caliber. Now tracked separately
with dedicated scraper URLs per retailer. Also fixes broken URLs for
Lucky Gunner, True Shot Ammo, and Palmetto State Armory.
2026-03-12 12:27:59 -05:00
0340fc85be
Increase scrape delay to 5-15 minutes between requests 2026-03-12 10:06:43 -05:00
9eb85faebd
Add 4 new retailer scrapers for broader price coverage
Add True Shot Ammo (Shopify JSON), Palmetto State Armory (Magento HTML),
Bulk Ammo (Magento HTML), and Natchez Shooters Supply (GraphQL API).

Extends scraper infrastructure with HttpClient.post_json/3 and an
optional fetch/2 callback on the Scraper behaviour for scrapers that
need custom HTTP flows (e.g. GraphQL POST requests).
2026-03-12 10:00:59 -05:00
0583bd14cf
Add 16 gauge shotgun caliber 2026-03-12 09:04:00 -05:00
d7c4772ad8
Only show subsonic filter when subsonic products exist for caliber 2026-03-12 09:01:16 -05:00
cd8c080846
Seed retailers and calibers via database migration 2026-03-12 08:55:12 -05:00
56d02cf8cb
Remove icons from category headings on home page 2026-03-12 08:10:06 -05:00
cd731ee22b
Fix executable permissions on release scripts 2026-03-12 07:55:19 -05:00
22c9854afd
Run database migrations on each server startup 2026-03-11 16:54:06 -05:00
324c5c556c
Add Dokku deployment configuration 2026-03-11 16:51:44 -05:00
a24448b2da
Rebrand to AmmoCPR with ammocpr.com domain 2026-03-11 16:34:58 -05:00
16a28ac01c
Add Target Sports USA scraper, product filtering, and subsonic detection
- Add subsonic boolean field to products with composite filter indexes
- Add TextDetector utility for detecting subsonic/casing from product titles
- Wire TextDetector into SgAmmo (casing + subsonic) and LuckyGunner (subsonic + fallback casing)
- Extend latest_prices_for_caliber with casing, grain_weight, and subsonic filter options
- Add distinct_grain_weights_for_caliber and distinct_casings_for_caliber queries
- Add filter UI with toggleable casing, grain weight, and subsonic buttons
- Add grain weight column and subsonic badge to product table
- Refresh filter options on PubSub price updates for real-time UI
- Implement Target Sports USA scraper with 18 caliber mappings
- Display all prices in dollar format ($0.38 instead of 38¢)
2026-03-11 16:28:04 -05:00
bbe5bde145
Initial implementation of ammo price tracker
Phoenix 1.8 app that scrapes ammunition retailers (Lucky Gunner,
SGAmmo) for price data and displays historical price trends.

- Data model: retailers, calibers, products, price snapshots
- Scraper infrastructure with Req, Floki, realistic browser headers
- Oban-scheduled scrape jobs (every 4h with randomized delays)
- LiveView pages: homepage with category cards, caliber detail with
  price table, Chart.js price history, and price stats banner
- 18 seeded calibers across handgun/rifle/rimfire/shotgun categories
- 77 tests
2026-03-11 15:58:12 -05:00