diff --git a/CLAUDE.md b/CLAUDE.md index 784dfef..6f72694 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co ## Project Overview -Ammoprices is a Phoenix 1.8 web application (Elixir ~> 1.15) with LiveView, Ecto/PostgreSQL, and Tailwind CSS v4 with daisyUI. Currently a fresh scaffold with no domain logic implemented yet. +Ammoprices is a Phoenix 1.8 web application that scrapes ammunition prices from multiple online retailers, tracks price history, and displays analytics. Built with Elixir ~> 1.15, LiveView, Ecto/PostgreSQL, Tailwind CSS v4, and daisyUI. ## Common Commands @@ -27,46 +27,88 @@ mix ecto.reset # Drop and recreate database ## Architecture -### Application Structure +### Domain Contexts -- **`Ammoprices.Application`** — Supervision tree: Telemetry, Repo, DNSCluster, PubSub, Endpoint -- **`Ammoprices.Repo`** — Ecto repo using PostgreSQL adapter -- **`Ammoprices.Mailer`** — Swoosh mailer (local adapter in dev, viewable at `/dev/mailbox`) +**Catalog** (`lib/ammoprices/catalog/`) — Retailers, calibers, and products. +- `Retailer`: name, slug (unique), base_url, enabled flag, last_scraped_at +- `Caliber`: name, slug (unique), category (handgun/rifle/rimfire/shotgun), aliases array +- `Product`: belongs_to retailer & caliber. Fields: title, url (unique per retailer), brand, grain_weight, round_count, casing (brass/steel/aluminum/alloy/composite), subsonic, in_stock. Upserted on (retailer_id, url) conflict -### Web Layer (`lib/ammoprices_web/`) +**Prices** (`lib/ammoprices/prices/`) — Immutable price snapshots and analytics. +- `PriceSnapshot`: price_cents, price_per_round_cents, in_stock, recorded_at. Append-only (no updated_at) +- Queries: latest prices per product (with filters), daily averages, price stats (min/max/avg), cheapest per caliber -- **`AmmopricesWeb`** (`ammoprices_web.ex`) — Defines `use` macros for `:router`, `:controller`, `:live_view`, `:live_component`, `:html`. The `html_helpers/0` block imports `CoreComponents`, aliases `Layouts` and `Phoenix.LiveView.JS`, and sets up verified routes -- **`AmmopricesWeb.Router`** — Browser pipeline only. Currently just `GET / → PageController.home` -- **`AmmopricesWeb.CoreComponents`** — UI component library using daisyUI + Tailwind. Includes `flash/1`, `button/1`, `icon/1`, standard form/table components -- **`AmmopricesWeb.Layouts`** — `app/1` layout with navbar and theme toggle (light/dark/system). Templates wrap content with `` +### Scraping Pipeline (`lib/ammoprices/scraping/`) -### Config +``` +ScrapeJob (Oban, every 4h) → Runner → Scraper (per retailer) → HttpClient → Floki/JSON parse + ↓ + Upsert products → Create snapshots → PubSub broadcast +``` -- **`config/config.exs`** — Generator defaults: UTC timestamps, binary IDs -- **`config/dev.exs`** — DB: `ammoprices_dev`, live reload watchers for esbuild/tailwind -- **`config/test.exs`** — DB: `ammoprices_test`, SQL sandbox mode +- **Scraper behaviour**: Required callbacks `retailer_slug/0`, `category_url/1`, `parse_products/1`. Optional `fetch/2` for custom HTTP (e.g., GraphQL) +- **Runner**: Checks `function_exported?` for `fetch/2` — uses it if available, otherwise standard HTTP GET + parse flow +- **HttpClient**: Req-based with realistic Chrome headers, retries. Has `get/1` and `post_json/3` +- **TextDetector**: Regex detection of subsonic and casing type from product titles +- **CaliberMatcher**: Matches products to calibers via name/alias lookup +- **ScrapeJob**: Oban worker, queue `:scraping` (concurrency 2), 5-15 min random delay between requests in prod, 0 in test -### Assets (`assets/`) +**Retailers** (7 scrapers in `lib/ammoprices/scraping/retailers/`): +| Retailer | Type | Notes | +|---|---|---| +| Lucky Gunner | HTML (Floki) | Slug-to-URL mapping per caliber | +| SG Ammo | HTML (Floki) | Standard HTML parsing | +| Target Sports USA | HTML (Floki) | Standard HTML parsing | +| True Shot Ammo | JSON API (Shopify) | `/collections/{handle}/products.json`, `parse_products` accepts decoded map | +| Palmetto State Armory | HTML (Magento) | `data-price-amount` attrs, brand = first word of title | +| Bulk Ammo | HTML (Magento) | `a.product-name`, brand via regex from title | +| Natchez | GraphQL | Uses `fetch/2` callback, `HttpClient.post_json/3`, requires `Store: default` header | -- **Tailwind CSS v4** with `@import "tailwindcss"` syntax (no `tailwind.config.js`) -- **daisyUI** plugin with custom light/dark themes defined in `app.css` -- **esbuild** bundles `app.js` — only `app.js` and `app.css` bundles are supported -- Vendor libs: `topbar.js`, `heroicons.js`, `daisyui.js`, `daisyui-theme.js` -- JS hooks for LiveView use colocated hook syntax (names prefixed with `.`) +### Web Layer -### Test Support (`test/support/`) +**Routes:** +- `GET /` → `HomeLive` — Calibers grouped by category with cheapest CPR per caliber +- `GET /calibers/:slug` → `CaliberLive.Show` — Price table, Chart.js chart, filter bar, stats -- **`ConnCase`** — For controller/LiveView tests, sets up conn and DB sandbox -- **`DataCase`** — For context/schema tests, provides `errors_on/1` helper -- **`LazyHTML`** — Available in tests for HTML assertion selectors +**CaliberLive.Show filters:** in_stock (toggle), casing (mutually exclusive), grain_weight (mutually exclusive), subsonic (toggle, only shown when subsonic products exist). Chart range: 7d/30d/90d/1y/all. + +**Real-time:** PubSub on `"prices:updated"` topic. CaliberLive.Show subscribes on connect; broadcasts trigger reload of filter options, product stream, chart, and stats. + +**Components:** +- `PriceComponents`: `price_per_round/1` (formats cents as $0.38), `stock_badge/1`, `retailer_link/1` +- `CoreComponents`: Standard Phoenix component library (daisyUI + Tailwind) + +### Assets + +- **Tailwind CSS v4** — `@import "tailwindcss"` syntax in `app.css`, no `tailwind.config.js` +- **daisyUI** — Custom light/dark themes defined in `app.css` +- **Chart.js** — Vendored in `assets/vendor/chart.js`, used via colocated `.PriceChart` hook +- **esbuild** — Bundles `app.js` only. `@` alias maps to `assets/vendor/` for imports +- Colocated hook imports **must** use `@/vendor/...` alias, NOT relative paths (hooks compile to `_build/`) + +### Database + +- Binary UUIDs, UTC timestamps throughout +- Products use `on_conflict: {:replace, [fields...]}` for upserts on (retailer_id, url) +- Product filter indexes: (caliber_id, in_stock), (caliber_id, casing), (caliber_id, grain_weight), (caliber_id, subsonic) +- Retailers and calibers seeded via database migration (not seeds.exs) + +### Test Infrastructure + +- **Fixtures**: `test/support/fixtures.ex` — factory functions (retailer_fixture, caliber_fixture, product_fixture, snapshot_fixture) +- **HTML/JSON fixtures**: `test/fixtures/{retailer_name}/` — scraped response samples for each retailer +- **HTTP stubbing**: `Req.Test` with plug config in test.exs +- **Oban**: `:manual` mode in test (jobs don't auto-run) +- **Scrape delay**: `{0, 0}` in test config ## Key Conventions - **HTTP client**: Use `Req` (already included). Never use HTTPoison, Tesla, or `:httpc` +- **Prices**: Display in dollar format ($0.38/rd), stored as integer cents internally - **LiveView templates**: Always begin with `` wrapper +- **Collections**: Always use LiveView streams, never assign raw lists - **Icons**: Use `<.icon name="hero-x-mark" />` component, never Heroicons modules - **Forms**: Always use `to_form/2` assigned in LiveView, access via `@form[:field]` in templates -- **Collections**: Always use LiveView streams, never assign raw lists - **JS in templates**: Use colocated hook `