Comprehensive CLAUDE.md update with full project architecture and setup
This commit is contained in:
parent
b11e0bac84
commit
8c45125e7c
1 changed files with 117 additions and 19 deletions
136
CLAUDE.md
136
CLAUDE.md
|
|
@ -4,9 +4,28 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
|
|||
|
||||
## Project Overview
|
||||
|
||||
Microwaveprop is a Phoenix 1.8 web application (Elixir ~> 1.15) with LiveView, PostgreSQL via Ecto, and Tailwind CSS v4 + daisyUI for styling. HTTP server is Bandit.
|
||||
Microwaveprop is a Phoenix 1.8 web application for the North Texas Microwave Society (NTMS) that predicts microwave radio propagation conditions (10-241 GHz). It helps amateur radio operators plan and coordinate microwave contacts by showing real-time and forecast propagation maps.
|
||||
|
||||
**All UI must be built with Phoenix LiveView.** Write idiomatic Elixir throughout. Phoenix uses esbuild for JS bundling — never run npm or use Node.js tooling.
|
||||
**Stack:** Elixir ~> 1.15, Phoenix 1.8 + LiveView, PostgreSQL via Ecto, Tailwind CSS v4 + daisyUI, Bandit HTTP server, Nx/Axon/EXLA for ML. esbuild for JS bundling — never npm.
|
||||
|
||||
### What It Does
|
||||
|
||||
1. **Propagation Map** (`/map`) — Full-screen Leaflet map with colored overlay showing propagation scores (0-100) per grid point across CONUS at 0.125° resolution. Includes 18-hour forecast timeline, band selector (10-241 GHz), Maidenhead grid overlay, point detail with factor breakdown and forecast sparkline, and terrain viewshed analysis.
|
||||
|
||||
2. **QSO Submission** (`/submit`) — Users submit microwave contacts. On submission, the system automatically enqueues weather/HRRR/terrain/IEMRE enrichment jobs for that QSO.
|
||||
|
||||
3. **Algorithm Documentation** (`/algo`) — Renders `algo.md` as HTML. Documents the scoring algorithm, meteorological foundations, ITU-R models, and calibration data.
|
||||
|
||||
4. **Hourly Scoring Pipeline** — Every hour, the PropagationGridWorker fetches HRRR forecast hours f00-f18, computes propagation scores across all bands, and pushes updates to connected clients via PubSub.
|
||||
|
||||
### Key Data Sources
|
||||
|
||||
- **HRRR** (High-Resolution Rapid Refresh) — 3 km NWP model from NOAA AWS S3. Surface + pressure level profiles. Analysis + 18-hour forecasts.
|
||||
- **ASOS** (Automated Surface Observing System) — Surface weather via Iowa Environmental Mesonet.
|
||||
- **RAOB** (Radiosonde) — Upper-air soundings via IEM.
|
||||
- **IEMRE** — Gridded hourly reanalysis at 0.125° resolution.
|
||||
- **SRTM** — 90m terrain elevation for path analysis (local tile cache or API fallback).
|
||||
- **Commercial Links** — 7 microwave links near DFW polled via SNMP at 5-min intervals.
|
||||
|
||||
## Commands
|
||||
|
||||
|
|
@ -22,7 +41,7 @@ iex -S mix phx.server # with IEx shell
|
|||
mix test
|
||||
|
||||
# Run a single test file
|
||||
mix test test/microwaveprop_web/controllers/page_controller_test.exs
|
||||
mix test test/microwaveprop/terrain/terrain_analysis_test.exs
|
||||
|
||||
# Run previously failed tests
|
||||
mix test --failed
|
||||
|
|
@ -34,7 +53,6 @@ mix format
|
|||
mix credo
|
||||
|
||||
# Pre-commit check (compile with warnings-as-errors, unlock unused deps, format, test)
|
||||
# Runs in test env
|
||||
mix precommit
|
||||
|
||||
# Database
|
||||
|
|
@ -43,6 +61,9 @@ mix ecto.migrate
|
|||
mix ecto.gen.migration migration_name_using_underscores
|
||||
mix ecto.reset # drop + setup
|
||||
|
||||
# Run propagation grid manually
|
||||
mix propagation_grid
|
||||
|
||||
# Assets
|
||||
mix assets.build
|
||||
mix assets.deploy # minified + digest
|
||||
|
|
@ -50,21 +71,88 @@ mix assets.deploy # minified + digest
|
|||
|
||||
## Architecture
|
||||
|
||||
Standard Phoenix 1.8 structure:
|
||||
### Directory Structure
|
||||
|
||||
- `lib/microwaveprop/` - Business logic (contexts, schemas, repo)
|
||||
- `repo.ex` - Ecto PostgreSQL repository
|
||||
- `application.ex` - OTP supervision tree
|
||||
- `lib/microwaveprop_web/` - Web layer
|
||||
- `router.ex` - Routes and pipelines. The `:browser` scope is aliased to `MicrowavepropWeb`
|
||||
- `components/core_components.ex` - UI component library (flash, button, input, table, list, icon)
|
||||
- `components/layouts.ex` - Layout components; `Layouts` is aliased in `microwaveprop_web.ex`
|
||||
- `controllers/` - Traditional controllers
|
||||
- `config/` - Environment configs (`dev.exs`, `test.exs`, `prod.exs`, `runtime.exs`)
|
||||
- `assets/` - JS (esbuild, not npm) and CSS (Tailwind v4). Only `app.js` and `app.css` bundles are supported
|
||||
- `test/support/` - `ConnCase` and `DataCase` test helpers
|
||||
- `lib/microwaveprop/` — Business logic (contexts, schemas, workers)
|
||||
- `propagation/` — Scoring engine: `scorer.ex`, `band_config.ex`, `grid.ex`, `model.ex` (ML), `freshness_monitor.ex`
|
||||
- `weather/` — Data ingestion: `hrrr_client.ex`, `iem_client.ex`, `sounding_params.ex`, GRIB2 decoder
|
||||
- `terrain/` — Path analysis: `terrain_analysis.ex` (ITU-R P.526-16), `viewshed.ex`, `elevation_client.ex`
|
||||
- `radio/` — QSO schema, Maidenhead grid conversion
|
||||
- `commercial/` — SNMP polling for commercial microwave links
|
||||
- `workers/` — Oban background jobs for all data pipelines
|
||||
- `lib/microwaveprop_web/` — Web layer
|
||||
- `live/map_live.ex` — Main propagation map with forecast timeline
|
||||
- `live/submit_live.ex` — QSO submission form
|
||||
- `live/algo_live.ex` — Algorithm documentation page
|
||||
- `live/qso_live/` — QSO list and detail views
|
||||
- `plugs/remote_ip.ex` — X-Forwarded-For client IP extraction
|
||||
- `config/` — Environment configs. `runtime.exs` has production Oban config (no backfill queues)
|
||||
- `assets/js/propagation_map_hook.js` — Leaflet map, canvas heatmap, timeline, grid overlay
|
||||
- `algo.md` — Full algorithm documentation with meteorological foundations
|
||||
- `priv/models/` — Saved ML model weights (gitignored)
|
||||
|
||||
## Key Conventions (from AGENTS.md)
|
||||
### Background Job Queues (Oban)
|
||||
|
||||
| Queue | Workers | Purpose |
|
||||
|---|---|---|
|
||||
| `propagation` | PropagationGridWorker | Hourly HRRR fetch (f00-f18) + score computation |
|
||||
| `weather` | WeatherFetchWorker | ASOS/RAOB data for QSO enrichment |
|
||||
| `hrrr` | HrrrFetchWorker | HRRR profiles for individual QSOs |
|
||||
| `terrain` | TerrainProfileWorker | ITU-R P.526-16 terrain diffraction analysis |
|
||||
| `iemre` | IemreFetchWorker | IEMRE gridded weather for QSO enrichment |
|
||||
| `commercial` | PollWorker | SNMP polling of commercial links |
|
||||
| `solar` | SolarIndexWorker | Daily solar indices |
|
||||
| `enqueue` | QsoWeatherEnqueueWorker | Batch enqueue enrichment jobs (dev cron only) |
|
||||
|
||||
**Production** runs: propagation, commercial, solar, weather, hrrr, terrain, iemre queues. No cron backfill — enrichment is triggered by QSO submission only.
|
||||
|
||||
**Dev** currently has propagation disabled for backfill. Re-enable by removing the Oban override in `config/dev.exs`.
|
||||
|
||||
### Scoring Algorithm
|
||||
|
||||
9-factor weighted composite score (0-100) per grid point per band:
|
||||
|
||||
| Factor | Weight | Source |
|
||||
|---|---|---|
|
||||
| Humidity | 20% | HRRR surface temp + dewpoint |
|
||||
| Time of day | 20% | UTC hour, cyclical |
|
||||
| T-Td depression | 12% | HRRR surface |
|
||||
| Refractivity | 10% | HRRR profile gradient (thresholds calibrated for HRRR resolution) |
|
||||
| Sky cover | 10% | HRRR cloud cover |
|
||||
| Season | 10% | Month, band-dependent (inverted for 24G+) |
|
||||
| Rain | 8% | HRRR precipitation, ITU-R P.838-3 |
|
||||
| Wind | 6% | HRRR 10m wind |
|
||||
| Pressure | 4% | HRRR surface pressure |
|
||||
|
||||
Humidity effect reverses by frequency: beneficial at 10 GHz (refractivity), harmful at 24+ GHz (absorption).
|
||||
|
||||
### Terrain Analysis (ITU-R P.526-16)
|
||||
|
||||
- Knife-edge diffraction loss (Eq. 31)
|
||||
- Deygout 3-edge method for multiple obstacles
|
||||
- Dynamic k-factor from HRRR refractivity gradient
|
||||
- Profiles stored per QSO path (58k+ paths analyzed)
|
||||
|
||||
### ML Model (Nx/Axon/EXLA)
|
||||
|
||||
Skeleton feed-forward network for future propagation prediction:
|
||||
- 13 features: 8 atmospheric + 4 cyclical temporal + 1 log-frequency
|
||||
- Architecture: Dense(64) → Dropout(0.2) → Dense(32) → Dropout(0.1) → Dense(1, sigmoid)
|
||||
- Weights saved to `priv/models/propagation_v1.nx`
|
||||
- Not yet trained — scaffolding only
|
||||
|
||||
### Data Flow
|
||||
|
||||
```
|
||||
HRRR (NOAA S3) → PropagationGridWorker (hourly, f00-f18)
|
||||
→ store hrrr_profiles → compute scores → upsert propagation_scores
|
||||
→ PubSub broadcast → MapLive → JS hook → canvas heatmap + timeline
|
||||
|
||||
User submits QSO → enqueue_for_qso() → weather/hrrr/terrain/iemre workers
|
||||
→ enrich QSO with atmospheric + terrain data
|
||||
```
|
||||
|
||||
## Key Conventions
|
||||
|
||||
### Phoenix / LiveView
|
||||
- LiveView templates must start with `<Layouts.app flash={@flash} ...>` wrapping all content
|
||||
|
|
@ -88,7 +176,7 @@ Standard Phoenix 1.8 structure:
|
|||
- Assets are bundled via esbuild and Tailwind mix tasks — never use npm, npx, or Node.js directly
|
||||
- Tailwind v4: no `tailwind.config.js`, uses `@import "tailwindcss" source(none)` syntax in `app.css`
|
||||
- Never use `@apply` in CSS
|
||||
- No inline `<script>` tags; use colocated JS hooks (`:type={Phoenix.LiveView.ColocatedHook}`) with `.` prefix names
|
||||
- No inline `<script>` tags; use colocated JS hooks with `.` prefix names
|
||||
- Vendor deps must be imported into `app.js`/`app.css`, no external script `src` or link `href`
|
||||
|
||||
### Elixir
|
||||
|
|
@ -102,15 +190,25 @@ Standard Phoenix 1.8 structure:
|
|||
- Use `Req` for HTTP requests, never httpoison/tesla/httpc
|
||||
|
||||
### Ecto
|
||||
- All schemas use `@primary_key {:id, :binary_id, autogenerate: true}`
|
||||
- Preload associations in queries when accessed in templates
|
||||
- `field :name, :string` for both string and text columns
|
||||
- Use `Ecto.Changeset.get_field/2` to access changeset fields
|
||||
- Programmatic fields (e.g. `user_id`) must not be in `cast` calls
|
||||
- Generate migrations with `mix ecto.gen.migration`
|
||||
- Upserts via `on_conflict: {:replace_all_except, [:id, ...]}` with `conflict_target`
|
||||
|
||||
### Testing
|
||||
- Use `start_supervised!/1` for process cleanup
|
||||
- Use `Process.monitor/1` + `assert_receive {:DOWN, ...}` instead of `Process.sleep`
|
||||
- Use `LazyHTML` selectors for DOM assertions, never raw HTML matching
|
||||
- Test against element IDs defined in templates
|
||||
- Debug with `LazyHTML.filter/2` + `IO.inspect` instead of raw HTML dumps
|
||||
- Stub HTTP calls with `Req.Test.stub` in test setup
|
||||
- Oban runs inline in test (`testing: :inline`)
|
||||
- FreshnessMonitor disabled in test via `config :microwaveprop, start_freshness_monitor: false`
|
||||
|
||||
### Deployment
|
||||
- Production runs on Dokku with nginx proxy
|
||||
- Push to `github` remote for code. Push to `dokku` remote only when explicitly asked — it triggers production deploy.
|
||||
- Production Oban config in `config/runtime.exs` — no backfill cron, enrichment triggered by QSO submission
|
||||
- SRTM tiles at `/srtm` in production container
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue