216 lines
10 KiB
Markdown
216 lines
10 KiB
Markdown
# CLAUDE.md
|
|
|
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
|
|
## Project Overview
|
|
|
|
Microwaveprop is a Phoenix 1.8 web application for the North Texas Microwave Society (NTMS) that predicts microwave radio propagation conditions (10-241 GHz). It helps amateur radio operators plan and coordinate microwave contacts by showing real-time and forecast propagation maps.
|
|
|
|
**Stack:** Elixir ~> 1.15, Phoenix 1.8 + LiveView, PostgreSQL via Ecto, Tailwind CSS v4 + daisyUI, Bandit HTTP server, Nx/Axon/EXLA for ML. esbuild for JS bundling — never npm.
|
|
|
|
### What It Does
|
|
|
|
1. **Propagation Map** (`/map`) — Full-screen Leaflet map with colored overlay showing propagation scores (0-100) per grid point across CONUS at 0.125° resolution. Includes 18-hour forecast timeline, band selector (10-241 GHz), Maidenhead grid overlay, point detail with factor breakdown and forecast sparkline, and terrain viewshed analysis.
|
|
|
|
2. **QSO Submission** (`/submit`) — Users submit microwave contacts. On submission, the system automatically enqueues weather/HRRR/terrain/IEMRE enrichment jobs for that QSO.
|
|
|
|
3. **Algorithm Documentation** (`/algo`) — Renders `algo.md` as HTML. Documents the scoring algorithm, meteorological foundations, ITU-R models, and calibration data.
|
|
|
|
4. **Hourly Scoring Pipeline** — Every hour, the PropagationGridWorker fetches HRRR forecast hours f00-f18, computes propagation scores across all bands, and pushes updates to connected clients via PubSub.
|
|
|
|
### Key Data Sources
|
|
|
|
- **HRRR** (High-Resolution Rapid Refresh) — 3 km NWP model from NOAA AWS S3. Surface + pressure level profiles. Analysis + 18-hour forecasts.
|
|
- **ASOS** (Automated Surface Observing System) — Surface weather via Iowa Environmental Mesonet.
|
|
- **RAOB** (Radiosonde) — Upper-air soundings via IEM.
|
|
- **IEMRE** — Gridded hourly reanalysis at 0.125° resolution.
|
|
- **SRTM** — 90m terrain elevation for path analysis (local tile cache or API fallback).
|
|
- **Commercial Links** — 7 microwave links near DFW polled via SNMP at 5-min intervals.
|
|
|
|
## Commands
|
|
|
|
```bash
|
|
# Initial setup (deps, DB, assets)
|
|
mix setup
|
|
|
|
# Run dev server (live reloads automatically, no restart needed)
|
|
mix phx.server
|
|
iex -S mix phx.server # with IEx shell
|
|
|
|
# Run all tests
|
|
mix test
|
|
|
|
# Run a single test file
|
|
mix test test/microwaveprop/terrain/terrain_analysis_test.exs
|
|
|
|
# Run previously failed tests
|
|
mix test --failed
|
|
|
|
# Format code (runs Styler plugin automatically)
|
|
mix format
|
|
|
|
# Static analysis
|
|
mix credo
|
|
|
|
# Pre-commit check (compile with warnings-as-errors, unlock unused deps, format, test)
|
|
mix precommit
|
|
|
|
# Database
|
|
mix ecto.create
|
|
mix ecto.migrate
|
|
mix ecto.gen.migration migration_name_using_underscores
|
|
mix ecto.reset # drop + setup
|
|
|
|
# Run propagation grid manually
|
|
mix propagation_grid
|
|
|
|
# Assets
|
|
mix assets.build
|
|
mix assets.deploy # minified + digest
|
|
```
|
|
|
|
## Architecture
|
|
|
|
### Directory Structure
|
|
|
|
- `lib/microwaveprop/` — Business logic (contexts, schemas, workers)
|
|
- `propagation/` — Scoring engine: `scorer.ex`, `band_config.ex`, `grid.ex`, `model.ex` (ML), `freshness_monitor.ex`
|
|
- `weather/` — Data ingestion: `hrrr_client.ex`, `iem_client.ex`, `sounding_params.ex`, GRIB2 decoder
|
|
- `terrain/` — Path analysis: `terrain_analysis.ex` (ITU-R P.526-16), `viewshed.ex`, `elevation_client.ex`
|
|
- `radio/` — QSO schema, Maidenhead grid conversion
|
|
- `commercial/` — SNMP polling for commercial microwave links
|
|
- `workers/` — Oban background jobs for all data pipelines
|
|
- `lib/microwaveprop_web/` — Web layer
|
|
- `live/map_live.ex` — Main propagation map with forecast timeline
|
|
- `live/submit_live.ex` — QSO submission form
|
|
- `live/algo_live.ex` — Algorithm documentation page
|
|
- `live/qso_live/` — QSO list and detail views
|
|
- `plugs/remote_ip.ex` — X-Forwarded-For client IP extraction
|
|
- `config/` — Environment configs. `runtime.exs` has production Oban config (no backfill queues)
|
|
- `assets/js/propagation_map_hook.js` — Leaflet map, canvas heatmap, timeline, grid overlay
|
|
- `algo.md` — Full algorithm documentation with meteorological foundations
|
|
- `priv/models/` — Saved ML model weights (gitignored)
|
|
|
|
### Background Job Queues (Oban)
|
|
|
|
| Queue | Workers | Purpose |
|
|
|---|---|---|
|
|
| `propagation` | PropagationGridWorker | Hourly HRRR fetch (f00-f18) + score computation |
|
|
| `weather` | WeatherFetchWorker | ASOS/RAOB data for QSO enrichment |
|
|
| `hrrr` | HrrrFetchWorker | HRRR profiles for individual QSOs |
|
|
| `terrain` | TerrainProfileWorker | ITU-R P.526-16 terrain diffraction analysis |
|
|
| `iemre` | IemreFetchWorker | IEMRE gridded weather for QSO enrichment |
|
|
| `commercial` | PollWorker | SNMP polling of commercial links |
|
|
| `solar` | SolarIndexWorker | Daily solar indices |
|
|
| `enqueue` | QsoWeatherEnqueueWorker | Batch enqueue enrichment jobs (dev cron only) |
|
|
|
|
**Production** runs: propagation, commercial, solar, weather, hrrr, terrain, iemre queues. No cron backfill — enrichment is triggered by QSO submission only.
|
|
|
|
**Dev** runs the same hourly `PropagationGridWorker` + `AsosAdjustmentWorker` (10-minute nudge) cron as production.
|
|
|
|
### Scoring Algorithm
|
|
|
|
10-factor weighted composite score (0-100) per grid point per band (recalibrated 2026-04-11 via gradient descent):
|
|
|
|
| Factor | Weight | Source |
|
|
|---|---|---|
|
|
| Rain | 13.6% | HRRR precipitation, ITU-R P.838-3 |
|
|
| Humidity | 12.4% | HRRR surface temp + dewpoint |
|
|
| PWAT | 11.3% | HRRR precipitable water (column-integrated) |
|
|
| Season | 11.1% | Month, band-dependent (inverted for 24G+) |
|
|
| Refractivity | 10.5% | Native HRRR dM/dh (10-50m), fallback to pressure-level (250m) |
|
|
| Pressure | 10.3% | HRRR surface pressure (frontal activity proxy) |
|
|
| T-Td depression | 9.8% | HRRR surface |
|
|
| Sky cover | 8.0% | HRRR cloud cover |
|
|
| Wind | 8.0% | HRRR 10m wind |
|
|
| Time of day | 5.0% | Solar-time adjusted diurnal cycle |
|
|
|
|
Humidity effect reverses by frequency: beneficial at 10 GHz (refractivity), harmful at 24+ GHz (absorption).
|
|
|
|
### Terrain Analysis (ITU-R P.526-16)
|
|
|
|
- Knife-edge diffraction loss (Eq. 31)
|
|
- Deygout 3-edge method for multiple obstacles
|
|
- Dynamic k-factor from HRRR refractivity gradient
|
|
- Profiles stored per QSO path (58k+ paths analyzed)
|
|
|
|
### ML Model (Nx/Axon/EXLA)
|
|
|
|
Skeleton feed-forward network for future propagation prediction:
|
|
- 13 features: 8 atmospheric + 4 cyclical temporal + 1 log-frequency
|
|
- Architecture: Dense(64) → Dropout(0.2) → Dense(32) → Dropout(0.1) → Dense(1, sigmoid)
|
|
- Weights saved to `priv/models/propagation_v1.nx`
|
|
- Not yet trained — scaffolding only
|
|
|
|
### Data Flow
|
|
|
|
```
|
|
HRRR (NOAA S3) → PropagationGridWorker (hourly, f00-f18)
|
|
→ store hrrr_profiles → compute scores → upsert propagation_scores
|
|
→ PubSub broadcast → MapLive → JS hook → canvas heatmap + timeline
|
|
|
|
User submits QSO → enqueue_for_qso() → weather/hrrr/terrain/iemre workers
|
|
→ enrich QSO with atmospheric + terrain data
|
|
```
|
|
|
|
## Key Conventions
|
|
|
|
### Phoenix / LiveView
|
|
- LiveView templates must start with `<Layouts.app flash={@flash} ...>` wrapping all content
|
|
- Use `<.icon name="hero-x-mark" class="w-5 h-5"/>` for icons, never Heroicons modules
|
|
- Use `<.input>` from core_components for form inputs
|
|
- Use `to_form/2` for forms, never pass changesets directly to templates
|
|
- Use LiveView streams for collections, never `phx-update="append"`/`"prepend"`
|
|
- Avoid LiveComponents unless strongly justified
|
|
- LiveView names use `Live` suffix: `MicrowavepropWeb.ThingLive`
|
|
- Use `<.link navigate={...}>` / `<.link patch={...}>`, never `live_redirect`/`live_patch`
|
|
- Router scope aliases prefix automatically; don't add redundant aliases
|
|
|
|
### HEEx Templates
|
|
- Use `{...}` for attribute interpolation, `<%= %>` only for block constructs (if/for/cond) in tag bodies
|
|
- Class lists must use `[...]` syntax for conditional classes
|
|
- Use `<%!-- comment --%>` for HTML comments
|
|
- Use `phx-no-curly-interpolation` on tags containing literal curly braces
|
|
- Never use `<% Enum.each %>`, always use `<%= for item <- @collection do %>`
|
|
|
|
### JS / CSS
|
|
- Assets are bundled via esbuild and Tailwind mix tasks — never use npm, npx, or Node.js directly
|
|
- Tailwind v4: no `tailwind.config.js`, uses `@import "tailwindcss" source(none)` syntax in `app.css`
|
|
- Never use `@apply` in CSS
|
|
- No inline `<script>` tags; use colocated JS hooks with `.` prefix names
|
|
- Vendor deps must be imported into `app.js`/`app.css`, no external script `src` or link `href`
|
|
|
|
### Elixir
|
|
- Write idiomatic Elixir: use pattern matching, pipe operator, and the standard library
|
|
- Styler auto-formats on `mix format` (alias sorting, pipe chains, moduledoc enforcement, etc.)
|
|
- No index access on lists (`mylist[i]`); use `Enum.at/2`
|
|
- Bind results of `if`/`case`/`cond` blocks to variables (immutable rebinding)
|
|
- Never nest multiple modules in one file
|
|
- Use `struct.field` not `struct[:field]` (structs don't implement Access)
|
|
- Predicate functions: `thing?` not `is_thing` (reserve `is_` for guards)
|
|
- Use `Req` for HTTP requests, never httpoison/tesla/httpc
|
|
|
|
### Ecto
|
|
- All schemas use `@primary_key {:id, :binary_id, autogenerate: true}`
|
|
- Preload associations in queries when accessed in templates
|
|
- `field :name, :string` for both string and text columns
|
|
- Use `Ecto.Changeset.get_field/2` to access changeset fields
|
|
- Programmatic fields (e.g. `user_id`) must not be in `cast` calls
|
|
- Generate migrations with `mix ecto.gen.migration`
|
|
- Upserts via `on_conflict: {:replace_all_except, [:id, ...]}` with `conflict_target`
|
|
|
|
### Testing
|
|
- Use `start_supervised!/1` for process cleanup
|
|
- Use `Process.monitor/1` + `assert_receive {:DOWN, ...}` instead of `Process.sleep`
|
|
- Use `LazyHTML` selectors for DOM assertions, never raw HTML matching
|
|
- Test against element IDs defined in templates
|
|
- Stub HTTP calls with `Req.Test.stub` in test setup
|
|
- Oban runs inline in test (`testing: :inline`)
|
|
- FreshnessMonitor disabled in test via `config :microwaveprop, start_freshness_monitor: false`
|
|
|
|
### Deployment
|
|
- Production runs on Kubernetes (`home-cluster` context, `prop` namespace) — use `kubectl -n prop ...`
|
|
- Deployment name `prop` (3 replicas). Inspect with `kubectl -n prop get pods`, `kubectl -n prop logs deploy/prop`, `kubectl -n prop exec deploy/prop -- ...`
|
|
- Push to `github` remote for code. The `dokku` remote is obsolete — do not push there.
|
|
- Production Oban config in `config/runtime.exs` — no backfill cron, enrichment triggered by QSO submission
|
|
- SRTM tiles at `/srtm` in production container
|