From fe1c10ce0cbc7cfccc757565dbef62ca78a5ac72 Mon Sep 17 00:00:00 2001 From: Graham McIntire Date: Wed, 29 Apr 2026 17:13:08 -0500 Subject: [PATCH] docs(hrdps): record stage progress + dormant scaffolding state CLAUDE.md adds HRDPS to Key Data Sources and the Background Job Queues table with explicit "dormant until Rust ships" framing. The 2026-04-29 plan gets a Status snapshot at the top: which stages shipped (0-3 Elixir + 8) vs which are deferred to the Rust port sessions (4-7, 9). Activation path is one-paragraph: land Rust HRDPS, add cron entry, widen map bounds, extend FreshnessMonitor. --- CLAUDE.md | 2 ++ .../2026-04-29-hrdps-canadian-prop-grid.md | 29 +++++++++++++++++++ 2 files changed, 31 insertions(+) diff --git a/CLAUDE.md b/CLAUDE.md index 9503fee5..e0ca2d02 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -21,6 +21,7 @@ Microwaveprop is a Phoenix 1.8 web application for the North Texas Microwave Soc ### Key Data Sources - **HRRR** (High-Resolution Rapid Refresh) — 3 km NWP model from NOAA AWS S3. Surface + pressure level profiles. Analysis + 18-hour forecasts. +- **HRDPS** (High Resolution Deterministic Prediction System) — 2.5 km Canadian NWP model from MSC Datamart. 4×/day at 00/06/12/18Z, 48h forecasts. Coverage capped at 60°N for v1 (SRTM stops there). `HrdpsClient` + scaffolding shipped 2026-04-29; full pipeline activates once the Rust `prop-grid-rs` HRDPS branch ships. Plan: `docs/plans/2026-04-29-hrdps-canadian-prop-grid.md`. - **ASOS** (Automated Surface Observing System) — Surface weather via Iowa Environmental Mesonet. - **RAOB** (Radiosonde) — Upper-air soundings via IEM. - **IEMRE** — Gridded hourly reanalysis at 0.125° resolution. @@ -103,6 +104,7 @@ mix assets.deploy # minified + digest | `commercial` | PollWorker | SNMP polling of commercial links | | `solar` | SolarIndexWorker | Daily solar indices | | `enqueue` | QsoWeatherEnqueueWorker | Batch enqueue enrichment jobs (dev cron only) | +| `hrdps` | HrdpsGridWorker | Canadian propagation chain seed (4×/day). Dormant until Rust `prop-grid-rs` HRDPS branch ships — module exists, cron entry deferred. | **Production** runs: propagation, commercial, solar, weather, hrrr, terrain, iemre queues. No cron backfill — enrichment is triggered by QSO submission only. diff --git a/docs/plans/2026-04-29-hrdps-canadian-prop-grid.md b/docs/plans/2026-04-29-hrdps-canadian-prop-grid.md index 1e46fde0..89efed29 100644 --- a/docs/plans/2026-04-29-hrdps-canadian-prop-grid.md +++ b/docs/plans/2026-04-29-hrdps-canadian-prop-grid.md @@ -14,6 +14,35 @@ --- +## Status snapshot (2026-04-29) + +**Shipped — Elixir foundation complete:** + +- ✅ Stage 0: Probe wedge (`mix hrdps.probe`) +- ✅ Stage 1: `HrdpsClient` with URL/variable catalog, concurrent fetch + byte-concat, `fetch_grid/3`, `cycle_available?/1` — `lib/microwaveprop/weather/hrdps_client.ex` +- ✅ Stage 2: `hrdps_profiles` partitioned table + `HrdpsProfile` schema +- ✅ Stage 3 (Elixir-side): `Grid.hrdps_only_points/0`, `Grid.point_source/1`, `grid_tasks.source` column, `GridTaskEnqueuer` source-aware seeding, `HrdpsGridWorker` seeder +- ✅ Stage 8: SRTM scope decision recorded — Option 1 (cap Canadian coverage at 60°N for v1; defer Arctic CDEM to a follow-up plan) + +**Deferred — needs the Rust counterpart before it can ship:** + +- ⏸ Stage 4: `prop-grid-rs` Rust HRDPS branch (~4 days). Until this ships, `HrdpsGridWorker` stays out of the cron — its module doc explicitly notes it is dormant scaffolding. +- ⏸ Stage 5: `hrrr-point-rs` Rust HRDPS branch (~1.5 days) +- ⏸ Stage 6: cycle scheduling cron entry — depends on Stage 4 +- ⏸ Stage 7: map overlay bounds widening — depends on real data in `/data/scores` +- ⏸ Stage 9: FreshnessMonitor HRDPS staleness tracking — depends on real cycle data flowing + +**Activation path once Rust ships:** + +1. Land the Rust HRDPS branch + redeploy `prop-grid-rs` and `hrrr-point-rs`. +2. Add to `config/runtime.exs` cron: `{"35 5,11,17,23 * * *", Microwaveprop.Workers.HrdpsGridWorker}`. +3. Widen `MapLive.@initial_bounds` to `(23.5, 60.0, -141.0, -52.0)`. +4. Extend `FreshnessMonitor` with `hrdps_last_run_at`. + +The hardest decisions (cycle cadence, border stitching, variable mapping, projection) are settled and encoded in code. The remaining work is mechanical port-and-wire-up. + +--- + ## Stage 0: Probe wedge (DONE) `mix hrdps.probe` proves the GRIB2 fetch + wgrib2 reprojection works end-to-end against the real ECCC datamart. Output verifies plausible Canadian temperatures at 5 cities. Throwaway code; will be deleted in Stage 10 once `HrdpsClient` covers the same surface area.