prop/docs/plans/2026-04-29-hrdps-canadian-prop-grid.md
Graham McIntire fe1c10ce0c
docs(hrdps): record stage progress + dormant scaffolding state
CLAUDE.md adds HRDPS to Key Data Sources and the Background Job
Queues table with explicit "dormant until Rust ships" framing.

The 2026-04-29 plan gets a Status snapshot at the top: which stages
shipped (0-3 Elixir + 8) vs which are deferred to the Rust port
sessions (4-7, 9). Activation path is one-paragraph: land Rust HRDPS,
add cron entry, widen map bounds, extend FreshnessMonitor.
2026-04-29 17:13:08 -05:00

411 lines
24 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# HRDPS Canadian Propagation Grid Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Supersedes:** `docs/plans/2026-04-13-hrdps-canadian-prop-grid.md` — that plan was written before the probe wedge surfaced two architectural realities that change Stage 1's design: ECCC moved the datamart URL structure (the old `/model_hrdps/` root 404s now), and HRDPS publishes one variable per GRIB2 file rather than HRRR's bundled wrfsfcf/wrfprsf format. Major filenames, fetch shape, and idx assumptions are different.
**Goal:** Extend the propagation scoring grid into Canada at 0.125° (matching the existing CONUS sample density) sourced from HRDPS, so `/map` shows scores across all of North America. Hourly Canadian scores via the cycle's f00..f18 forecast hours; HRRR wins where it has coverage and HRDPS fills the rest.
**Architecture:** Build a parallel NWP ingestion pipeline (`HrdpsClient``HrdpsGridWorker``hrdps_profiles` table) that mirrors the HRRR shape. The single-variable-per-file difference is hidden inside `HrdpsClient`: it fetches the ~14 needed variable files concurrently, then runs `wgrib2 -merge` to produce a single multi-variable GRIB2 that downstream code can consume identically to a `wrfsfcf`/`wrfprsf` blob. HRDPS runs at 00/06/12/18Z and seeds f00..f18 (hourly) into the score chain, so Canadian scores refresh once per cycle but provide hourly forecast steps until the next cycle lands. Border stitching: a static `Grid.hrdps_only_points/0` mask carves Canada-only cells so HRRR and HRDPS never both write the same `(lat, lon)`.
**Tech Stack:** Elixir 1.15, Req, Postgres + Ecto, Oban, wgrib2 (already in image), Rust (for `prop-grid-rs` and `hrrr-point-rs` HRDPS branches)
**Prerequisite:** Stage 0 already done — the probe at `lib/mix/tasks/hrdps_probe.ex` retired the URL/projection/decode risks against the live datamart on 2026-04-29.
---
## Status snapshot (2026-04-29)
**Shipped — Elixir foundation complete:**
- ✅ Stage 0: Probe wedge (`mix hrdps.probe`)
- ✅ Stage 1: `HrdpsClient` with URL/variable catalog, concurrent fetch + byte-concat, `fetch_grid/3`, `cycle_available?/1``lib/microwaveprop/weather/hrdps_client.ex`
- ✅ Stage 2: `hrdps_profiles` partitioned table + `HrdpsProfile` schema
- ✅ Stage 3 (Elixir-side): `Grid.hrdps_only_points/0`, `Grid.point_source/1`, `grid_tasks.source` column, `GridTaskEnqueuer` source-aware seeding, `HrdpsGridWorker` seeder
- ✅ Stage 8: SRTM scope decision recorded — Option 1 (cap Canadian coverage at 60°N for v1; defer Arctic CDEM to a follow-up plan)
**Deferred — needs the Rust counterpart before it can ship:**
- ⏸ Stage 4: `prop-grid-rs` Rust HRDPS branch (~4 days). Until this ships, `HrdpsGridWorker` stays out of the cron — its module doc explicitly notes it is dormant scaffolding.
- ⏸ Stage 5: `hrrr-point-rs` Rust HRDPS branch (~1.5 days)
- ⏸ Stage 6: cycle scheduling cron entry — depends on Stage 4
- ⏸ Stage 7: map overlay bounds widening — depends on real data in `/data/scores`
- ⏸ Stage 9: FreshnessMonitor HRDPS staleness tracking — depends on real cycle data flowing
**Activation path once Rust ships:**
1. Land the Rust HRDPS branch + redeploy `prop-grid-rs` and `hrrr-point-rs`.
2. Add to `config/runtime.exs` cron: `{"35 5,11,17,23 * * *", Microwaveprop.Workers.HrdpsGridWorker}`.
3. Widen `MapLive.@initial_bounds` to `(23.5, 60.0, -141.0, -52.0)`.
4. Extend `FreshnessMonitor` with `hrdps_last_run_at`.
The hardest decisions (cycle cadence, border stitching, variable mapping, projection) are settled and encoded in code. The remaining work is mechanical port-and-wire-up.
---
## Stage 0: Probe wedge (DONE)
`mix hrdps.probe` proves the GRIB2 fetch + wgrib2 reprojection works end-to-end against the real ECCC datamart. Output verifies plausible Canadian temperatures at 5 cities. Throwaway code; will be deleted in Stage 10 once `HrdpsClient` covers the same surface area.
Findings that drive the rest of the plan:
- **URL:** `https://dd.weather.gc.ca/{YYYYMMDD}/WXO-DD/model_hrdps/continental/2.5km/{HH}/{FFF}/{YYYYMMDD}T{HH}Z_MSC_HRDPS_{VAR}_{LEVEL}_RLatLon0.0225_PT{FFF}H.grib2`
- **Variable naming:** MSC's `{VAR}_{LEVEL_TYPE}-{LEVEL_VALUE}` (e.g. `TMP_AGL-2m`, `TMP_ISBL_0850`) — different from NCEP's HRRR short-name format
- **One variable per file** (~3.5 MB each); no idx file / no byte-range partials
- **wgrib2 reprojects rotated lat/lon natively** via `-lon LON LAT` — no manual rotation math
- **Publish latency:** ~3-4h after each cycle (verified by cycle availability at probe time)
---
## Stage 1: `HrdpsClient` — per-variable fetch + wgrib2 -merge
**Decision recorded:** Fetch each variable's GRIB2 file concurrently, then run `wgrib2 -merge` to combine into a single multi-variable file before downstream decode. Keeps the existing `hrrr_profiles`-shaped derive pipeline intact downstream of the client. The disk-staging cost (~14 files × 3.5 MB into a tmp dir per cycle/forecast hour) is small compared to the fetch time.
**Files:**
- Create: `lib/microwaveprop/weather/hrdps_client.ex`
- Create: `test/microwaveprop/weather/hrdps_client_test.exs`
- Create: `test/support/fixtures/hrdps_sample_*.grib2` (small fixtures captured from a real cycle)
### Task 1.1: URL builder (TDD)
Write a failing test for `HrdpsClient.grib_url(:tmp_2m, run_time, forecast_hour)` returning the date-prefixed datamart URL with the MSC filename format. Verify against a known-good URL from the probe (`20260429T12Z_MSC_HRDPS_TMP_AGL-2m_RLatLon0.0225_PT000H.grib2`). Implement; commit.
### Task 1.2: Variable catalog
Define a `@variables` map mapping internal atoms (`:tmp_2m`, `:dpt_2m`, `:pres_sfc`, `:wind_u_10m`, `:wind_v_10m`, `:apcp_1h`, `:tcdc`, `:pwat`, `:tmp_850mb`, `:tmp_700mb`, `:tmp_500mb`, `:dpt_850mb`, `:dpt_700mb`, `:hgt_850mb`) to MSC variable + level slugs (`{VAR_AGL-2m}`, `{VAR_ISBL_0850}`, etc.).
This is the variable-mapping table the old plan called out as a Stage-1 risk; build it explicitly. TDD: a `variable_slug/1` test for each atom. Commit.
### Task 1.3: Concurrent fetch + tmp staging (TDD)
Write a failing test for `fetch_cycle_files(run_time, forecast_hour, opts)` that returns `{:ok, [paths]}` of the per-variable GRIB2 files staged in a tmp dir. Use `Task.async_stream` with `max_concurrency: 4` (datamart isn't aggressively rate-limited but we don't need >4 in flight). Mock with `Req.Test.stub` per project conventions.
Critical: every `{:exit, _}` and `{:error, _}` clause from the async_stream must `Logger.error` with variable name + URL — per CLAUDE.md "always log anything that would normally be swallowed" and the path-calculator outage memory. A silently-dropped fetch in production is a Canadian dead-zone we won't notice for hours.
Implement; commit.
### Task 1.4: wgrib2 -merge into multi-variable file (TDD)
Write a failing test for `merge_to_multivar(paths, output_path)`. Implement via `System.cmd("wgrib2", ["-merge", path1, path2, ..., "-grib", output_path])`. Verify the merged output's `wgrib2 file -V` lists all expected variables.
Commit.
### Task 1.5: `fetch_grid/3` — public API matching `HrrrClient.fetch_grid/3`
Same signature as HRRR so Stage 3's worker can call either polymorphically:
```elixir
@spec fetch_grid([{float(), float()}], DateTime.t(), keyword()) ::
{:ok, %{{float(), float()} => map()}} | {:error, term()}
def fetch_grid(points, run_time, opts \\ [])
```
Internally: `fetch_cycle_files``merge_to_multivar` → reuse `Microwaveprop.Weather.GribDecoder` (the existing HRRR decoder, projection-agnostic since wgrib2 handles rotation).
TDD; commit.
### Task 1.6: Projection sanity check (TDD)
Pick CYYR (Goose Bay, 53.32°N, -60.42°W) and a recent cycle for which we have a UWYO Canadian sounding (those exist via `CanadianSoundingFetchWorker`). Test that the HRDPS surface temp at CYYR is within ±3°C of the radiosonde surface temp for the same `valid_time`. If they don't match, the projection is wrong and `wgrib2 -lon` is being called incorrectly.
This is the most important test in the stage — burn whatever time is needed to make it pass. A 10 km projection offset is invisible on the map but destroys terrain analysis at Stage 5+.
Commit.
---
## Stage 2: `hrdps_profiles` schema
**Files:**
- Create: `priv/repo/migrations/{timestamp}_create_hrdps_profiles.exs`
- Create: `lib/microwaveprop/weather/hrdps_profile.ex`
- Create: `test/microwaveprop/weather/hrdps_profile_test.exs`
### Task 2.1: Migration mirrors `hrrr_profiles`
Same columns, same indexes, daily partitioning on `valid_time`. Separate table (not a unified `nwp_profiles`) so HRDPS can be dropped/reprocessed without disturbing the 4500+ HRRR profiles already in prod.
`@primary_key {:id, :binary_id, autogenerate: true}` per project convention.
Run migration in dev; verify table + indexes; commit.
### Task 2.2: `HrdpsProfile` schema + changeset (TDD)
Identical to `HrrrProfile`. Same derived fields (`surface_refractivity`, `min_refractivity_gradient`, `ducting_detected`, `duct_characteristics`, `pwat_mm`, `hpbl_m`).
Commit.
### Task 2.3: Partition pruning cron
Add an `Oban.Plugins.Cron` entry mirroring whatever HRRR uses (or add one if HRRR doesn't have one yet — the prod DB will fill up either way).
Commit.
---
## Stage 3: `HrdpsGridWorker` — score Canadian grid per forecast hour
**Files:**
- Create: `lib/microwaveprop/workers/hrdps_grid_worker.ex`
- Create: `test/microwaveprop/workers/hrdps_grid_worker_test.exs`
- Modify: `lib/microwaveprop/propagation/grid.ex` — add `hrdps_only_points/0`
### Task 3.1: Define the Canadian-only grid (TDD)
`hrdps_only_points/0` returns the subset of a 0.125°-spaced grid that is INSIDE the Canadian land mask AND OUTSIDE the existing CONUS HRRR footprint. Implement as a precomputed list (compile-time `@external_resource`) — the cells don't change.
Approximate bounds: 42°N to 60°N (north-of-60°N deferred to Stage 8), -141°W to -52°W. Filter to land using a coarse polygon — Natural Earth's `ne_50m_admin_0_countries` clipped to Canada is fine; even a rectangular bounding-box pass-through is acceptable for a first pass since over-water cells just get scored and ignored.
The mask must NEVER overlap HRRR cells. Test: `MapSet.disjoint?(MapSet.new(Grid.conus_points()), MapSet.new(Grid.hrdps_only_points()))`.
Commit.
### Task 3.2: Worker `perform/1` (TDD)
Mirror `PropagationGridWorker.process_forecast_hour/4`. Args: `{run_time, forecast_hour}`. Calls `HrdpsClient.fetch_grid(hrdps_only_points(), run_time, opts)`, derives profiles via the existing `Microwaveprop.Weather.SoundingParams`-style derivation, persists to `hrdps_profiles`, then scores via `Microwaveprop.Propagation.Scorer.score/3`.
`start_async`/`Task.async_stream` failure paths must `Logger.error` with point + reason (per CLAUDE.md and the swallowed-async-errors memory).
Commit.
### Task 3.3: Score upserts into existing `propagation_scores`
Single shared scores table — HRDPS rows look identical to HRRR rows, distinguished only by `(lat, lon)` falling in the HRDPS-only mask. Use the same `on_conflict: {:replace_all_except, [...]}` upsert pattern.
Confirm via SQL: a sample run produces no `(lat, lon, valid_time, band_mhz)` collisions with HRRR rows.
Commit.
### Task 3.4: Score-file `.prop` writer
The `.prop` file at `/data/scores/...` is what `MicrowavepropWeb.ScoresController.cells/2` reads (and what `ScoreCache` mirrors in ETS). Either:
(a) extend the `.prop` writer to handle the larger union grid (CONUS + Canada), or
(b) emit a separate `_canada.prop` file that the controller stitches at read time.
Decision: **(a)** — single file, simpler downstream. The Rust scorer (Stage 4) writes the same file; both must agree on the cell layout. Update `Microwaveprop.Propagation.ScoresFile` to support a wider grid extent. The cell-pack format already handles arbitrary lat/lon arrays per the `PSCR` magic spec.
TDD: write a fixture `.prop` file at the expanded extent, read it back, verify all cells round-trip. Commit.
---
## Stage 4: `prop-grid-rs` Rust worker — HRDPS branch
The Rust worker at `rust/prop_grid_rs/` claims tasks from `grid_tasks` via `FOR UPDATE SKIP LOCKED` and writes `.prop` files to `/data/scores`. It needs an HRDPS branch since it owns the hot path for f01..f18.
**Files:**
- Modify: `rust/prop_grid_rs/src/sources/` (likely add `hrdps.rs` next to `hrrr.rs`)
- Modify: `rust/prop_grid_rs/src/main.rs` — task source dispatch
- Migration: `priv/repo/migrations/{timestamp}_add_source_to_grid_tasks.exs`
### Task 4.1: Add `source` column to `grid_tasks`
`source TEXT NOT NULL DEFAULT 'hrrr'`. Backfill is a no-op since existing rows are all HRRR. Index on `(source, run_time)` for the worker's claim query.
Commit.
### Task 4.2: Rust HRDPS fetcher
Mirror the existing `hrrr.rs` shape. Differences:
- URL builder using the date-prefixed pattern + per-variable filenames
- Concurrent fetch of ~14 variable files (use `tokio::join_all` with a semaphore at 4)
- Shell out to `wgrib2 -merge` (same binary the Elixir client uses; no Rust GRIB2 reimplementation)
- Reuse the existing point-extraction code that wraps `wgrib2 -lon`
Keep the Rust HRDPS code as close to the Rust HRRR code as possible — copy-then-modify is fine; resist refactoring for shared abstraction until both branches are working.
Commit.
### Task 4.3: Task dispatch on `source` column
Worker's main loop branches on `task.source`: `"hrrr"` → existing path, `"hrdps"` → new path. The task seeder (`Microwaveprop.Workers.GridTaskEnqueuer`) populates `source` based on which worker enqueued the task.
Commit.
### Task 4.4: Build + push image
Existing CI pipeline (`.forgejo/workflows/build-grid-rs.yaml`) produces the `prop-grid-rs:main-*` image consumed by both `prop-grid-rs` and `hrrr-point-rs` deployments. New code rolls out automatically once merged. Verify in Argo / pod logs.
---
## Stage 5: `hrrr-point-rs` — HRDPS branch for per-QSO Canadian QSOs
**Files:**
- Modify: `rust/prop_grid_rs/src/bin/hrrr_point_worker.rs`
- Migration: `priv/repo/migrations/{timestamp}_add_source_to_hrrr_fetch_tasks.exs`
- Modify: `lib/microwaveprop/workers/hrrr_fetch_enqueue_worker.ex` (or wherever per-QSO fetches are enqueued)
### Task 5.1: Routing decision
When a QSO's lat/lon falls outside the HRRR CONUS footprint, enqueue an HRDPS-source task instead of an HRRR task. `Grid.point_source({lat, lon})` returns `:hrrr | :hrdps | :neither` (e.g. for a southern-hemisphere QSO). For `:neither`, fall back to the existing IEMRE/RAOB enrichment path.
TDD: cells at (45.0, -90.0) → `:hrrr`, (53.0, -113.0) → `:hrdps`, (-30.0, 150.0) → `:neither`.
Commit.
### Task 5.2: `hrrr_fetch_tasks.source` column + Rust branch
Same shape as Stage 4.1/4.3: add `source` column, branch the Rust worker, reuse Stage 1's URL/variable patterns ported to Rust.
Commit.
### Task 5.3: Per-QSO upserts into `hrdps_profiles`
QSO enrichment status column: add `hrdps_status` to `contacts` (similar to existing `hrrr_status`). LiveView/admin UI reads either, depending on which model covered the QSO.
Commit.
---
## Stage 6: Cycle scheduling + border stitching
**Files:**
- Modify: `config/runtime.exs` — Oban cron + queue
- Modify: `lib/microwaveprop/workers/grid_task_enqueuer.ex` (or equivalent)
### Task 6.1: Add `hrdps` queue
Production runtime config adds `hrdps: 1` (single concurrent fetch — wgrib2 working set during merge can hit ~500 MB and we don't want it competing with `propagation`).
### Task 6.2: Cycle cron at HH:35 of cycle hours
```elixir
{"35 5,11,17,23 * * *", Microwaveprop.Workers.HrdpsCycleSeedWorker}
```
Why these times: HRDPS runs at 00/06/12/18Z and publishes ~3-4h later, so 5h after cycle (at 05:35Z, 11:35Z, 17:35Z, 23:35Z) the f00 file is reliably available. Offset by 30 minutes from HRRR's `*/15` schedule to avoid wgrib2 contention.
`HrdpsCycleSeedWorker` enqueues f00..f18 grid tasks with `source: "hrdps"`. The existing chain worker (Rust + Elixir) drains them.
### Task 6.3: Border stitching rule
Already enforced at Stage 3.1: HRDPS-only mask cells are disjoint from HRRR's CONUS points. No runtime conflict. The seam at 49°N is implicit: HRRR cells exist down to ~25°N, HRDPS cells exist 49°N+. The boundary is a one-cell band of HRRR coverage abutting HRDPS — no overlap, no double-write.
Document the rule in `Grid.point_source/1`'s `@moduledoc` so future readers understand why the masks are exclusive.
Commit.
---
## Stage 7: Map overlay extension
**Files:**
- Modify: `lib/microwaveprop_web/live/map_live.ex``@initial_bounds`
- Modify: `assets/js/propagation_map_hook.ts` — heatmap rendering bounds
### Task 7.1: Widen `@initial_bounds`
CONUS-only `(23.5, 49.5, -124.5, -66.9)``(23.5, 60.0, -141.0, -52.0)` (excluding Arctic until Stage 8 decides). Default zoom adjusted so all of North America is visible.
### Task 7.2: Smoke test (manual + LiveView test)
`mix phx.server`, open `/map`, confirm colored cells render north of 49°N after `HrdpsCycleSeedWorker` fires. The cell-pack endpoint at `/scores/cells` already serves whatever lat/lon points exist in the `.prop` file — no controller changes needed since Stage 3.4 expanded the file.
LiveView test: assert that an `/scores/cells?band=10000&south=50&north=55&west=-115&east=-105` request returns >0 cells after HRDPS data exists in fixtures.
### Task 7.3: ScoreCache chunk layout still correct
The ScoreCache layout (just shipped in commit 6a90d153 follow-up) uses 5°×5° chunks. Extending the grid into Canada adds ~30 new chunks but the same lookup logic applies. No code change; one regression test that fetches a Canadian-bounds viewport and asserts hit/miss telemetry works correctly.
Commit.
---
## Stage 8: SRTM gap above 60°N — scope decision
**Decision gate.** SRTM1 stops at 60°N. Yellowknife (62.45°N), Whitehorse (60.72°N), Inuvik (68.36°N), and Resolute (74.70°N) have NO SRTM coverage — terrain diffraction analysis would silently return wrong losses for paths involving those stations.
Three options:
1. **Cap Canadian coverage at 60°N for v1.** Document explicitly in `algo.md`. Most VE QSOs are below 60°N anyway. Defer Arctic to a follow-up plan.
2. **Use CDEM (Canadian Digital Elevation Model) above 60°N.** ~1 week of additional work — different format (TIFF tiles), different mounting strategy on the production NFS share. Touches `terrain_analysis.ex` and `elevation_client.ex`.
3. **Use ASTER GDEM v3.** Global, free, 30 m resolution, but registration-walled. Not great.
**Recommendation: Option 1 for v1.** Ship the 42-60°N coverage. If users report Arctic gaps, do CDEM as a separate plan.
Decision recorded in this stage; file a follow-up issue if Option 2 is pursued.
---
## Stage 9: Freshness monitoring + observability
**Files:**
- Modify: `lib/microwaveprop/propagation/freshness_monitor.ex`
- Modify: wherever `/status` exposes data freshness (LiveView or controller)
### Task 9.1: Track HRDPS run-time staleness
Add `hrdps_last_run_at` to the monitor state. Surface on the status page next to HRRR's freshness.
### Task 9.2: Alert at >5h stale
HRDPS publishes ~4h after each cycle, so a healthy pipeline never exceeds ~5h between runs. >5h = something broken (datamart outage, our worker failed silently, k8s scheduling glitch).
`Logger.warning` (not `error` — datamart outages are common and recoverable). Telemetry event for the existing dashboard scrape.
Commit.
---
## Stage 10: Precommit, docs, ship
### Task 10.1: `mix precommit`
Must show 0 failures, 0 credo issues, all dialyzer warnings resolved. Full project gate.
### Task 10.2: Update `CLAUDE.md`
- Add HRDPS to **Key Data Sources**
- Add `HrdpsCycleSeedWorker` + `HrdpsGridWorker` + `HrdpsFetchWorker` to the **Background Job Queues** table
- Note the 60°N coverage cap and the cycle cadence (4×/day vs HRRR's hourly)
### Task 10.3: Update `memory/reference_hrdps_canada.md`
Mark plan as executed. Capture any projection gotchas, the final cycle latency observed in prod, and the actual border-stitching seam location.
### Task 10.4: Delete the probe wedge
`lib/mix/tasks/hrdps_probe.ex` is throwaway scaffolding — its risks were retired in Stage 0; the production code in `HrdpsClient` covers the same surface. Delete in this stage's commit, not earlier (keeping it around during stages 1-3 is useful for cross-checking the real client's output).
### Task 10.5: Push to `github` AND `origin`
Per the push-both-remotes feedback memory. Do NOT push to `dokku`. Argo/Flux handles the rollout from the kustomize manifest pulled from `git.mcintire.me`.
---
## Known risks + gotchas
- **Projection** — HRDPS rotated lat/lon is the single biggest bug source. Stage 1.6's CYYR validation against UWYO is the safety net; do not skip.
- **Per-variable file fan-out** — ~14 fetches per cycle/forecast hour × 4 cycles × 19 forecast hours = 1064 files/day. Each is small (~3.5 MB) but datamart's `200 OK` latency is variable. Concurrency-cap at 4 to avoid hammering ECCC; build retry-with-backoff into `HrdpsClient`.
- **Memory pressure** — wgrib2 `-merge` working set during the multi-variable combine peaks at ~500 MB per cycle. The HRRR worker already runs at `:system_memory_high_watermark` headroom; running HRDPS on the same pod risks OOM. Either pin HRDPS to a dedicated `prop-backfill`-style pod (probably overkill) OR keep it on the hot pods but cap `hrdps: 1` so it never overlaps with `propagation: 1`.
- **Cycle publish latency** — datamart can lag 90 min on good days, 4+h under load. The `HrdpsCycleSeedWorker` cron at HH:35 has a 1.5h margin which is comfortable but not infinite. If a cycle hasn't published by HH:35, the worker should fall back to seeding the previous cycle (still gives current+forecast scores, just one cycle stale).
- **Border seam visibility** — at the 49°N seam, HRRR's 0.125° cells abut HRDPS's 0.125° cells with no overlap. If the two models disagree by 5-10% on a given factor at the seam, users will see a faint horizontal stripe. Probably acceptable; document in `algo.md`.
- **Scope creep** — keep RDPS pressure-level coverage out of this plan even though they're related (RDPS plan is `2026-04-13-rdps-vertical-profiles.md`). Ship CONUS+Canada-via-HRDPS first; revisit RDPS for wider/coarser coverage afterward.
## Rough effort estimate
| Stage | Estimate |
|---|---|
| 0: Probe wedge | DONE (~3h, 2026-04-29) |
| 1: HrdpsClient | 4 days (Task 1.6 projection debug dominates) |
| 2: hrdps_profiles schema | 0.5 day |
| 3: HrdpsGridWorker + .prop writer | 3 days |
| 4: Rust prop-grid-rs HRDPS branch | 4 days |
| 5: Rust hrrr-point-rs HRDPS branch | 1.5 days |
| 6: Cycle scheduling + border stitching | 1 day |
| 7: Map overlay extension | 0.5 day |
| 8: SRTM scope decision | 0.5 day (decision + documentation, no code if Option 1) |
| 9: Freshness monitoring | 0.5 day |
| 10: Precommit + docs + ship | 0.5 day |
| **Total** | **~16 days (3 weeks calendar)** |
The Rust worker stages (4 + 5) are the majority of the schedule risk. If schedule is tight, an alternative is to ship Elixir-only first (Stages 1-3 + 6-10) and follow up with the Rust port — at the cost of slower Canadian forecast-chain processing during the gap.
---
## Open questions for confirmation before Stage 1
1. **Variable set:** Does the catalog at Task 1.2 cover everything the scorer needs for Canadian cells? Worth a 30-min look at `Microwaveprop.Propagation.Scorer.factors/2` to confirm the surface + pressure-level set is complete (especially native-level duct analysis — HRDPS's hybrid sigma-pressure levels may be available under a `*_HY_*` or `native/` directory that the probe didn't explore).
2. **60°N cap:** Confirm Option 1 from Stage 8 (defer Arctic CDEM) before kickoff so we don't end up with the Stage 1.1 grid mask covering territory we won't actually serve.
3. **Forecast horizon:** Default plan uses HRDPS f00..f18 to match HRRR's window. HRDPS goes out to f48 — would f00..f24 or even f00..f48 (full Day-2 forecast, only on the Canadian side) be useful? Storage/compute roughly doubles per extra 12h of horizon.