prop/docs/plans/2026-04-19-rust-migration-phase3.md
Graham McIntire 48708621c5
chore(mrms): delete pipeline — consumer AsosAdjustmentWorker already disabled
MrmsFetchWorker fired every 2 min on the :propagation queue, decoding an
MRMS PrecipRate GRIB2 into MrmsCache. Its only consumer,
AsosAdjustmentWorker, is disabled in all configs (dev / config / runtime).
Net effect: 30 GRIB2 decodes/hour of dead work on hot pods.

Removed:
- MrmsFetchWorker, MrmsClient, MrmsCache and their tests
- cron entries in config.exs, dev.exs, runtime.exs
- mrms_req_options stub in test.exs
- MrmsCache from supervision tree in application.ex
- stale MRMS references in PropagationGridWorker docstring and test

Also adds docs/plans/2026-04-19-rust-migration-phase3.md which tracks
the broader Stream A (f00 port) and Stream C (HrrrFetchWorker port)
follow-on work.
2026-04-19 17:25:59 -05:00

607 lines
24 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Rust Migration Phase 3 — f00 Port + MRMS Deletion + HRRR Per-QSO Port
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** Remove every remaining GRIB2 decode from BEAM so hot pods can shrink to 1 Gi, and delete one dead pipeline (MRMS) along the way.
**Architecture:** Three independent work streams against the same shared `prop-grid-rs` binary. Stream A ports f00's native-duct + NEXRAD + commercial-link merge from Elixir into Rust via a new `grid_tasks.kind` column so the same work queue handles both forecast hours and analysis-hour enrichment. Stream B deletes the MRMS worker/cache/client entirely (the only consumer `AsosAdjustmentWorker` is already disabled). Stream C ports the per-QSO `HrrrFetchWorker` into a new Rust binary sharing the same crate, taking the `:hrrr` queue off BEAM.
**Tech Stack:** Rust (tokio, sqlx, reqwest, wgrib2 subprocess), Elixir (Oban, Postgrex.Notifications), Postgres (grid_tasks work queue + LISTEN/NOTIFY), Kubernetes/flux.
---
## Work Streams Overview
| Stream | Priority | What | Payoff |
|---|---|---|---|
| **A: Port f00** | 1 | Move f00 chain step (native duct + NEXRAD + commercial + ProfilesFile write) from Elixir to Rust. | Finishes hourly-chain memory story. Hot pod can drop from 2Gi/replica × N replicas to 1Gi. |
| **B: Delete MRMS** | 2 | Remove MrmsFetchWorker, MrmsClient, MrmsCache, and the `*/2 * * * *` cron. | Removes 30 GRIB2 decodes/hour of dead work. Zero risk — AsosAdjustmentWorker (only consumer) is already disabled. |
| **C: Port HrrrFetchWorker** | 3 | Move per-QSO HRRR profile fetch to Rust. New `hrrr_point_tasks` queue. Elixir enqueues, Rust fulfills. | Takes the `:hrrr` queue off BEAM. Removes per-QSO refc-binary churn and the explicit `:erlang.garbage_collect` that's papering over it. |
Each stream is independently mergeable and reversible. Recommended order: **B → A → C**. B first because it's a pure delete (simplest; shrinks the attack surface before A touches more code). A second because it's the real memory win. C last because it's optional polish.
---
## Stream B: Delete MRMS (start here — warm-up)
**Context:** `MrmsFetchWorker` runs every 2 minutes on the `:propagation` queue. It decodes an MRMS PrecipRate GRIB2 and caches the regridded result in `MrmsCache`. The only consumer that reads that cache, `AsosAdjustmentWorker`, is disabled in all three configs (see `config/runtime.exs:212-216`). Net effect: 30 GRIB2 decodes/hour of dead work on the hot pods.
### Task B1: Confirm no live consumers
**Files to check:**
- Search for `MrmsCache` and `Mrms\.` across `lib/` and `test/` — confirm only the worker/client/cache/tests reference it.
- Search for `AsosAdjustmentWorker` — confirm the only references are disabled cron entries + the worker file itself.
**Step 1: Run the searches**
```bash
grep -rn "MrmsCache\|Mrms\." lib/ config/ | grep -v "#"
grep -rn "AsosAdjustmentWorker" config/ lib/
```
**Expected output:** References only in `workers/mrms_fetch_worker.ex`, `weather/mrms_client.ex`, `weather/mrms_cache.ex`, `application.ex` (supervisor), and commented-out cron lines. No LiveView, context, or worker depends on it.
**If you find a live consumer, STOP** — add that consumer to the removal list before proceeding.
### Task B2: Remove MRMS from supervisor + cron
**Files:**
- Modify: `lib/microwaveprop/application.ex` (drop `MrmsCache` from the supervision tree)
- Modify: `config/runtime.exs:206-211` (drop the `MrmsFetchWorker` cron line and its comment block)
- Modify: `config/config.exs:101` (same removal in dev-carrying config)
- Modify: `config/dev.exs:100` (same)
**Step 1: Write the failing test**
Add to `test/microwaveprop/application_test.exs` (create if missing):
```elixir
defmodule Microwaveprop.ApplicationTest do
use ExUnit.Case, async: true
test "MrmsCache is not in the supervision tree" do
children = Microwaveprop.Application.children_for_env(:test)
refute Enum.any?(children, &match?({Microwaveprop.Weather.MrmsCache, _}, &1))
refute Enum.any?(children, fn
Microwaveprop.Weather.MrmsCache -> true
_ -> false
end)
end
end
```
If `children_for_env/1` doesn't exist, either extract it from `start/2` or inline the list in the test.
**Step 2: Run test to verify it fails**
```bash
mix test test/microwaveprop/application_test.exs
```
Expected: FAIL — MrmsCache still in supervision tree.
**Step 3: Remove MrmsCache from supervision tree**
Edit `lib/microwaveprop/application.ex`, delete the line that adds `MrmsCache`.
**Step 4: Run test to verify it passes**
```bash
mix test test/microwaveprop/application_test.exs
```
Expected: PASS.
**Step 5: Remove cron entries**
Delete the `MrmsFetchWorker` entry + its leading comment block from all three config files.
**Step 6: Commit**
```bash
git add lib/microwaveprop/application.ex config/ test/microwaveprop/application_test.exs
git commit -m "chore(mrms): remove from supervision tree and cron (consumer disabled)"
```
### Task B3: Delete MRMS source files and tests
**Files:**
- Delete: `lib/microwaveprop/workers/mrms_fetch_worker.ex`
- Delete: `lib/microwaveprop/weather/mrms_client.ex`
- Delete: `lib/microwaveprop/weather/mrms_cache.ex`
- Delete: `test/microwaveprop/workers/mrms_fetch_worker_test.exs`
- Delete: `test/microwaveprop/weather/mrms_cache_test.exs`
- Modify: `config/test.exs:66` (drop `mrms_req_options`)
**Step 1: Delete files**
```bash
rm lib/microwaveprop/workers/mrms_fetch_worker.ex \
lib/microwaveprop/weather/mrms_client.ex \
lib/microwaveprop/weather/mrms_cache.ex \
test/microwaveprop/workers/mrms_fetch_worker_test.exs \
test/microwaveprop/weather/mrms_cache_test.exs
```
**Step 2: Drop the test config line**
Edit `config/test.exs:66`, remove the `mrms_req_options` line.
**Step 3: Run full test suite**
```bash
mix precommit
```
Expected: clean compile (warnings-as-errors), all tests pass. If `mix xref` surfaces a caller, fix it.
**Step 4: Commit**
```bash
git add -A
git commit -m "chore(mrms): delete worker, client, cache, and tests"
```
### Task B4: Drop MRMS k8s resources (if any)
**Files to inspect:**
- `k8s/kustomization.yaml`, any `mrms_*` resource
**Step 1: Check**
```bash
grep -r "mrms\|Mrms" k8s/
```
**Step 2: If found, remove. Otherwise skip.**
**Step 3: Commit if changed.**
---
## Stream A: Port f00 to Rust
**Context:** Today, Elixir's `PropagationGridWorker.perform/1` handles f00 only — the analysis-hour step that does three things Rust doesn't: native-level HRRR duct merge (the ~530 MB native GRIB2), NEXRAD composite reflectivity overlay, and commercial-link degradation lookup. It also writes the `ProfilesFile` that `/weather` reads. Forecast hours f01f18 already run in Rust via the `grid_tasks` queue.
**Strategy:** Add a `kind` column to `grid_tasks` so the same queue serves both forecast hours (`kind='forecast'`) and analysis-hour enrichment (`kind='analysis'`). Rust's worker dispatches on `kind`. Elixir's seeder switches from `Oban.insert_all` for f00 to seeding a `kind='analysis'` row alongside the 18 `kind='forecast'` rows. When the analysis task completes, Rust writes the same `/data/scores/<band>/<iso>.ntms` + `/data/profiles/<iso>.ntms` files and NOTIFYs `propagation_ready` with the existing payload.
### Task A1: Add `kind` column to grid_tasks
**Files:**
- Create: `priv/repo/migrations/YYYYMMDDHHMMSS_add_kind_to_grid_tasks.exs`
**Step 1: Generate migration**
```bash
mix ecto.gen.migration add_kind_to_grid_tasks
```
**Step 2: Write the migration**
```elixir
defmodule Microwaveprop.Repo.Migrations.AddKindToGridTasks do
use Ecto.Migration
def change do
alter table(:grid_tasks) do
add :kind, :string, null: false, default: "forecast"
end
create index(:grid_tasks, [:status, :kind])
# Replace the old unique index with one that includes kind so an
# analysis task and a forecast task for the same (run_time, f00)
# can coexist without conflict.
drop_if_exists unique_index(:grid_tasks, [:run_time, :forecast_hour])
create unique_index(:grid_tasks, [:run_time, :forecast_hour, :kind])
end
end
```
**Step 3: Run migration**
```bash
mix ecto.migrate
```
**Step 4: Commit**
```bash
git add priv/repo/migrations/
git commit -m "feat(grid_tasks): add kind column to separate forecast vs analysis work"
```
### Task A2: Rust reads `kind` and routes
**Files:**
- Modify: `rust/prop_grid_rs/src/db.rs` — add `kind: String` to the task struct, select it in `claim_next`
- Modify: `rust/prop_grid_rs/src/bin/worker.rs` — match on `kind` and dispatch to forecast vs. analysis pipeline
- Test: `rust/prop_grid_rs/src/db.rs` test for `claim_next` returning the `kind`
**Step 1: Write failing test**
In `rust/prop_grid_rs/src/db.rs`:
```rust
#[tokio::test]
async fn claim_next_returns_kind() {
// seed one task with kind='analysis', claim it, assert kind round-trips
// ...
}
```
**Step 2: Run test**
```bash
cd rust/prop_grid_rs && cargo test claim_next_returns_kind
```
Expected: FAIL (field doesn't exist).
**Step 3: Add `kind` field to the claim struct + SELECT list**
Minimal change to `claim_next` to read and return `kind`. Default to `"forecast"` if unmapped so the compile stays green during rollout.
**Step 4: Run test**
Expected: PASS.
**Step 5: Commit**
```bash
git add rust/prop_grid_rs/src/db.rs
git commit -m "feat(prop-grid-rs): surface grid_tasks.kind in claim_next"
```
### Task A3: Port `HrrrNativeClient.fetch_native_duct_grid` to Rust
This is the biggest subtask — the 530 MB GRIB2 native-level pull + duct-metric computation.
**Files:**
- Create: `rust/prop_grid_rs/src/native_duct.rs`
- Test: `rust/prop_grid_rs/src/native_duct.rs` (inline `#[cfg(test)] mod tests`)
**Step 1: Write golden-fixture test**
Follow the same pattern Phase 1 used for surface/pressure: a Mix task dumps the Elixir-computed duct grid for a known `(date, hour)` to a bincode file in `rust/prop_grid_rs/tests/golden/`. Rust loads it and asserts exact match on every `{lat, lon}` key and per-field values.
Create `lib/mix/tasks/rust.golden_duct.ex`:
```elixir
defmodule Mix.Tasks.Rust.GoldenDuct do
use Mix.Task
@shortdoc "Dump Elixir-computed native duct grid for Rust comparison"
def run([date_str, hour_str]) do
Mix.Task.run("app.start")
date = Date.from_iso8601!(date_str)
hour = String.to_integer(hour_str)
grid_spec = Microwaveprop.Propagation.Grid.wgrib2_grid_spec()
{:ok, duct_grid} =
Microwaveprop.Weather.HrrrNativeClient.fetch_native_duct_grid(date, hour, grid_spec, 0)
path = "rust/prop_grid_rs/tests/golden/duct_#{date_str}_t#{hour_str}z.bincode"
File.mkdir_p!(Path.dirname(path))
File.write!(path, :erlang.term_to_binary(duct_grid))
IO.puts("Wrote #{map_size(duct_grid)} cells to #{path}")
end
end
```
**Step 2: Run the golden dump against a recent hour**
```bash
mix rust.golden_duct 2026-04-19 14
```
Expected: writes a bincode file with ~92k cells.
**Step 3: Write the Rust failing test**
In `rust/prop_grid_rs/src/native_duct.rs`:
```rust
#[tokio::test]
async fn matches_elixir_golden() {
let golden = load_golden("tests/golden/duct_2026-04-19_t14z.bincode");
let rust = fetch_native_duct_grid(Date::from_ymd(2026, 4, 19), 14, grid_spec(), 0).await.unwrap();
assert_eq!(rust.len(), golden.len());
for (pt, expected) in &golden {
let got = rust.get(pt).expect(&format!("missing {:?}", pt));
assert!((got.native_min_gradient - expected.native_min_gradient).abs() < 1e-3);
assert_eq!(got.duct_count, expected.duct_count);
// ... other fields
}
}
```
**Step 4: Run test**
```bash
cargo test matches_elixir_golden
```
Expected: FAIL — `fetch_native_duct_grid` doesn't exist.
**Step 5: Implement the Rust port**
Port these pieces in order, one commit each:
1. **Byte-range fetch for duct variables.** Port `duct_byte_ranges/1` (selects TMP/SPFH/HGT/PRES on all 50 hybrid levels from the idx). Reuse the existing `fetcher::download_ranges_to_file` that Phase 1 built.
2. **wgrib2 subprocess for `extract_grid_from_file_mapped`.** Port the `-lola` + match-pattern invocation. Parse the flat output into a `HashMap<(lat, lon), HashMap<String, f32>>`. Stream cell-by-cell rather than buffering all values — matches Elixir's "peak memory ~86 MB" design.
3. **`compute_duct_metrics` per cell.** Port `build_native_profile/1`, `min_m_gradient/1`, and `Duct.analyze/1`. `Duct.analyze/1` is in `lib/microwaveprop/propagation/duct.ex` — that's its own port.
4. **Wire into `native_duct.rs::fetch_native_duct_grid`** as the public entry point.
**Step 6: Run test to verify it passes**
```bash
cargo test matches_elixir_golden
```
Expected: PASS with max-per-cell delta under 1e-3.
**Step 7: Commit**
```bash
git add rust/prop_grid_rs/src/native_duct.rs rust/prop_grid_rs/tests/golden/ lib/mix/tasks/rust.golden_duct.ex
git commit -m "feat(prop-grid-rs): port HrrrNativeClient.fetch_native_duct_grid"
```
### Task A4: Port NEXRAD composite reflectivity fetch
**Context:** `NexradClient.fetch_frame/2` (in `lib/microwaveprop/weather/nexrad_client.ex`) pulls the current MRMS composite reflectivity mosaic and returns `[%{lat, lon, max_reflectivity_dbz}]`. The merge into `grid_data` is line-level (`apply_nexrad_observations`). It's another GRIB2 decode but much smaller than the native-level file.
**Files:**
- Create: `rust/prop_grid_rs/src/nexrad.rs`
- Test: inline fixture test
**Step 1: Golden fixture**
Same pattern: `mix rust.golden_nexrad 2026-04-19T14:00:00Z` dumps the Elixir observations list as bincode.
**Step 2: Write failing test**
```rust
#[tokio::test]
async fn matches_elixir_golden() { /* ... */ }
```
**Step 3: Implement `fetch_frame`** by porting the HTTP GET + GRIB2 decode from `NexradClient`.
**Step 4: Run test.** Expected PASS.
**Step 5: Commit.**
### Task A5: Port commercial-link degradation lookup
**Context:** `Commercial.build_link_lookup/1` (in `lib/microwaveprop/commercial.ex`) returns a `[{{lat, lon}, degradation}]` list from Postgres. `link_degradation_from_lookup/2` is a pure haversine match. Only 7 links cluster around DFW so most cells return nil.
**Files:**
- Create: `rust/prop_grid_rs/src/commercial.rs`
- Test: inline SQL + haversine round-trip
**Step 1: Golden fixture (simpler — just dump the 7 link rows)**
**Step 2: Failing test**
```rust
#[tokio::test]
async fn haversine_matches_elixir() { /* 7 links × 10 sample points */ }
```
**Step 3: Port `build_link_lookup` (one SELECT) + haversine matcher.**
**Step 4: Commit.**
### Task A6: Port ProfilesFile write
**Context:** `Microwaveprop.Propagation.ProfilesFile.write!/2` serializes the fully-enriched `grid_data` to `/data/profiles/<iso>.ntms` using `:erlang.term_to_binary/1`. This is the tricky one — the file format is Erlang ETF.
**Decision point:** Two options.
1. **Port the reader to Rust + ETF encoder.** The `eetf` crate exists but is not heavily maintained. Risk: ETF subtleties (ref counting, atom tables) could produce bit-different output that breaks Elixir's reader.
2. **Change the format.** Replace ETF with a simple framed bincode/cbor/msgpack file. Rust writes, Elixir reads via a new decoder. Clean break.
**Recommendation: Option 2.** The profiles file is read from exactly one place (`/weather` and point-detail LiveView in `lib/microwaveprop/weather.ex`). Porting the reader to a new format is ~50 lines. Fighting with eetf is a tarpit.
**Files:**
- Create: `rust/prop_grid_rs/src/profiles_file.rs` (writes new format)
- Modify: `lib/microwaveprop/propagation/profiles_file.ex` (reads old format; add reader for new format; write-path deleted once Rust owns f00)
- Modify: `lib/microwaveprop/weather.ex` (call sites that read profiles)
**Step 1: Design the format.** Header: magic `PROF\x00`, version u8, record count u32, `{lat: f32, lon: f32, field_count: u8, [{key: varint-len-prefixed string, value: tagged-union}]...}`. Or — simpler — MessagePack. `rmp-serde` in Rust, `Msgpax` in Elixir. Both are well-maintained.
**Step 2: Write Elixir reader for MessagePack variant.** Keep the old ETF reader around under a `read_etf!/1` alias for backward compatibility during rollout.
**Step 3: Failing test in Rust.** Round-trip: write a sample `grid_data`, read it back, assert equal.
**Step 4: Implement writer.**
**Step 5: Dual-read test in Elixir.** Old ETF files still read; new MessagePack files read identically.
**Step 6: Commit.**
### Task A7: Wire analysis pipeline in Rust worker
**Files:**
- Modify: `rust/prop_grid_rs/src/pipeline.rs` — add `run_analysis_task` alongside the existing forecast pipeline
- Modify: `rust/prop_grid_rs/src/bin/worker.rs` — dispatch on `task.kind`
**Step 1: Write failing integration test**
In `rust/prop_grid_rs/tests/analysis_pipeline_test.rs`:
```rust
#[tokio::test]
async fn analysis_task_writes_scores_and_profiles() {
// seed one kind='analysis' task, run pipeline, assert
// /data/scores/10000/<iso>.ntms exists
// /data/profiles/<iso>.ntms exists
// NOTIFY propagation_ready fired
}
```
**Step 2: Run test.** Expected FAIL.
**Step 3: Implement `run_analysis_task`.** Same shape as forecast pipeline but:
- Calls `native_duct::fetch_native_duct_grid` after the regular surface+pressure merge.
- Calls `nexrad::fetch_frame` and merges.
- Calls `commercial::build_link_lookup` and merges.
- Writes `ProfilesFile` to `/data/profiles/<iso>.ntms`.
- Scores all bands as before.
**Step 4: Run test.** Expected PASS.
**Step 5: Commit.**
### Task A8: Flip Elixir seeder to enqueue analysis as grid_task
**Files:**
- Modify: `lib/microwaveprop/workers/propagation_grid_worker.ex:80-99` (seed_chain)
- Modify: `lib/microwaveprop/propagation/grid_task_enqueuer.ex` (add `seed_with_analysis/1`)
**Step 1: Write failing test**
In `test/microwaveprop/workers/propagation_grid_worker_test.exs`, update the seed test to assert:
- **Zero** Oban jobs enqueued (f00 is no longer an Oban job).
- **19** grid_tasks rows: 1 with `kind='analysis' AND forecast_hour=0`, 18 with `kind='forecast' AND forecast_hour IN (1..18)`.
**Step 2: Run test.** Expected FAIL.
**Step 3: Modify `seed_chain` to call `GridTaskEnqueuer.seed_with_analysis(run_time)` and return `:ok`.** Delete the `Oban.insert_all` line. Delete `run_chain_step`, `process_forecast_hour`, and all the merge helpers from `propagation_grid_worker.ex`. The module becomes a thin cron handler.
**Step 4: Run test.** Expected PASS.
**Step 5: Delete now-unused Elixir modules** after `mix xref callers` confirms no callers remain:
- `lib/microwaveprop/weather/hrrr_native_client.ex`
- `lib/microwaveprop/weather/nexrad_client.ex` (assuming no other caller)
- The `merge_native_duct_data`, `merge_nexrad_data`, `merge_commercial_link_data`, `compute_scores_*` helpers
**Step 6: Commit.**
### Task A9: Shrink hot pod memory limit
**Files:**
- Modify: `k8s/deployment.yaml`
**Step 1: Wait for 24h of clean post-cutover runs.**
**Step 2: Reduce memory limit from 2Gi to 1Gi, requests from 512Mi to 256Mi.**
**Step 3: Watch `kubectl -n prop top pod` for OOM signals over 24h.** If any pod OOMKills, revert immediately and investigate before retrying.
**Step 4: Commit.**
---
## Stream C: Port HrrrFetchWorker (optional polish)
**Context:** Each QSO submission enqueues an `HrrrFetchWorker` job that fetches a HRRR profile at the contact location. Per-job payload is small (one point, ~100 kB of GRIB2 bytes) but the job calls `HrrrClient` which on BEAM holds refc binaries that the worker explicitly `:erlang.garbage_collect`s at the end. That GC is a signal the binary pressure is real.
**Strategy:** New Rust binary `hrrr-point-worker` shares the existing `prop_grid_rs` crate (adds a second bin). A new `hrrr_point_tasks` table mirrors `grid_tasks`. Elixir's ingestion switches from `HrrrFetchWorker` (Oban) to inserting into `hrrr_point_tasks`.
### Task C1: Migration for `hrrr_point_tasks`
**Files:**
- Create: `priv/repo/migrations/YYYYMMDDHHMMSS_create_hrrr_point_tasks.exs`
**Columns:** `id uuid PK`, `qso_id uuid`, `valid_time timestamp`, `lat double precision`, `lon double precision`, `status text`, `claimed_at`, `completed_at`, `error text`, `attempt smallint`. Unique on `(qso_id, valid_time)`.
**Step 1: Generate + write migration.**
**Step 2: Run migration.**
**Step 3: Commit.**
### Task C2: Rust `hrrr-point-worker` binary
**Files:**
- Create: `rust/prop_grid_rs/src/bin/hrrr_point_worker.rs`
- Create: `rust/prop_grid_rs/src/point_fetcher.rs` (per-point HRRR profile extract)
**Step 1: Golden fixture for HrrrClient.fetch_profile.**
**Step 2: Failing test.**
**Step 3: Implement point fetcher + worker loop.** Reuse the existing `fetcher` module for byte-range downloads; add a new `extract_point_profile` that takes a single `(lat, lon)` instead of a grid spec.
**Step 4: Commit.**
### Task C3: k8s deployment for hrrr-point-worker
**Files:**
- Create: `k8s/deployment-hrrr-point-rs.yaml` (same image, different ENTRYPOINT)
- Modify: `k8s/kustomization.yaml`
**Step 1: Write manifest. 1 replica on talos5 initially, 256 Mi memory.**
**Step 2: Apply.**
**Step 3: Commit.**
### Task C4: Switch Elixir ingestion to enqueue into `hrrr_point_tasks`
**Files:**
- Modify: wherever `HrrrFetchWorker.new(...)` is called (check `lib/microwaveprop/qsos.ex` or `lib/microwaveprop/radio.ex` via `grep -rn "HrrrFetchWorker.new"`)
- Delete: `lib/microwaveprop/workers/hrrr_fetch_worker.ex` (after `mix xref callers` is clean)
- Delete: corresponding test
**Step 1: Failing test.** QSO submission inserts a row into `hrrr_point_tasks` and does NOT enqueue an Oban job.
**Step 2: Implement.**
**Step 3: Commit.**
---
## Critical Files Reference
**Ported out (deleted at end of Stream A):**
- `lib/microwaveprop/weather/hrrr_native_client.ex`
- `lib/microwaveprop/weather/nexrad_client.ex` (confirm no per-QSO callers first)
- `lib/microwaveprop/weather/mrms_client.ex` (Stream B)
- `lib/microwaveprop/weather/mrms_cache.ex` (Stream B)
- `lib/microwaveprop/workers/mrms_fetch_worker.ex` (Stream B)
- `lib/microwaveprop/workers/hrrr_fetch_worker.ex` (Stream C)
- Most of `lib/microwaveprop/workers/propagation_grid_worker.ex` — becomes a thin seeder
**Kept as-is:**
- `lib/microwaveprop/commercial.ex` — Elixir still needs this for the SNMP poller and LiveView sensor widget. Rust just adds a parallel reader over the same `commercial_samples` table.
- `lib/microwaveprop/propagation/duct.ex` — ported to Rust, original stays because other callers (per-contact analysis) still use it.
**New Rust modules (Stream A):**
- `rust/prop_grid_rs/src/native_duct.rs`
- `rust/prop_grid_rs/src/nexrad.rs`
- `rust/prop_grid_rs/src/commercial.rs`
- `rust/prop_grid_rs/src/profiles_file.rs`
- `rust/prop_grid_rs/src/duct.rs` (ported from `lib/microwaveprop/propagation/duct.ex`)
**New Rust module (Stream C):**
- `rust/prop_grid_rs/src/point_fetcher.rs`
- `rust/prop_grid_rs/src/bin/hrrr_point_worker.rs`
---
## Verification
**Stream B:**
- `grep -r "Mrms" lib/ test/ config/` returns nothing.
- `mix precommit` green.
- 24h later: `kubectl -n prop get events` shows no MrmsFetchWorker executions.
**Stream A:**
- `cargo test -p prop_grid_rs` green with golden-duct, golden-nexrad, golden-commercial fixtures.
- `mix test` green.
- One full post-cutover hourly cycle shows `kind='analysis'` task `status='done'` in `grid_tasks`, `/data/profiles/<iso>.ntms` written, `/weather` page loads correctly.
- Hot pod `beam_memory_total` < 700 MiB peak over 24 h (down from ~1 GiB under Phase 2).
- Hot pod memory limit reduced to 1 Gi with zero OOMKilled events over 7 days.
**Stream C:**
- QSO submission inserts a `hrrr_point_tasks` row within 1 s.
- Rust `hrrr-point-worker` logs show claimcomplete within 30 s of insert.
- HRRR profile appears on the QSO detail page within 1 min.
- `:hrrr` Oban queue is empty and can be removed from the Oban config.
---
## Rollback Plans
**Stream B:** `git revert` restores MRMS. Supervision tree re-adds MrmsCache on next deploy. No data loss MrmsCache was in-memory only.
**Stream A:** `git revert` restores the Elixir f00 chain step. The `grid_tasks.kind` column can stay (default `'forecast'` keeps existing Rust behaviour). Bump hot pod memory back to 2Gi if Phase 3 shrink was already deployed.
**Stream C:** `git revert` restores `HrrrFetchWorker`. `hrrr_point_tasks` rows become orphan; a one-off Mix task migrates them back to Oban jobs.