docs: update prediction.md status, runbook NOTIFY payload + file formats
Some checks failed
Build and Push / Build and Push Docker Image (push) Failing after 4m46s

This commit is contained in:
Graham McIntire 2026-08-01 19:29:33 -05:00
parent 2fd88a94ea
commit b2c667a75d
No known key found for this signature in database
GPG key ID: F4ABF488E6029E59
2 changed files with 19 additions and 2 deletions

View file

@ -18,8 +18,10 @@ prop-grid-rs (worker nodes) Postgres (10.0.15.30) prop pods (nod
1. Rust completes `{band, valid_time}` compute and writes the `.prop`
file via atomic rename. (Readers also accept legacy `.ntms` files
with `NTMS` magic during a rolling deploy.)
2. Rust emits `NOTIFY propagation_ready '<iso valid_time>'`.
3. `NotifyListener` on each Elixir pod receives it, prunes
2. Rust emits `NOTIFY propagation_ready '<run_time_iso>|<valid_time_iso>'`
(pipe-delimited: run_time first, valid_time second — needed for
`retain_scores_window` which uses run_time as the retention anchor).
3. `NotifyListener` on each Elixir pod splits the payload, prunes
ScoreCache entries outside the active forecast window, and
broadcasts `propagation_updated` over PubSub.
4. LiveView clients see the broadcast and re-request scores. The
@ -91,6 +93,19 @@ to drain. All fetch/decode/score helpers moved to
`rust/prop_grid_rs/src/pipeline.rs`; the Elixir module is retained
only as the cron entry point, not as a fallback compute path.
## On-disk file formats
Score files live under `/data/scores/<band_mhz>/<valid_time_iso>.<ext>`:
| Extension | Source | Writer | Description |
|-----------|--------|--------|-------------|
| `.prop` | HRRR (f00f48) | `prop-grid-rs` (Rust) | Primary CONUS scores, 0.125° grid, 92k cells per band |
| `.hrdps.prop` | HRDPS (Canadian) | `hrdps_grid_worker` (Elixir) | Canadian coverage, merged at read with HRRR > HRDPS priority |
| `.gefs.prop` | GEFS (f024f168) | `gefs_fetch_worker` (Elixir) | Extended-range ensemble mean, merged at read with HRRR > HRDPS > GEFS priority |
| `.ntms` | Legacy | (deprecated) | Old format, still accepted by readers during rolling deploys |
All files use atomic rename on write (write to `.tmp.<uuid>`, then `rename(2)`).
## Monitoring signals
- `avg_over_time(beam_memory_total_bytes[1h])` on prop pods — steady

View file

@ -1,5 +1,7 @@
# Propagation Prediction System Design
> **Status (2026-08-01): All three phases are implemented.** The Rust pipeline (`prop-grid-rs`) fetches HRRR f00f48 hourly, scores all 22 bands across the full CONUS grid, and writes per-valid_time `.prop` files. The map timeline shows the next 48 hours of real NWP-forecast scores (not persistence). `Propagation.point_forecast/3` provides the per-point sparkline. See `docs/runbook_propagation_pipeline.md` for operations; `scripts/validate_forecast.py` measures forecast skill by lead time.
## The Opportunity
No tool exists for microwave operators to answer: **"When should I get on the air in the next 24 hours?"**