prop/docs/plans/2026-06-10-extend-hrrr-forecast-to-48h.md
Graham McIntire aab3c3736c
feat: extend HRRR forecast horizon from 18h to 48h
HRRR publishes f21-f48 at 3-hourly intervals in addition to the
hourly f01-f18 we already fetch. This extends the grid_tasks seed
from 19 rows (1 analysis + 18 forecast) to 29 rows (1 analysis +
18 hourly + 10 3-hourly), and bumps the UI forecast window
constant from 18h to 48h so the map timeline and path calculator
forecast chart automatically show the extended horizon.

No Rust logic changes needed — the pipeline is data-driven and
u8 forecast_hour already handles f48.
2026-06-10 11:36:58 -05:00

12 KiB
Raw Blame History

Extend HRRR Forecast Horizon to 48 Hours — Implementation Plan

For Claude: REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

Goal: Extend the propagation forecast from 18 hours to 48 hours using HRRR's own extended forecast hours (f21f48 at 3-hourly intervals), without adding a new data source.

Architecture: The system is data-driven — whatever forecast hours are seeded into grid_tasks, Rust processes. Two constants gate the current 18-hour cap: GridTaskEnqueuer.@max_forecast_hour and Propagation.@hrrr_forecast_horizon_hours. Changing these, plus adjusting a few validation limits and comment strings, is the entire change. No Rust code changes needed.

Tech Stack: Elixir (Oban, Ecto, Phoenix LiveView), Rust (prop_grid_rs), HRRR NOAA S3

Relationship to prior plan: docs/plans/2026-04-18-extended-horizon-forecasts.md covers adding GFS as a separate data source for 10-day forecasts. This plan is the zero-new-infrastructure quick win — HRRR already publishes f21f48, we just don't fetch them. The GFS plan is still the right long-term approach for multi-day planning; this extends the existing hourly chain at near-zero cost.


HRRR forecast hour availability

Range Interval Count Notes
f00 1 Analysis (Elixir-owned, unchanged)
f01f18 1-hourly 18 What we fetch today
f21f48 3-hourly 10 New: f21, f24, f27, f30, f33, f36, f39, f42, f45, f48

Total: 1 analysis + 18 hourly + 10 3-hourly = 29 grid_tasks rows per cycle (up from 19).

Accuracy degrades beyond f18 — HRRR is a 3 km convection-allowing model; small-scale features (ducting, refractivity gradients) lose fidelity at longer lead times. The 3-hourly output spacing reflects this. Accepted trade-off per user request.


Task 1: Define the forecast hour list and update GridTaskEnqueuer

Files:

  • Modify: lib/microwaveprop/propagation/grid_task_enqueuer.ex

Changes:

Replace the @max_forecast_hour 18 module attribute with a list of all forecast hours Rust should process:

# Before (line 19):
@max_forecast_hour 18

# After:
@forecast_hours [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 21, 24, 27, 30, 33, 36, 39, 42, 45, 48]

Update seed_with_analysis/2 (line 70) — the for fh <- 1..@max_forecast_hour comprehension:

# Before (line 70):
for fh <- 1..@max_forecast_hour do

# After:
for fh <- @forecast_hours do

Update seed/2 (line 103) — same pattern:

# Before (line 103):
for fh <- 1..@max_forecast_hour do

# After:
for fh <- @forecast_hours do

Update seed_current_hour/2 (lines 174, 192) — the clamp ceiling:

# Before (line 174):
# Forecast hour is clamped to `1..@max_forecast_hour`.
# (line 176):
# `run_time + @max_forecast_hour`, we cap there.
# (line 192):
|> min(@max_forecast_hour)

# After (line 174):
# Forecast hour is clamped to the available forecast hour list.
# (line 176):
# the max forecast hour, we cap there.
# (line 192):
|> min(Enum.max(@forecast_hours))

Update doc comments (lines 3637, 92):

# Before (line 36-37):
# Seed one kind='analysis' row (f00) plus 18 kind='forecast' rows
# (f01..f18) for `run_time`.

# After:
# Seed one kind='analysis' row (f00) plus forecast rows
# (f01..f18 hourly, f21..f48 3-hourly) for `run_time`.

# Before (line 92):
# Legacy: seed only kind='forecast' rows (f01..f18).

# After:
# Legacy: seed only kind='forecast' rows.

Step 1: Update the test to expect the new forecast hour list

File: test/microwaveprop/propagation/grid_task_enqueuer_test.exs:22

# Before:
assert rows |> Enum.map(& &1.forecast_hour) |> Enum.sort() == Enum.to_list(1..18)

# After:
@forecast_hours [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 21, 24, 27, 30, 33, 36, 39, 42, 45, 48]
assert rows |> Enum.map(& &1.forecast_hour) |> Enum.sort() == @forecast_hours

Update the worker test at test/microwaveprop/workers/propagation_grid_worker_test.exs:

# Line 7 (comment):
# Before: inserts 19 grid_tasks rows (1 analysis f00 + 18 forecast f01..f18)
# After:  inserts grid_tasks rows (1 analysis f00 + forecast f01..f18 hourly, f21..f48 3-hourly)

# Line 70:
# Before: test "inserts 1 analysis + 18 forecast grid_tasks rows; no Oban fan-out" do
# After:  test "inserts 1 analysis + forecast grid_tasks rows; no Oban fan-out" do

# Line 120:
# Before: assert kinds == %{"analysis" => 1, "forecast" => 18}
# After:  assert kinds == %{"analysis" => 1, "forecast" => 28}

# Line 125:
# Before: assert forecast_fhs == Enum.to_list(1..18)
# After:  assert forecast_fhs == @forecast_hours  (with the module attribute defined above)

Step 2: Run tests to verify they fail

mix test test/microwaveprop/propagation/grid_task_enqueuer_test.exs test/microwaveprop/workers/propagation_grid_worker_test.exs

Expected: tests fail because they still expect 18 forecast hours.

Step 3: Apply the GridTaskEnqueuer changes

Make the code changes listed above.

Step 4: Run tests to verify they pass

mix test test/microwaveprop/propagation/grid_task_enqueuer_test.exs test/microwaveprop/workers/propagation_grid_worker_test.exs

Expected: PASS.

Step 5: Commit

git add lib/microwaveprop/propagation/grid_task_enqueuer.ex test/microwaveprop/propagation/grid_task_enqueuer_test.exs test/microwaveprop/workers/propagation_grid_worker_test.exs
git commit -m "feat: extend grid_tasks seeding to 48h forecast horizon"

Task 2: Update propagation forecast horizon constant

Files:

  • Modify: lib/microwaveprop/propagation.ex

Changes:

# Before (line 261):
@hrrr_forecast_horizon_hours 18

# After:
@hrrr_forecast_horizon_hours 48

Update the comment on lines 258260:

# Before:
# HRRR forecast horizon: f00..f18 covers the next 18 hours from cycle
# time. Anything beyond that in the score store is a leftover from a
# stale cycle and clutters the timeline without adding information.

# After:
# HRRR forecast horizon: f00..f48 covers the next 48 hours from cycle
# time (f01-f18 hourly, f21-f48 3-hourly). Anything beyond that in the
# score store is a leftover from a stale cycle and clutters the
# timeline without adding information.

Update the @doc for hot_cache_window/0 (line 275):

# Before:
through HRRR's 18-hour forecast horizon.

# After:
through HRRR's 48-hour forecast horizon.

This single constant change automatically extends:

  • The map timeline (available_valid_times/1hot_cache_window/0)
  • The path calculator forecast chart (point_forecast/3forecast_window/2)
  • The score cache ETS retention window (hot_cache_window/0 used by NotifyListener)

Step 1: Make the change

Step 2: Run propagation tests

mix test test/microwaveprop/propagation_test.exs

Step 3: Commit

git add lib/microwaveprop/propagation.ex
git commit -m "feat: extend forecast horizon window from 18h to 48h"

Task 3: Update validation limits and UI clamps

Files:

  • Modify: lib/microwaveprop/propagation/run_timing.ex:38
  • Modify: lib/microwaveprop_web/live/rover_live.ex:185

Changes:

run_timing.ex — bump the validation ceiling and update doc:

# Before (line 2-7 doc):
# `forecast_hour` ∈ 0..18.

# After:
# `forecast_hour` ∈ 0..48.

# Before (line 38):
|> validate_number(:forecast_hour, greater_than_or_equal_to: 0, less_than_or_equal_to: 18)

# After:
|> validate_number(:forecast_hour, greater_than_or_equal_to: 0, less_than_or_equal_to: 48)

rover_live.ex — the forecast hour selector clamp:

# Before (line 185):
hour = v |> parse_int(@default_forecast_hour) |> clamp(0, 18)

# After:
hour = v |> parse_int(@default_forecast_hour) |> clamp(0, 48)

Step 1: Make the changes

No new tests needed — these are just ceiling bumps on existing validation. Existing tests exercise the validation path.

Step 2: Run affected tests

mix test test/microwaveprop/propagation/run_timing_test.exs test/microwaveprop_web/live/rover_live_test.exs

Step 3: Commit

git add lib/microwaveprop/propagation/run_timing.ex lib/microwaveprop_web/live/rover_live.ex
git commit -m "feat: bump forecast hour validation limits to 48h"

Task 4: Update comments and documentation strings

Files: ~15 files with "f01..f18", "18-hour forecast", or equivalent references in comments/docstrings.

These are cosmetic — no behavior change:

File What to update
lib/microwaveprop/workers/propagation_grid_worker.ex:8,50 "f01..f18" → "f01..f48" in comments/log
lib/microwaveprop/weather/hrrr_client.ex:190 Comment about f18 idx probe (no change needed — we still probe f18, it's still valid)
lib/microwaveprop/propagation/path_compute.ex:9,73 "18-hour forecast" → "48-hour forecast"
lib/microwaveprop/propagation/score_cache.ex:175 "18 h forward" → "48 h forward"
lib/microwaveprop/weather.ex:931,985,1297 Comment references to f01..f18 → f01..f48
lib/microwaveprop/weather/sounding_params.ex:24 "f01..f18" → "f01..f48"
lib/microwaveprop_web/router.ex:72 "18-hour forecast" → "48-hour forecast"
lib/microwaveprop_web/live/about_live.ex:81 "18-hour forecast" → "48-hour forecast"
lib/microwaveprop_web/live/api_docs_live.ex:841,845 "18-hour" → "48-hour"
lib/microwaveprop_web/live/skewt_live.ex:341 "18 forecast hours" → "48 forecast hours"
lib/microwaveprop_web/controllers/page_controller.ex:25 "18-hour" → "48-hour"
lib/microwaveprop_web/controllers/api/v1/score_controller.ex:8 "18-hour" → "48-hour"
rust/prop_grid_rs/src/pipeline.rs:1,72 "f01..f18" → "f01..f48" in doc comments
rust/prop_grid_rs/src/db.rs:20 "f01..f18" → "f01..f48" in doc comment
rust/prop_grid_rs/src/lib.rs:12 "f01..f18" → "f01..f48"
rust/prop_grid_rs/src/decoder.rs:2 "f01..f18" → "f01..f48"
rust/prop_grid_rs/src/sounding_params.rs:8 "f01..f18" → "f01..f48"
rust/prop_grid_rs/src/weather_scalar_file.rs:161 "f01..f18" → "f01..f48"
lib/microwaveprop/workers/hrdps_grid_worker.ex:25 "18 forecast hours" → update context

We update path_live.ex's forecast_chart header on line 1133 — the dynamic string already adapts from length(@forecast), but the alt text and surrounding prose says "18-Hour":

# Line 1132-1133 in path_live.ex:
# "{length(@forecast)}-Hour Propagation Forecast"
# This is already dynamic — no change needed to the label itself.
# The section comment on line 907:
# Before: <%!-- 18-Hour Forecast --%>
# After:  <%!-- 48-Hour Forecast --%>

Step 1: Apply all comment/doc updates

Step 2: Run the full test suite

mix test

Step 3: Verify compilation is clean

mix compile --warnings-as-errors
mix credo --strict

Step 4: Commit

git add -u
git commit -m "docs: update forecast horizon references from 18h to 48h"

What does NOT need to change

  • Rust pipeline (pipeline.rs, db.rs, fetcher.rs) — forecast_hour: u8 handles f48 fine (max 255). The pipeline reads whatever grid_tasks rows are seeded and processes them identically regardless of forecast hour.
  • HrrrClient.cycle_available?/1 — still probes f18 idx (still the last hourly file, and a reliable signal that the cycle is fully published).
  • ScoresFile — file format is per (band, valid_time), agnostic to forecast hour. New hours just create new files.
  • ProfilesFile — same, per-valid_time storage.
  • retain_window calls — parameterized on max_forecast_hour, already dynamic; callers pass the constant.
  • ScoreCache — horizon-aware via hot_cache_window/0, which reads the constant.
  • HRRR URL builder (hrrr_url/4) — already handles any forecast_hour integer, pads to 2 digits.
  • HrrrClient.cycle_available? — still probes f18 which is still in the set.

Performance impact

Each forecast hour costs ~34 minutes of Rust pipeline wall time. The 10 additional 3-hourly steps add ~3040 minutes to the hourly chain. Current chain runs ~60 minutes for 19 steps; this extends it to ~90100 minutes. The f01f18 results still land at the same latency; extended hours arrive later in the cycle.

Rollout

  1. Deploy the code change. The next hourly cron tick seeds 29 grid_tasks rows instead of 19.
  2. Rust picks them up and processes f21f48 alongside f01f18.
  3. Map timeline and path calculator forecast chart automatically show the extended horizon via the changed constant.
  4. No migration, no feature flag, no downtime.