feat: extend HRRR forecast horizon from 18h to 48h
HRRR publishes f21-f48 at 3-hourly intervals in addition to the hourly f01-f18 we already fetch. This extends the grid_tasks seed from 19 rows (1 analysis + 18 forecast) to 29 rows (1 analysis + 18 hourly + 10 3-hourly), and bumps the UI forecast window constant from 18h to 48h so the map timeline and path calculator forecast chart automatically show the extended horizon. No Rust logic changes needed — the pipeline is data-driven and u8 forecast_hour already handles f48.
This commit is contained in:
parent
d67186176f
commit
aab3c3736c
29 changed files with 473 additions and 72 deletions
362
docs/plans/2026-06-10-extend-hrrr-forecast-to-48h.md
Normal file
362
docs/plans/2026-06-10-extend-hrrr-forecast-to-48h.md
Normal file
|
|
@ -0,0 +1,362 @@
|
|||
# Extend HRRR Forecast Horizon to 48 Hours — Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Extend the propagation forecast from 18 hours to 48 hours using HRRR's own extended forecast hours (f21–f48 at 3-hourly intervals), without adding a new data source.
|
||||
|
||||
**Architecture:** The system is data-driven — whatever forecast hours are seeded into `grid_tasks`, Rust processes. Two constants gate the current 18-hour cap: `GridTaskEnqueuer.@max_forecast_hour` and `Propagation.@hrrr_forecast_horizon_hours`. Changing these, plus adjusting a few validation limits and comment strings, is the entire change. No Rust code changes needed.
|
||||
|
||||
**Tech Stack:** Elixir (Oban, Ecto, Phoenix LiveView), Rust (prop_grid_rs), HRRR NOAA S3
|
||||
|
||||
**Relationship to prior plan:** `docs/plans/2026-04-18-extended-horizon-forecasts.md` covers adding GFS as a separate data source for 10-day forecasts. This plan is the zero-new-infrastructure quick win — HRRR already publishes f21–f48, we just don't fetch them. The GFS plan is still the right long-term approach for multi-day planning; this extends the existing hourly chain at near-zero cost.
|
||||
|
||||
---
|
||||
|
||||
## HRRR forecast hour availability
|
||||
|
||||
| Range | Interval | Count | Notes |
|
||||
|---|---|---|---|
|
||||
| f00 | — | 1 | Analysis (Elixir-owned, unchanged) |
|
||||
| f01–f18 | 1-hourly | 18 | What we fetch today |
|
||||
| f21–f48 | 3-hourly | 10 | New: f21, f24, f27, f30, f33, f36, f39, f42, f45, f48 |
|
||||
|
||||
Total: 1 analysis + 18 hourly + 10 3-hourly = 29 `grid_tasks` rows per cycle (up from 19).
|
||||
|
||||
Accuracy degrades beyond f18 — HRRR is a 3 km convection-allowing model; small-scale features (ducting, refractivity gradients) lose fidelity at longer lead times. The 3-hourly output spacing reflects this. Accepted trade-off per user request.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Define the forecast hour list and update `GridTaskEnqueuer`
|
||||
|
||||
**Files:**
|
||||
- Modify: `lib/microwaveprop/propagation/grid_task_enqueuer.ex`
|
||||
|
||||
**Changes:**
|
||||
|
||||
Replace the `@max_forecast_hour 18` module attribute with a list of all forecast hours Rust should process:
|
||||
|
||||
```elixir
|
||||
# Before (line 19):
|
||||
@max_forecast_hour 18
|
||||
|
||||
# After:
|
||||
@forecast_hours [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 21, 24, 27, 30, 33, 36, 39, 42, 45, 48]
|
||||
```
|
||||
|
||||
Update `seed_with_analysis/2` (line 70) — the `for fh <- 1..@max_forecast_hour` comprehension:
|
||||
|
||||
```elixir
|
||||
# Before (line 70):
|
||||
for fh <- 1..@max_forecast_hour do
|
||||
|
||||
# After:
|
||||
for fh <- @forecast_hours do
|
||||
```
|
||||
|
||||
Update `seed/2` (line 103) — same pattern:
|
||||
|
||||
```elixir
|
||||
# Before (line 103):
|
||||
for fh <- 1..@max_forecast_hour do
|
||||
|
||||
# After:
|
||||
for fh <- @forecast_hours do
|
||||
```
|
||||
|
||||
Update `seed_current_hour/2` (lines 174, 192) — the clamp ceiling:
|
||||
|
||||
```elixir
|
||||
# Before (line 174):
|
||||
# Forecast hour is clamped to `1..@max_forecast_hour`.
|
||||
# (line 176):
|
||||
# `run_time + @max_forecast_hour`, we cap there.
|
||||
# (line 192):
|
||||
|> min(@max_forecast_hour)
|
||||
|
||||
# After (line 174):
|
||||
# Forecast hour is clamped to the available forecast hour list.
|
||||
# (line 176):
|
||||
# the max forecast hour, we cap there.
|
||||
# (line 192):
|
||||
|> min(Enum.max(@forecast_hours))
|
||||
```
|
||||
|
||||
Update doc comments (lines 36–37, 92):
|
||||
|
||||
```elixir
|
||||
# Before (line 36-37):
|
||||
# Seed one kind='analysis' row (f00) plus 18 kind='forecast' rows
|
||||
# (f01..f18) for `run_time`.
|
||||
|
||||
# After:
|
||||
# Seed one kind='analysis' row (f00) plus forecast rows
|
||||
# (f01..f18 hourly, f21..f48 3-hourly) for `run_time`.
|
||||
|
||||
# Before (line 92):
|
||||
# Legacy: seed only kind='forecast' rows (f01..f18).
|
||||
|
||||
# After:
|
||||
# Legacy: seed only kind='forecast' rows.
|
||||
```
|
||||
|
||||
**Step 1: Update the test to expect the new forecast hour list**
|
||||
|
||||
File: `test/microwaveprop/propagation/grid_task_enqueuer_test.exs:22`
|
||||
|
||||
```elixir
|
||||
# Before:
|
||||
assert rows |> Enum.map(& &1.forecast_hour) |> Enum.sort() == Enum.to_list(1..18)
|
||||
|
||||
# After:
|
||||
@forecast_hours [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 21, 24, 27, 30, 33, 36, 39, 42, 45, 48]
|
||||
assert rows |> Enum.map(& &1.forecast_hour) |> Enum.sort() == @forecast_hours
|
||||
```
|
||||
|
||||
Update the worker test at `test/microwaveprop/workers/propagation_grid_worker_test.exs`:
|
||||
|
||||
```elixir
|
||||
# Line 7 (comment):
|
||||
# Before: inserts 19 grid_tasks rows (1 analysis f00 + 18 forecast f01..f18)
|
||||
# After: inserts grid_tasks rows (1 analysis f00 + forecast f01..f18 hourly, f21..f48 3-hourly)
|
||||
|
||||
# Line 70:
|
||||
# Before: test "inserts 1 analysis + 18 forecast grid_tasks rows; no Oban fan-out" do
|
||||
# After: test "inserts 1 analysis + forecast grid_tasks rows; no Oban fan-out" do
|
||||
|
||||
# Line 120:
|
||||
# Before: assert kinds == %{"analysis" => 1, "forecast" => 18}
|
||||
# After: assert kinds == %{"analysis" => 1, "forecast" => 28}
|
||||
|
||||
# Line 125:
|
||||
# Before: assert forecast_fhs == Enum.to_list(1..18)
|
||||
# After: assert forecast_fhs == @forecast_hours (with the module attribute defined above)
|
||||
```
|
||||
|
||||
**Step 2: Run tests to verify they fail**
|
||||
|
||||
```bash
|
||||
mix test test/microwaveprop/propagation/grid_task_enqueuer_test.exs test/microwaveprop/workers/propagation_grid_worker_test.exs
|
||||
```
|
||||
|
||||
Expected: tests fail because they still expect 18 forecast hours.
|
||||
|
||||
**Step 3: Apply the `GridTaskEnqueuer` changes**
|
||||
|
||||
Make the code changes listed above.
|
||||
|
||||
**Step 4: Run tests to verify they pass**
|
||||
|
||||
```bash
|
||||
mix test test/microwaveprop/propagation/grid_task_enqueuer_test.exs test/microwaveprop/workers/propagation_grid_worker_test.exs
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add lib/microwaveprop/propagation/grid_task_enqueuer.ex test/microwaveprop/propagation/grid_task_enqueuer_test.exs test/microwaveprop/workers/propagation_grid_worker_test.exs
|
||||
git commit -m "feat: extend grid_tasks seeding to 48h forecast horizon"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Update propagation forecast horizon constant
|
||||
|
||||
**Files:**
|
||||
- Modify: `lib/microwaveprop/propagation.ex`
|
||||
|
||||
**Changes:**
|
||||
|
||||
```elixir
|
||||
# Before (line 261):
|
||||
@hrrr_forecast_horizon_hours 18
|
||||
|
||||
# After:
|
||||
@hrrr_forecast_horizon_hours 48
|
||||
```
|
||||
|
||||
Update the comment on lines 258–260:
|
||||
|
||||
```elixir
|
||||
# Before:
|
||||
# HRRR forecast horizon: f00..f18 covers the next 18 hours from cycle
|
||||
# time. Anything beyond that in the score store is a leftover from a
|
||||
# stale cycle and clutters the timeline without adding information.
|
||||
|
||||
# After:
|
||||
# HRRR forecast horizon: f00..f48 covers the next 48 hours from cycle
|
||||
# time (f01-f18 hourly, f21-f48 3-hourly). Anything beyond that in the
|
||||
# score store is a leftover from a stale cycle and clutters the
|
||||
# timeline without adding information.
|
||||
```
|
||||
|
||||
Update the `@doc` for `hot_cache_window/0` (line 275):
|
||||
|
||||
```elixir
|
||||
# Before:
|
||||
through HRRR's 18-hour forecast horizon.
|
||||
|
||||
# After:
|
||||
through HRRR's 48-hour forecast horizon.
|
||||
```
|
||||
|
||||
This single constant change automatically extends:
|
||||
- The map timeline (`available_valid_times/1` → `hot_cache_window/0`)
|
||||
- The path calculator forecast chart (`point_forecast/3` → `forecast_window/2`)
|
||||
- The score cache ETS retention window (`hot_cache_window/0` used by `NotifyListener`)
|
||||
|
||||
**Step 1: Make the change**
|
||||
|
||||
**Step 2: Run propagation tests**
|
||||
|
||||
```bash
|
||||
mix test test/microwaveprop/propagation_test.exs
|
||||
```
|
||||
|
||||
**Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add lib/microwaveprop/propagation.ex
|
||||
git commit -m "feat: extend forecast horizon window from 18h to 48h"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Update validation limits and UI clamps
|
||||
|
||||
**Files:**
|
||||
- Modify: `lib/microwaveprop/propagation/run_timing.ex:38`
|
||||
- Modify: `lib/microwaveprop_web/live/rover_live.ex:185`
|
||||
|
||||
**Changes:**
|
||||
|
||||
`run_timing.ex` — bump the validation ceiling and update doc:
|
||||
|
||||
```elixir
|
||||
# Before (line 2-7 doc):
|
||||
# `forecast_hour` ∈ 0..18.
|
||||
|
||||
# After:
|
||||
# `forecast_hour` ∈ 0..48.
|
||||
|
||||
# Before (line 38):
|
||||
|> validate_number(:forecast_hour, greater_than_or_equal_to: 0, less_than_or_equal_to: 18)
|
||||
|
||||
# After:
|
||||
|> validate_number(:forecast_hour, greater_than_or_equal_to: 0, less_than_or_equal_to: 48)
|
||||
```
|
||||
|
||||
`rover_live.ex` — the forecast hour selector clamp:
|
||||
|
||||
```elixir
|
||||
# Before (line 185):
|
||||
hour = v |> parse_int(@default_forecast_hour) |> clamp(0, 18)
|
||||
|
||||
# After:
|
||||
hour = v |> parse_int(@default_forecast_hour) |> clamp(0, 48)
|
||||
```
|
||||
|
||||
**Step 1: Make the changes**
|
||||
|
||||
No new tests needed — these are just ceiling bumps on existing validation. Existing tests exercise the validation path.
|
||||
|
||||
**Step 2: Run affected tests**
|
||||
|
||||
```bash
|
||||
mix test test/microwaveprop/propagation/run_timing_test.exs test/microwaveprop_web/live/rover_live_test.exs
|
||||
```
|
||||
|
||||
**Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add lib/microwaveprop/propagation/run_timing.ex lib/microwaveprop_web/live/rover_live.ex
|
||||
git commit -m "feat: bump forecast hour validation limits to 48h"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Update comments and documentation strings
|
||||
|
||||
**Files:** ~15 files with "f01..f18", "18-hour forecast", or equivalent references in comments/docstrings.
|
||||
|
||||
These are cosmetic — no behavior change:
|
||||
|
||||
| File | What to update |
|
||||
|---|---|
|
||||
| `lib/microwaveprop/workers/propagation_grid_worker.ex:8,50` | "f01..f18" → "f01..f48" in comments/log |
|
||||
| `lib/microwaveprop/weather/hrrr_client.ex:190` | Comment about f18 idx probe (no change needed — we still probe f18, it's still valid) |
|
||||
| `lib/microwaveprop/propagation/path_compute.ex:9,73` | "18-hour forecast" → "48-hour forecast" |
|
||||
| `lib/microwaveprop/propagation/score_cache.ex:175` | "18 h forward" → "48 h forward" |
|
||||
| `lib/microwaveprop/weather.ex:931,985,1297` | Comment references to f01..f18 → f01..f48 |
|
||||
| `lib/microwaveprop/weather/sounding_params.ex:24` | "f01..f18" → "f01..f48" |
|
||||
| `lib/microwaveprop_web/router.ex:72` | "18-hour forecast" → "48-hour forecast" |
|
||||
| `lib/microwaveprop_web/live/about_live.ex:81` | "18-hour forecast" → "48-hour forecast" |
|
||||
| `lib/microwaveprop_web/live/api_docs_live.ex:841,845` | "18-hour" → "48-hour" |
|
||||
| `lib/microwaveprop_web/live/skewt_live.ex:341` | "18 forecast hours" → "48 forecast hours" |
|
||||
| `lib/microwaveprop_web/controllers/page_controller.ex:25` | "18-hour" → "48-hour" |
|
||||
| `lib/microwaveprop_web/controllers/api/v1/score_controller.ex:8` | "18-hour" → "48-hour" |
|
||||
| `rust/prop_grid_rs/src/pipeline.rs:1,72` | "f01..f18" → "f01..f48" in doc comments |
|
||||
| `rust/prop_grid_rs/src/db.rs:20` | "f01..f18" → "f01..f48" in doc comment |
|
||||
| `rust/prop_grid_rs/src/lib.rs:12` | "f01..f18" → "f01..f48" |
|
||||
| `rust/prop_grid_rs/src/decoder.rs:2` | "f01..f18" → "f01..f48" |
|
||||
| `rust/prop_grid_rs/src/sounding_params.rs:8` | "f01..f18" → "f01..f48" |
|
||||
| `rust/prop_grid_rs/src/weather_scalar_file.rs:161` | "f01..f18" → "f01..f48" |
|
||||
| `lib/microwaveprop/workers/hrdps_grid_worker.ex:25` | "18 forecast hours" → update context |
|
||||
|
||||
We update `path_live.ex`'s `forecast_chart` header on line 1133 — the dynamic string already adapts from `length(@forecast)`, but the alt text and surrounding prose says "18-Hour":
|
||||
|
||||
```elixir
|
||||
# Line 1132-1133 in path_live.ex:
|
||||
# "{length(@forecast)}-Hour Propagation Forecast"
|
||||
# This is already dynamic — no change needed to the label itself.
|
||||
# The section comment on line 907:
|
||||
# Before: <%!-- 18-Hour Forecast --%>
|
||||
# After: <%!-- 48-Hour Forecast --%>
|
||||
```
|
||||
|
||||
**Step 1: Apply all comment/doc updates**
|
||||
|
||||
**Step 2: Run the full test suite**
|
||||
|
||||
```bash
|
||||
mix test
|
||||
```
|
||||
|
||||
**Step 3: Verify compilation is clean**
|
||||
|
||||
```bash
|
||||
mix compile --warnings-as-errors
|
||||
mix credo --strict
|
||||
```
|
||||
|
||||
**Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add -u
|
||||
git commit -m "docs: update forecast horizon references from 18h to 48h"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## What does NOT need to change
|
||||
|
||||
- **Rust pipeline** (`pipeline.rs`, `db.rs`, `fetcher.rs`) — `forecast_hour: u8` handles f48 fine (max 255). The pipeline reads whatever `grid_tasks` rows are seeded and processes them identically regardless of forecast hour.
|
||||
- **`HrrrClient.cycle_available?/1`** — still probes f18 idx (still the last hourly file, and a reliable signal that the cycle is fully published).
|
||||
- **`ScoresFile`** — file format is per `(band, valid_time)`, agnostic to forecast hour. New hours just create new files.
|
||||
- **`ProfilesFile`** — same, per-`valid_time` storage.
|
||||
- **`retain_window` calls** — parameterized on `max_forecast_hour`, already dynamic; callers pass the constant.
|
||||
- **`ScoreCache`** — horizon-aware via `hot_cache_window/0`, which reads the constant.
|
||||
- **HRRR URL builder** (`hrrr_url/4`) — already handles any `forecast_hour` integer, pads to 2 digits.
|
||||
- **`HrrrClient.cycle_available?`** — still probes f18 which is still in the set.
|
||||
|
||||
## Performance impact
|
||||
|
||||
Each forecast hour costs ~3–4 minutes of Rust pipeline wall time. The 10 additional 3-hourly steps add ~30–40 minutes to the hourly chain. Current chain runs ~60 minutes for 19 steps; this extends it to ~90–100 minutes. The f01–f18 results still land at the same latency; extended hours arrive later in the cycle.
|
||||
|
||||
## Rollout
|
||||
|
||||
1. Deploy the code change. The next hourly cron tick seeds 29 `grid_tasks` rows instead of 19.
|
||||
2. Rust picks them up and processes f21–f48 alongside f01–f18.
|
||||
3. Map timeline and path calculator forecast chart automatically show the extended horizon via the changed constant.
|
||||
4. No migration, no feature flag, no downtime.
|
||||
|
|
@ -255,10 +255,11 @@ defmodule Microwaveprop.Propagation do
|
|||
the past, but always includes the most recent valid_time so there's
|
||||
always data to display.
|
||||
"""
|
||||
# HRRR forecast horizon: f00..f18 covers the next 18 hours from cycle
|
||||
# time. Anything beyond that in the score store is a leftover from a
|
||||
# stale cycle and clutters the timeline without adding information.
|
||||
@hrrr_forecast_horizon_hours 18
|
||||
# HRRR forecast horizon: f00..f48 covers the next 48 hours from cycle
|
||||
# time (f01-f18 hourly, f21-f48 3-hourly). Anything beyond that in the
|
||||
# score store is a leftover from a stale cycle and clutters the
|
||||
# timeline without adding information.
|
||||
@hrrr_forecast_horizon_hours 48
|
||||
|
||||
@spec available_valid_times(non_neg_integer()) :: [DateTime.t()]
|
||||
def available_valid_times(band_mhz) do
|
||||
|
|
@ -272,7 +273,7 @@ defmodule Microwaveprop.Propagation do
|
|||
|
||||
@doc """
|
||||
The active forecast window for the `/map` UI: one hour in the past
|
||||
through HRRR's 18-hour forecast horizon. Used by `NotifyListener` to
|
||||
through HRRR's 48-hour forecast horizon. Used by `NotifyListener` to
|
||||
bound ETS growth — long-horizon GEFS `.prop` files on disk must not
|
||||
balloon `propagation_score_cache` past the memory the UI actually
|
||||
reads.
|
||||
|
|
|
|||
|
|
@ -16,7 +16,9 @@ defmodule Microwaveprop.Propagation.GridTaskEnqueuer do
|
|||
|
||||
require Logger
|
||||
|
||||
@max_forecast_hour 18
|
||||
# HRRR forecast hours: f01-f18 hourly, then f21-f48 3-hourly.
|
||||
# Rust processes whatever rows are seeded — no code changes needed there.
|
||||
@forecast_hours [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 21, 24, 27, 30, 33, 36, 39, 42, 45, 48]
|
||||
|
||||
# A `running` row older than this is an orphan: the Rust worker that
|
||||
# claimed it was SIGKILLed / OOMed / evicted before it could emit a
|
||||
|
|
@ -33,9 +35,10 @@ defmodule Microwaveprop.Propagation.GridTaskEnqueuer do
|
|||
@max_reclaim_attempts 5
|
||||
|
||||
@doc """
|
||||
Seed one kind='analysis' row (f00) plus 18 kind='forecast' rows
|
||||
(f01..f18) for `run_time`. This is the Phase 3 Stream A cutover
|
||||
shape: Rust owns the entire chain end-to-end.
|
||||
Seed one kind='analysis' row (f00) plus kind='forecast' rows
|
||||
(f01..f18 hourly, f21..f48 3-hourly) for `run_time`. This is the
|
||||
Phase 3 Stream A cutover shape: Rust owns the entire chain
|
||||
end-to-end.
|
||||
|
||||
## Options
|
||||
|
||||
|
|
@ -67,7 +70,7 @@ defmodule Microwaveprop.Propagation.GridTaskEnqueuer do
|
|||
}
|
||||
|
||||
forecast_rows =
|
||||
for fh <- 1..@max_forecast_hour do
|
||||
for fh <- @forecast_hours do
|
||||
%{
|
||||
id: Ecto.UUID.bingenerate(),
|
||||
run_time: run_time,
|
||||
|
|
@ -89,7 +92,7 @@ defmodule Microwaveprop.Propagation.GridTaskEnqueuer do
|
|||
end
|
||||
|
||||
@doc """
|
||||
Legacy: seed only kind='forecast' rows (f01..f18). Retained for
|
||||
Legacy: seed only kind='forecast' rows. Retained for
|
||||
tooling that needs to re-seed the forecast lane without touching
|
||||
the analysis row. `seed_with_analysis/2` is the production path.
|
||||
"""
|
||||
|
|
@ -100,7 +103,7 @@ defmodule Microwaveprop.Propagation.GridTaskEnqueuer do
|
|||
now = DateTime.truncate(DateTime.utc_now(), :microsecond)
|
||||
|
||||
rows =
|
||||
for fh <- 1..@max_forecast_hour do
|
||||
for fh <- @forecast_hours do
|
||||
%{
|
||||
id: Ecto.UUID.bingenerate(),
|
||||
run_time: run_time,
|
||||
|
|
@ -171,9 +174,9 @@ defmodule Microwaveprop.Propagation.GridTaskEnqueuer do
|
|||
the f01..f48 chain isn't worth the wgrib2 cost (each HRDPS chain step
|
||||
is ~20 min wall time at the production point density).
|
||||
|
||||
Forecast hour is clamped to `1..@max_forecast_hour`. When `now` lands
|
||||
before `run_time + 1h`, we still seed `fh=1`; when it's past
|
||||
`run_time + @max_forecast_hour`, we cap there. The Rust worker rejects
|
||||
Forecast hour is clamped to the available forecast hour list. When
|
||||
`now` lands before `run_time + 1h`, we still seed `fh=1`; when it's
|
||||
past the max forecast hour, we cap there. The Rust worker rejects
|
||||
`forecast_hour=0` (`F00Reserved`) so we never seed the analysis hour
|
||||
through this path.
|
||||
"""
|
||||
|
|
@ -189,7 +192,7 @@ defmodule Microwaveprop.Propagation.GridTaskEnqueuer do
|
|||
|> DateTime.diff(run_time, :second)
|
||||
|> div(3600)
|
||||
|> max(1)
|
||||
|> min(@max_forecast_hour)
|
||||
|> min(Enum.max(@forecast_hours))
|
||||
|
||||
row = %{
|
||||
id: Ecto.UUID.bingenerate(),
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ defmodule Microwaveprop.Propagation.PathCompute do
|
|||
same pipeline `/path` does: terrain analysis (ITU-R P.526), HRRR
|
||||
profile lookup at 9 evenly-spaced points, sounding & ionosphere
|
||||
readouts, native HRRR duct info, composite scoring, loss + power
|
||||
budgets, and the 18-hour propagation forecast at the path midpoint.
|
||||
budgets, and the 48-hour propagation forecast at the path midpoint.
|
||||
|
||||
Used by both:
|
||||
* the live `MicrowavepropWeb.PathLive` page (live recompute)
|
||||
|
|
@ -70,7 +70,7 @@ defmodule Microwaveprop.Propagation.PathCompute do
|
|||
"Loading sounding & duct data",
|
||||
"Scoring propagation conditions",
|
||||
"Computing link & power budget",
|
||||
"Loading 18-hour forecast",
|
||||
"Loading 48-hour forecast",
|
||||
"Loading ionosphere data"
|
||||
]
|
||||
@total_stages length(@stages)
|
||||
|
|
|
|||
|
|
@ -298,7 +298,7 @@ defmodule Microwaveprop.Propagation.ProfilesFile do
|
|||
|
||||
@doc """
|
||||
Every persisted valid_time, sorted ascending. Used to build the
|
||||
/weather forecast timeline from the on-disk f00..f18 profile files.
|
||||
/weather forecast timeline from the on-disk f00..f48 profile files.
|
||||
"""
|
||||
@spec list_valid_times() :: [DateTime.t()]
|
||||
def list_valid_times do
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ defmodule Microwaveprop.Propagation.RunTiming do
|
|||
how long that hour's fetch + score + persist cycle took.
|
||||
|
||||
Rows are keyed by `(run_time, forecast_hour)`. `run_time` is the HRRR
|
||||
model cycle and `forecast_hour` ∈ 0..18.
|
||||
model cycle and `forecast_hour` ∈ 0..48.
|
||||
"""
|
||||
use Ecto.Schema
|
||||
|
||||
|
|
@ -35,7 +35,7 @@ defmodule Microwaveprop.Propagation.RunTiming do
|
|||
row
|
||||
|> cast(attrs, @required ++ @optional)
|
||||
|> validate_required(@required)
|
||||
|> validate_number(:forecast_hour, greater_than_or_equal_to: 0, less_than_or_equal_to: 18)
|
||||
|> validate_number(:forecast_hour, greater_than_or_equal_to: 0, less_than_or_equal_to: 48)
|
||||
|> validate_number(:duration_ms, greater_than_or_equal_to: 0)
|
||||
|> unique_constraint([:run_time, :forecast_hour])
|
||||
end
|
||||
|
|
|
|||
|
|
@ -172,7 +172,7 @@ defmodule Microwaveprop.Propagation.ScoreCache do
|
|||
@doc """
|
||||
Delete every entry whose `valid_time` falls outside the inclusive
|
||||
window `[past_cutoff, future_cutoff]`. Used to bound the cache to
|
||||
the active forecast horizon (past ~1 h + HRRR's 18 h forward) so
|
||||
the active forecast horizon (past ~1 h + HRRR's 48 h forward) so
|
||||
long-horizon GEFS valid_times never accumulate in ETS.
|
||||
"""
|
||||
@spec prune_outside_window(DateTime.t(), DateTime.t()) :: non_neg_integer()
|
||||
|
|
|
|||
|
|
@ -928,7 +928,7 @@ defmodule Microwaveprop.Weather do
|
|||
|
||||
@doc """
|
||||
All persisted weather valid_times sorted ascending. The grid worker
|
||||
writes a ProfilesFile for every forecast hour (f00..f18), and the
|
||||
writes a ProfilesFile for every forecast hour (f00..f18 hourly, f21..f48 3-hourly), and the
|
||||
derived `ScalarFile` mirrors that for any hour that has been
|
||||
materialized. We union both so the timeline survives an aggressive
|
||||
retention sweep on either side as long as one artifact remains.
|
||||
|
|
@ -982,7 +982,7 @@ defmodule Microwaveprop.Weather do
|
|||
Returns `[]` if no profile file exists for that valid_time.
|
||||
|
||||
Deliberately does NOT write forecast-hour grids back into `GridCache`
|
||||
on a miss: caching 18 forecast hours × 92k points would add ~300 MB
|
||||
on a miss: caching 48 forecast hours × 92k points would add ~300 MB
|
||||
per pod. The ProfilesFile read is a single ~2 MB ETF decode per scrub,
|
||||
which is fast enough for a user click.
|
||||
"""
|
||||
|
|
@ -1294,7 +1294,7 @@ defmodule Microwaveprop.Weather do
|
|||
# unchanged. `surface_refractivity` and `min_refractivity_gradient`
|
||||
# aren't persisted by Rust — they flow through from the per-cell
|
||||
# sounding derivation; `refractivity_gradient` falls back to the
|
||||
# `native_min_gradient` scalar the Rust f01..f18 pipeline does write.
|
||||
# `native_min_gradient` scalar the Rust f01..f48 pipeline does write.
|
||||
sounding = derive_sounding(profile[:profile])
|
||||
|
||||
row = %{
|
||||
|
|
|
|||
|
|
@ -21,7 +21,7 @@ defmodule Microwaveprop.Weather.SoundingParams do
|
|||
* Legacy shape — string keys `"pres"` / `"hght"` / `"tmpc"` / `"dwpc"`
|
||||
(Elixir's HrrrClient, NARR, GEFS, UWYO clients).
|
||||
* Rust shape — `"pres_mb"` / `"hght_m"` / `"tmpc"` / `"dwpc"` (string
|
||||
or atom keys from the Rust hrrr_points worker and the f01..f18
|
||||
or atom keys from the Rust hrrr_points worker and the f01..f48
|
||||
ProfilesFile cells).
|
||||
|
||||
Both are normalized at entry to the canonical legacy shape so the
|
||||
|
|
|
|||
|
|
@ -22,7 +22,7 @@ defmodule Microwaveprop.Workers.HrdpsGridWorker do
|
|||
|
||||
Seeds exactly one forecast row per fire — the one whose valid_time
|
||||
matches the current hour. The Canadian propagation/weather UI only
|
||||
surfaces the present moment, so producing 18 forecast hours per cycle
|
||||
surfaces the present moment, so producing forecast hours per cycle
|
||||
was wasted wgrib2 work (~20 min/chain step at production point
|
||||
density). If forecast hours come back into scope later, swap the
|
||||
enqueuer call back to `GridTaskEnqueuer.seed_with_analysis/2`.
|
||||
|
|
|
|||
|
|
@ -4,8 +4,8 @@ defmodule Microwaveprop.Workers.PropagationGridWorker do
|
|||
|
||||
Post-Phase-3-cutover, Elixir no longer runs any fetch/decode/score
|
||||
work for the propagation grid. The hourly cron fires this worker
|
||||
with empty args, and it inserts 19 `grid_tasks` rows (1 analysis
|
||||
f00 + 18 forecast f01..f18) for Rust to drain. Rust owns everything
|
||||
with empty args, and it inserts grid_tasks rows (1 analysis
|
||||
f00 + forecast f01..f18 hourly, f21..f48 3-hourly) for Rust to drain. Rust owns everything
|
||||
from there: HRRR fetch, wgrib2 decode, native-level duct merge,
|
||||
NEXRAD composite, commercial-link degradation, band scoring, and
|
||||
both the ProfilesFile (MessagePack) and the per-band score files.
|
||||
|
|
@ -47,7 +47,9 @@ defmodule Microwaveprop.Workers.PropagationGridWorker do
|
|||
|
||||
run_time = pick_run_time(DateTime.utc_now())
|
||||
|
||||
Logger.info("PropagationGrid: seeding chain run_time=#{run_time} (f00 analysis + f01-f18 forecasts via grid_tasks)")
|
||||
Logger.info(
|
||||
"PropagationGrid: seeding chain run_time=#{run_time} (f00 analysis + f01-f18 hourly, f21-f48 3-hourly forecasts via grid_tasks)"
|
||||
)
|
||||
|
||||
case GridTaskEnqueuer.seed_with_analysis(run_time) do
|
||||
{:ok, count} ->
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ defmodule MicrowavepropWeb.Api.V1.ScoreController do
|
|||
* `GET /api/v1/scores?band=10000&lat=32.9&lon=-97.0&valid_time=ISO`
|
||||
returns the score + factor breakdown at a single grid point.
|
||||
* `GET /api/v1/forecast?band=10000&lat=32.9&lon=-97.0`
|
||||
returns the 18-hour score timeline at that point.
|
||||
returns the 48-hour score timeline at that point.
|
||||
* `GET /api/v1/scores/bands` returns the bands the engine scores for.
|
||||
"""
|
||||
|
||||
|
|
|
|||
|
|
@ -22,7 +22,7 @@ defmodule MicrowavepropWeb.PageController do
|
|||
|
||||
## Tools and maps
|
||||
|
||||
- [Propagation map](/map) — real-time CONUS propagation scores with 18-hour forecast overlay.
|
||||
- [Propagation map](/map) — real-time CONUS propagation scores with 48-hour forecast overlay.
|
||||
- [Weather map](/weather) — HRRR weather data overlay (temperature, dewpoint, refractivity, ducting).
|
||||
- [Weather map (Canada)](/weather-ca) — HRDPS/RDPS-based weather for Canadian coverage.
|
||||
- [Contact map](/contacts/map) — geographic view of 58,000+ recorded QSOs.
|
||||
|
|
|
|||
|
|
@ -78,7 +78,7 @@ defmodule MicrowavepropWeb.AboutLive do
|
|||
water for every grid cell. ASOS surface obs, 12-hourly upper-air
|
||||
soundings, and gridded IEMRE reanalysis fill in the gaps. A
|
||||
10-factor composite score is written out for every 0.125° cell
|
||||
across CONUS, for each hour of an 18-hour forecast.
|
||||
across CONUS, for each hour of a 48-hour forecast.
|
||||
</li>
|
||||
<li>
|
||||
<strong>The contacts.</strong>
|
||||
|
|
|
|||
|
|
@ -838,11 +838,11 @@ defmodule MicrowavepropWeb.ApiDocsLive do
|
|||
</.endpoint>
|
||||
|
||||
<.endpoint id="forecast" method="GET" path="/forecast">
|
||||
<:summary>18-hour score timeline</:summary>
|
||||
<:summary>48-hour score timeline</:summary>
|
||||
<:body>
|
||||
<p>
|
||||
Returns the score at one grid point across the available
|
||||
forecast horizon (currently 18 hours). Same <code>band</code>
|
||||
forecast horizon (currently 48 hours). Same <code>band</code>
|
||||
+ <code>lat</code>
|
||||
+ <code>lon</code>
|
||||
parameters as <code>/scores</code>.
|
||||
|
|
|
|||
|
|
@ -904,7 +904,7 @@ defmodule MicrowavepropWeb.PathLive do
|
|||
<.ionosphere_readout readout={@result.ionosphere} band_mhz={@result.band_mhz} />
|
||||
<% end %>
|
||||
|
||||
<%!-- 18-Hour Forecast --%>
|
||||
<%!-- 48-Hour Forecast --%>
|
||||
<%= if @result.forecast != [] do %>
|
||||
<.forecast_chart forecast={@result.forecast} band_config={@result.band_config} />
|
||||
<% end %>
|
||||
|
|
|
|||
|
|
@ -182,7 +182,7 @@ defmodule MicrowavepropWeb.RoverLive do
|
|||
end
|
||||
|
||||
def handle_event("set_forecast_hour", %{"value" => v}, socket) do
|
||||
hour = v |> parse_int(@default_forecast_hour) |> clamp(0, 18)
|
||||
hour = v |> parse_int(@default_forecast_hour) |> clamp(0, 48)
|
||||
{:noreply, assign(socket, forecast_hour: hour)}
|
||||
end
|
||||
|
||||
|
|
|
|||
|
|
@ -338,7 +338,7 @@ defmodule MicrowavepropWeb.SkewtLive do
|
|||
end
|
||||
|
||||
defp available_valid_times(lat, lon) do
|
||||
# The chain produces analysis + 18 forecast hours every wall-clock
|
||||
# The chain produces analysis + forecast hours every wall-clock
|
||||
# hour. By the time the next chain finishes the previous chain's
|
||||
# analysis is up to 70 min old, and during a missed cycle it can be
|
||||
# ~2 h old. A 1-hour past cutoff would leave the page empty while
|
||||
|
|
|
|||
|
|
@ -69,7 +69,7 @@ defmodule MicrowavepropWeb.Router do
|
|||
|
||||
## Primary Resources
|
||||
|
||||
- [Propagation map](/map) — real-time CONUS propagation scores + 18-hour forecast.
|
||||
- [Propagation map](/map) — real-time CONUS propagation scores + 48-hour forecast.
|
||||
- [Algorithm documentation](/algo) — full methodology, factor weights, ITU-R references (also available as `text/markdown`).
|
||||
- [Contact database](/contacts) — 58,000+ historical QSOs used for calibration.
|
||||
- [Contact map](/contacts/map) — geographic view of all recorded contacts.
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@ pub const NOTIFY_CHANNEL: &str = "propagation_ready";
|
|||
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum TaskKind {
|
||||
/// f01..f18 — fetch + decode + score the forecast-hour grid. This is
|
||||
/// f01..f48 — fetch + decode + score the forecast-hour grid. This is
|
||||
/// what Rust has owned since Phase 2 cutover.
|
||||
Forecast,
|
||||
/// f00 enrichment — adds native-level duct merge, NEXRAD composite,
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
//! wgrib2 subprocess wrapper. 1:1 port of the hot paths in
|
||||
//! `lib/microwaveprop/weather/grib2/wgrib2.ex` used by f01..f18:
|
||||
//! `lib/microwaveprop/weather/grib2/wgrib2.ex` used by f01..f48:
|
||||
//! * `extract_grid` (write GRIB2 to tmp → `wgrib2 -lola … bin` → parse)
|
||||
//! * inventory parsing from stdout
|
||||
//! * `parse_lola_binary` (Fortran-unformatted IEEE 754 little-endian f32)
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@
|
|||
//! tests can compare behavior 1:1. Module boundaries chosen so failures
|
||||
//! during TDD fall into the smallest possible blast radius.
|
||||
//!
|
||||
//! Post-Phase-3: f00 analysis and f01..f18 forecasts both live here.
|
||||
//! Post-Phase-3: f00 analysis and f01..f48 forecasts both live here.
|
||||
//! Port targets:
|
||||
//! band_config ← lib/microwaveprop/propagation/band_config.ex
|
||||
//! region ← lib/microwaveprop/propagation/region.ex
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
//! End-to-end f01..f18 chain step: fetch surface + pressure GRIBs, decode,
|
||||
//! End-to-end f01..f48 chain step: fetch surface + pressure GRIBs, decode,
|
||||
//! build per-cell `Conditions`, score every band, write score-grid files
|
||||
//! (`<band>/<iso>.prop`) to the shared scores directory.
|
||||
//!
|
||||
|
|
@ -69,7 +69,7 @@ pub struct ChainStepStats {
|
|||
pub point_count: u32,
|
||||
pub band_count: u32,
|
||||
/// Number of cells written to the per-valid_time profile file.
|
||||
/// Always `point_count` after the f01..f18 profile-file write was
|
||||
/// Always `point_count` after the f01..f48 profile-file write was
|
||||
/// added on 2026-04-25 — kept as a separate field so the log line
|
||||
/// is symmetric with the analysis step's `profile_cells_written`
|
||||
/// in `RunStepStats`, and so a future failure mode that writes
|
||||
|
|
@ -1026,7 +1026,7 @@ pub async fn run_analysis_step(
|
|||
|
||||
// Same wire format as Elixir's `ScalarFile`. See run_chain_step
|
||||
// (forecast hours) for the full rationale — both code paths go
|
||||
// through the same writer so f00 and f01..f18 land identical
|
||||
// through the same writer so f00 and f01..f48 land identical
|
||||
// chunked artifacts on NFS.
|
||||
let scores_dir_for_scalars = scores_dir_owned.clone();
|
||||
let scalar_future = {
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@
|
|||
//!
|
||||
//! The full Elixir module derives CAPE/LI/K-index/ducts — those are
|
||||
//! used for contact enrichment and f00's native-duct merge, neither of
|
||||
//! which is in the Rust f01..f18 scope.
|
||||
//! which is in the Rust f01..f48 scope.
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Level {
|
||||
|
|
|
|||
|
|
@ -158,7 +158,7 @@ pub fn derive_row(
|
|||
let inversion_strength = compute_inversion_strength(&sorted);
|
||||
let inversion_base_m = compute_inversion_base_m(&sorted);
|
||||
|
||||
// Ducts: the f01..f18 pipeline only stores the duct-summary scalars
|
||||
// Ducts: the f01..f48 pipeline only stores the duct-summary scalars
|
||||
// (`max_duct_thickness_m`, `duct_count`, `best_duct_freq_ghz`) per
|
||||
// cell. Duct base requires the full Duct list, which isn't threaded
|
||||
// through `CellValues` yet — matches the current Elixir behavior on
|
||||
|
|
|
|||
|
|
@ -6,9 +6,9 @@ defmodule Microwaveprop.Propagation.GridTaskEnqueuerTest do
|
|||
alias Microwaveprop.Propagation.GridTaskEnqueuer
|
||||
alias Microwaveprop.Repo
|
||||
|
||||
test "seeds one row per fh=1..18" do
|
||||
test "seeds one row per forecast hour" do
|
||||
run_time = ~U[2026-04-19 15:00:00Z]
|
||||
assert {:ok, 18} = GridTaskEnqueuer.seed(run_time)
|
||||
assert {:ok, 28} = GridTaskEnqueuer.seed(run_time)
|
||||
|
||||
rows =
|
||||
Repo.all(
|
||||
|
|
@ -18,8 +18,9 @@ defmodule Microwaveprop.Propagation.GridTaskEnqueuerTest do
|
|||
)
|
||||
)
|
||||
|
||||
assert length(rows) == 18
|
||||
assert rows |> Enum.map(& &1.forecast_hour) |> Enum.sort() == Enum.to_list(1..18)
|
||||
expected_fhs = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 21, 24, 27, 30, 33, 36, 39, 42, 45, 48]
|
||||
assert length(rows) == 28
|
||||
assert rows |> Enum.map(& &1.forecast_hour) |> Enum.sort() == expected_fhs
|
||||
assert Enum.all?(rows, &(&1.status == "queued"))
|
||||
|
||||
# Raw schemaless selects return NaiveDateTime; compare against the
|
||||
|
|
@ -34,17 +35,17 @@ defmodule Microwaveprop.Propagation.GridTaskEnqueuerTest do
|
|||
|
||||
test "is idempotent: reseeding the same run_time inserts nothing" do
|
||||
run_time = ~U[2026-04-19 16:00:00Z]
|
||||
assert {:ok, 18} = GridTaskEnqueuer.seed(run_time)
|
||||
assert {:ok, 28} = GridTaskEnqueuer.seed(run_time)
|
||||
assert {:ok, 0} = GridTaskEnqueuer.seed(run_time)
|
||||
|
||||
count = Repo.one(from(g in "grid_tasks", where: g.run_time == ^run_time, select: count()))
|
||||
|
||||
assert count == 18
|
||||
assert count == 28
|
||||
end
|
||||
|
||||
test "never emits a row for fh=0 — Elixir keeps that chain step" do
|
||||
run_time = ~U[2026-04-19 17:00:00Z]
|
||||
assert {:ok, 18} = GridTaskEnqueuer.seed(run_time)
|
||||
assert {:ok, 28} = GridTaskEnqueuer.seed(run_time)
|
||||
|
||||
f0_count = Repo.one(from(g in "grid_tasks", where: g.run_time == ^run_time and g.forecast_hour == 0, select: count()))
|
||||
|
||||
|
|
@ -54,7 +55,7 @@ defmodule Microwaveprop.Propagation.GridTaskEnqueuerTest do
|
|||
describe "source-aware seeding" do
|
||||
test "seed/2 defaults to source='hrrr'" do
|
||||
run_time = ~U[2026-04-29 12:00:00Z]
|
||||
assert {:ok, 18} = GridTaskEnqueuer.seed(run_time)
|
||||
assert {:ok, 28} = GridTaskEnqueuer.seed(run_time)
|
||||
|
||||
sources =
|
||||
Repo.all(from(g in "grid_tasks", where: g.run_time == ^run_time, select: g.source))
|
||||
|
|
@ -64,27 +65,27 @@ defmodule Microwaveprop.Propagation.GridTaskEnqueuerTest do
|
|||
|
||||
test "seed_with_analysis/2 with source='hrdps' tags every row" do
|
||||
run_time = ~U[2026-04-29 12:00:00Z]
|
||||
assert {:ok, 19} = GridTaskEnqueuer.seed_with_analysis(run_time, source: "hrdps")
|
||||
assert {:ok, 29} = GridTaskEnqueuer.seed_with_analysis(run_time, source: "hrdps")
|
||||
|
||||
sources =
|
||||
Repo.all(from(g in "grid_tasks", where: g.run_time == ^run_time, select: g.source))
|
||||
|
||||
assert length(sources) == 19
|
||||
assert length(sources) == 29
|
||||
assert Enum.uniq(sources) == ["hrdps"]
|
||||
end
|
||||
|
||||
test "HRRR and HRDPS chains for the same run_time coexist (4-column unique key)" do
|
||||
run_time = ~U[2026-04-29 18:00:00Z]
|
||||
assert {:ok, 19} = GridTaskEnqueuer.seed_with_analysis(run_time, source: "hrrr")
|
||||
assert {:ok, 19} = GridTaskEnqueuer.seed_with_analysis(run_time, source: "hrdps")
|
||||
assert {:ok, 29} = GridTaskEnqueuer.seed_with_analysis(run_time, source: "hrrr")
|
||||
assert {:ok, 29} = GridTaskEnqueuer.seed_with_analysis(run_time, source: "hrdps")
|
||||
|
||||
total = Repo.one(from(g in "grid_tasks", where: g.run_time == ^run_time, select: count()))
|
||||
assert total == 38
|
||||
assert total == 58
|
||||
end
|
||||
|
||||
test "reseeding the same run_time + source is idempotent" do
|
||||
run_time = ~U[2026-04-29 06:00:00Z]
|
||||
assert {:ok, 19} = GridTaskEnqueuer.seed_with_analysis(run_time, source: "hrdps")
|
||||
assert {:ok, 29} = GridTaskEnqueuer.seed_with_analysis(run_time, source: "hrdps")
|
||||
assert {:ok, 0} = GridTaskEnqueuer.seed_with_analysis(run_time, source: "hrdps")
|
||||
end
|
||||
end
|
||||
|
|
|
|||
|
|
@ -16,7 +16,7 @@ defmodule Microwaveprop.Propagation.UntestedFunctionsTest do
|
|||
assert_in_delta diff_s, 3600, 60
|
||||
|
||||
diff_future_s = DateTime.diff(future, now)
|
||||
assert_in_delta diff_future_s, 18 * 3600, 60
|
||||
assert_in_delta diff_future_s, 48 * 3600, 60
|
||||
end
|
||||
end
|
||||
|
||||
|
|
|
|||
|
|
@ -854,10 +854,10 @@ defmodule Microwaveprop.PropagationTest do
|
|||
assert Propagation.available_valid_times(10_000) == [valid_time]
|
||||
end
|
||||
|
||||
test "caps the forward horizon to 18 hours so stale cycles don't clutter the timeline" do
|
||||
test "caps the forward horizon to 48 hours so stale cycles don't clutter the timeline" do
|
||||
now = DateTime.truncate(DateTime.utc_now(), :second)
|
||||
in_range = DateTime.add(now, 3600, :second)
|
||||
past_hrrr_horizon = DateTime.add(now, 25 * 3600, :second)
|
||||
past_hrrr_horizon = DateTime.add(now, 55 * 3600, :second)
|
||||
|
||||
Enum.each([{in_range, 70}, {past_hrrr_horizon, 50}], fn {t, score} ->
|
||||
Propagation.replace_scores(
|
||||
|
|
|
|||
|
|
@ -4,10 +4,11 @@ defmodule Microwaveprop.Workers.PropagationGridWorkerTest do
|
|||
|
||||
Elixir no longer runs the chain step; the hourly cron fires
|
||||
`perform(%Oban.Job{args: %{}})` with empty args and the worker
|
||||
inserts 19 grid_tasks rows (1 analysis f00 + 18 forecast f01..f18)
|
||||
for the Rust `prop-grid-rs` worker to drain. Rust owns HRRR fetch,
|
||||
wgrib2 decode, native duct, NEXRAD, commercial, scoring, and both
|
||||
the ProfilesFile and band score-file writes from here on out.
|
||||
inserts grid_tasks rows (1 analysis f00 + forecast f01..f18 hourly,
|
||||
f21..f48 3-hourly) for the Rust `prop-grid-rs` worker to drain.
|
||||
Rust owns HRRR fetch, wgrib2 decode, native duct, NEXRAD,
|
||||
commercial, scoring, and both the ProfilesFile and band score-file
|
||||
writes from here on out.
|
||||
"""
|
||||
use Microwaveprop.DataCase, async: false
|
||||
use Oban.Testing, repo: Microwaveprop.Repo
|
||||
|
|
@ -67,7 +68,7 @@ defmodule Microwaveprop.Workers.PropagationGridWorkerTest do
|
|||
end
|
||||
|
||||
describe "perform/1 — chain seeding (empty args)" do
|
||||
test "inserts 1 analysis + 18 forecast grid_tasks rows; no Oban fan-out" do
|
||||
test "inserts 1 analysis + forecast grid_tasks rows; no Oban fan-out" do
|
||||
Oban.Testing.with_testing_mode(:manual, fn ->
|
||||
import Ecto.Query
|
||||
|
||||
|
|
@ -116,13 +117,44 @@ defmodule Microwaveprop.Workers.PropagationGridWorkerTest do
|
|||
order_by: [t.kind, t.forecast_hour]
|
||||
)
|
||||
|
||||
expected_fhs = [
|
||||
1,
|
||||
2,
|
||||
3,
|
||||
4,
|
||||
5,
|
||||
6,
|
||||
7,
|
||||
8,
|
||||
9,
|
||||
10,
|
||||
11,
|
||||
12,
|
||||
13,
|
||||
14,
|
||||
15,
|
||||
16,
|
||||
17,
|
||||
18,
|
||||
21,
|
||||
24,
|
||||
27,
|
||||
30,
|
||||
33,
|
||||
36,
|
||||
39,
|
||||
42,
|
||||
45,
|
||||
48
|
||||
]
|
||||
|
||||
kinds = rows |> Enum.map(& &1.kind) |> Enum.frequencies()
|
||||
assert kinds == %{"analysis" => 1, "forecast" => 18}
|
||||
assert kinds == %{"analysis" => 1, "forecast" => 28}
|
||||
|
||||
forecast_fhs =
|
||||
rows |> Enum.filter(&(&1.kind == "forecast")) |> Enum.map(& &1.fh) |> Enum.sort()
|
||||
|
||||
assert forecast_fhs == Enum.to_list(1..18)
|
||||
assert forecast_fhs == expected_fhs
|
||||
end)
|
||||
end
|
||||
end
|
||||
|
|
@ -159,7 +191,7 @@ defmodule Microwaveprop.Workers.PropagationGridWorkerTest do
|
|||
|
||||
assert_receive {:telemetry, [:microwaveprop, :propagation, :grid_worker, :seeded], %{queued_tasks: count}, _}
|
||||
|
||||
assert count == 19
|
||||
assert count == 29
|
||||
after
|
||||
:telemetry.detach(handler_id)
|
||||
end
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue