Graham McIntire
020f05f8eb
Stage 3 (Elixir foundation) of the HRDPS plan. Lands the scaffolding that lets the Rust prop-grid-rs HRDPS branch be wired up cleanly when it ships, without further DB or Elixir-side changes. What ships: - `grid_tasks.source` column (default 'hrrr', backfill is a no-op). Replaces the (run_time, forecast_hour, kind) unique index with a 4-column key (..., source) so HRRR and HRDPS analysis/forecast rows for the same cycle coexist. - `Grid.hrdps_only_points/0` — Canadian cells inside HRDPS bbox (49-60°N, -141 to -52°W) but outside HRRR's CONUS bbox. Disjoint from `conus_points/0` by construction so the two grids never double-write the same (lat, lon). - `Grid.point_source/1` — `:hrrr | :hrdps | :neither` classification for a (lat, lon) pair. Routes per-QSO enrichment to the right NWP source. - `GridTaskEnqueuer.seed_with_analysis/2` and `seed/2` accept a `:source` option; defaults to "hrrr" so existing callers are unchanged. - `HrdpsGridWorker` cycle seeder mirroring PropagationGridWorker's shape but for HRDPS's 4×/day cadence. `:hrdps` queue added to base Oban config. What's deferred: - The Rust `prop-grid-rs` worker doesn't yet branch on `task.source`. Until that ships, `HrdpsGridWorker` stays out of runtime.exs cron — it would just produce rows that the Rust worker mishandles as HRRR. Module doc explicitly notes the dormant state. - Map overlay extension and FreshnessMonitor HRDPS staleness deferred to follow-up sessions once real data flows. Activation path documented in the updated plan file. Coverage cap recorded as Stage 8 decision: 60°N for v1 (SRTM stops there). Arctic CDEM deferred to a follow-up plan.
90 lines
2.9 KiB
Elixir
90 lines
2.9 KiB
Elixir
defmodule Microwaveprop.Workers.HrdpsGridWorker do
|
||
@moduledoc """
|
||
Cycle seed worker for the HRDPS Canadian propagation chain. Mirrors
|
||
`Microwaveprop.Workers.PropagationGridWorker` but seeds `grid_tasks`
|
||
rows with `source: "hrdps"` so the Rust worker routes them through
|
||
the HRDPS fetch path.
|
||
|
||
## Cadence
|
||
|
||
HRDPS runs 4×/day at 00/06/12/18Z and publishes f000 ~3-4h after each
|
||
cycle. This worker is intended to fire at HH+5 of each cycle hour
|
||
(05:35Z, 11:35Z, 17:35Z, 23:35Z) — see plan stage 6 for the cron entry.
|
||
|
||
## Activation
|
||
|
||
**Not yet wired into runtime.exs cron.** The Rust prop-grid-rs worker
|
||
needs an HRDPS branch (plan stage 4) before it can drain
|
||
`source: "hrdps"` rows correctly. Until then this module is dormant
|
||
scaffolding — building blocks ready for the cron wire-up once the
|
||
Rust counterpart ships.
|
||
"""
|
||
|
||
use Oban.Worker,
|
||
queue: :hrdps,
|
||
priority: 0,
|
||
max_attempts: 5,
|
||
# Deduplicate seed jobs in a 6-hour window matching the cycle cadence.
|
||
unique: [period: 6 * 3600, states: [:available, :scheduled, :executing, :retryable]]
|
||
|
||
alias Microwaveprop.Instrument
|
||
alias Microwaveprop.Propagation.GridTaskEnqueuer
|
||
alias Microwaveprop.Weather.HrdpsClient
|
||
|
||
require Logger
|
||
|
||
@impl Oban.Worker
|
||
def perform(%Oban.Job{args: args}) when args == %{} do
|
||
Instrument.span([:hrdps, :grid_worker, :perform], %{}, fn ->
|
||
seed_chain()
|
||
end)
|
||
end
|
||
|
||
defp seed_chain do
|
||
# HRDPS shares grid_tasks with HRRR; the same reclaim sweep covers
|
||
# both because reclaim_stale_running/0 doesn't filter on source.
|
||
_ = GridTaskEnqueuer.reclaim_stale_running()
|
||
|
||
run_time = pick_run_time(DateTime.utc_now())
|
||
|
||
Logger.info("HrdpsGrid: seeding chain run_time=#{run_time} source=hrdps")
|
||
|
||
case GridTaskEnqueuer.seed_with_analysis(run_time, source: "hrdps") do
|
||
{:ok, count} ->
|
||
:telemetry.execute(
|
||
[:microwaveprop, :hrdps, :grid_worker, :seeded],
|
||
%{queued_tasks: count},
|
||
%{run_time: run_time}
|
||
)
|
||
|
||
:ok
|
||
|
||
{:error, reason} ->
|
||
:telemetry.execute(
|
||
[:microwaveprop, :hrdps, :grid_worker, :exception],
|
||
%{},
|
||
%{run_time: run_time, reason: inspect(reason)}
|
||
)
|
||
|
||
{:error, reason}
|
||
end
|
||
end
|
||
|
||
@doc """
|
||
Pick the freshest published HRDPS cycle. Probes `now-4h` (just past
|
||
typical publish latency) first; falls back to `now-10h` (one cycle
|
||
earlier) if the fresher cycle isn't yet on the datamart.
|
||
|
||
Public so tests can exercise selection logic without going through Oban.
|
||
"""
|
||
@spec pick_run_time(DateTime.t()) :: DateTime.t()
|
||
def pick_run_time(%DateTime{} = now) do
|
||
fresh = now |> DateTime.add(-4, :hour) |> HrdpsClient.nearest_hrdps_cycle() |> DateTime.truncate(:second)
|
||
|
||
if HrdpsClient.cycle_available?(fresh) do
|
||
fresh
|
||
else
|
||
now |> DateTime.add(-10, :hour) |> HrdpsClient.nearest_hrdps_cycle() |> DateTime.truncate(:second)
|
||
end
|
||
end
|
||
end
|