Graham McIntire
504f5147f6
The :propagation queue (2 slots) is shared by PropagationGridWorker, MrmsFetchWorker (every 2 min), and PropagationPruneWorker (every 15 min). Oban dispatches priority-then-FIFO, so a post-deploy MRMS/prune backlog could delay the hourly chain step until the siblings drained. Set PropagationGridWorker priority: 0 explicitly and drop the two siblings to priority: 5 so new chain steps always jump ahead. Backfill workers live on separate queues (:hrrr, :weather, :terrain, :iemre, :narr, :radar, :mechanism) and already don't interact with the predictor.
46 lines
1.5 KiB
Elixir
46 lines
1.5 KiB
Elixir
defmodule Microwaveprop.Workers.MrmsFetchWorker do
|
|
@moduledoc """
|
|
Every ~2 minutes, pull the newest NOAA MRMS PrecipRate grid and cache
|
|
the regridded version (0.125° CONUS) in `Microwaveprop.Weather.MrmsCache`.
|
|
`Microwaveprop.Workers.AsosAdjustmentWorker` reads that cache on its
|
|
own tick to overlay real radar rain rates onto the score grid.
|
|
|
|
Runs on the `propagation` queue so it shares the scoring node's crontab
|
|
and isn't competing with the heavier HRRR pipeline for slots.
|
|
"""
|
|
# Lower priority than PropagationGridWorker so an MRMS backlog
|
|
# (cron fires every 2 minutes) doesn't starve the hourly chain
|
|
# on the shared :propagation queue.
|
|
use Oban.Worker, queue: :propagation, priority: 5, max_attempts: 2
|
|
|
|
alias Microwaveprop.Weather.MrmsCache
|
|
alias Microwaveprop.Weather.MrmsClient
|
|
|
|
require Logger
|
|
|
|
@impl Oban.Worker
|
|
def perform(%Oban.Job{}) do
|
|
Microwaveprop.Instrument.span([:worker, :mrms_fetch], %{}, fn ->
|
|
case MrmsClient.fetch_latest(MrmsCache.valid_time()) do
|
|
{:ok, valid_time, grid} ->
|
|
MrmsCache.broadcast_put(valid_time, grid)
|
|
|
|
Logger.info("MrmsFetch: cached #{map_size(grid)} cells at #{valid_time} (#{count_rainy(grid)} with rain)")
|
|
|
|
:ok
|
|
|
|
{:up_to_date, valid_time} ->
|
|
Logger.debug("MrmsFetch: cache already has #{valid_time}")
|
|
:ok
|
|
|
|
{:error, reason} ->
|
|
Logger.warning("MrmsFetch: skipped — #{inspect(reason)}")
|
|
:ok
|
|
end
|
|
end)
|
|
end
|
|
|
|
defp count_rainy(grid) do
|
|
Enum.count(grid, fn {_, v} -> v > 0 end)
|
|
end
|
|
end
|