defmodule Microwaveprop.Weather.HrrrPointEnqueuer do @moduledoc """ Inserts rows into `hrrr_fetch_tasks` for the Rust hrrr-point-worker to drain. Called from `Microwaveprop.Workers.ContactWeatherEnqueueWorker` in place of the legacy `HrrrFetchWorker.new/1` fan-out. One row per `valid_time` with a JSONB array of `{lat, lon}` points that share that hour — additional points arriving from a later backfill tick union into the existing row rather than creating a second one, so the Rust worker fetches each GRIB2 once regardless of how many contacts feed it. `BackfillEnqueueWorker` re-discovers contacts missing HRRR enrichment every 30 minutes; each scan flows through `enqueue_for_contact` and lands here, which keeps the backfill pipeline intact after the cutover. """ import Ecto.Query alias Microwaveprop.Radio alias Microwaveprop.Repo alias Microwaveprop.Weather alias Microwaveprop.Weather.HrrrClient alias Microwaveprop.Weather.NarrClient require Logger @doc """ Enqueue `valid_time -> [{lat, lon}, ...]` groups into `hrrr_fetch_tasks`. On conflict the incoming points are array-unioned into the existing row and the status is reset to `queued` so a previously-done valid_time re-scheduled by backfill picks up the new points without duplicating a fetch. """ @spec enqueue(%{DateTime.t() => [{float(), float()}]}) :: {:ok, non_neg_integer()} def enqueue(groups) when is_map(groups) do now = DateTime.truncate(DateTime.utc_now(), :microsecond) count = Enum.reduce(groups, 0, fn {valid_time, points}, acc -> valid_time = DateTime.truncate(valid_time, :second) json_points = Enum.map(points, fn {lat, lon} -> %{"lat" => lat, "lon" => lon} end) # Union with the existing row's points (JSONB array) — use a # subquery with jsonb_array_elements to dedupe. Postgres # handles the set math so the Elixir side stays simple. # # Insert uses a wrapping schema query so Postgrex's jsonb # extension encodes the list directly; raw Repo.query! with # a binding would fall through to `text->jsonb` cast which # then stores the JSON as a jsonb *string* and breaks the # || union on round-trip. id = Ecto.UUID.bingenerate() _ = Repo.insert_all( "hrrr_fetch_tasks", [ %{ id: id, valid_time: valid_time, points: json_points, status: "queued", attempt: 0, inserted_at: now, updated_at: now } ], on_conflict: from(t in "hrrr_fetch_tasks", update: [ set: [ points: fragment( "(SELECT COALESCE(jsonb_agg(DISTINCT p), '[]'::jsonb) FROM jsonb_array_elements(? || EXCLUDED.points) AS p)", t.points ), status: fragment( "CASE WHEN ? IN ('done', 'failed') THEN 'queued' ELSE ? END", t.status, t.status ), attempt: fragment( "CASE WHEN ? IN ('done', 'failed') THEN 0 ELSE ? END", t.status, t.attempt ), updated_at: ^now ] ] ), conflict_target: [:valid_time] ) acc + 1 end) {:ok, count} rescue e -> Logger.error("HrrrPointEnqueuer: enqueue failed: #{inspect(e)}") {:ok, 0} end @doc """ Convenience: given a list of `Contact`s, extract the `(valid_time, [lat, lon])` groups and enqueue in one call. Mirrors the signature of the removed `ContactWeatherEnqueueWorker.build_hrrr_jobs/1` so the caller swap is a one-liner. """ @spec enqueue_for_contacts([map()]) :: {:ok, non_neg_integer()} def enqueue_for_contacts(contacts) do groups = contacts |> Enum.flat_map(&contact_points/1) |> Enum.group_by(fn {time, _pt} -> time end, fn {_time, pt} -> pt end) |> Map.new(fn {time, pts} -> {time, Enum.uniq(pts)} end) enqueue(groups) end # Per-contact expansion shared by `enqueue_for_contacts/1`. Returns a # flat list of `{valid_time, {lat, lon}}` tuples ready for grouping. # HRRR archive starts mid-2014; contacts older than that belong to # NARR, so they're dropped here. defp contact_points(contact) do cond do is_nil(contact.pos1) -> [] NarrClient.in_coverage?(contact.qso_timestamp) -> [] true -> rounded_time = HrrrClient.nearest_hrrr_hour(contact.qso_timestamp) contact |> Radio.contact_path_points() |> Enum.flat_map(&point_for_rounded_time(&1, rounded_time)) end end defp point_for_rounded_time({lat, lon}, rounded_time) do {rlat, rlon} = Weather.round_to_hrrr_grid(lat, lon) if Weather.has_hrrr_profile?(rlat, rlon, rounded_time) do [] else [{rounded_time, {rlat, rlon}}] end end end