Mix tasks that call app.start were also booting Oban's cron scheduler,
causing PropagationGridWorker and other cron jobs to fire during
backfills. Add Oban.pause_all_queues(Oban) immediately after app.start
in every mix task that only needs Repo access.
Per-point ERA5 fetches were tragically slow because every point-hour
triggered its own asynchronous CDS job (submit → poll → assemble →
download). For backfill this meant thousands of independent jobs
queued against Copernicus. The new path groups requests by calendar
month and a 2° × 2° lat/lon tile so one CDS cycle populates ~60k
profiles at once, and Oban uniqueness on (year, month, tile_lat,
tile_lon) collapses every duplicate enqueue.
- Era5BatchClient builds the monthly CDS requests, extracts every
(lat, lon, hour) from the GRIB2 blob with wgrib2, derives
refractivity params, and bulk-inserts in 2k-row chunks with
on_conflict: :nothing. fetch_month_into_db/1 short-circuits when
the month-tile already has any cached profile.
- Era5MonthBatchWorker runs the batch on the :era5 queue with a
generous backoff (10m → 1d) and the uniqueness key above.
- Era5FetchWorker is now a thin router: cache hit → :ok, cache miss
→ enqueue the month-batch for the point's tile-month and return.
No more per-point CDS calls.
- Wgrib2 grows extract_grid_messages/3 which preserves per-message
datetimes by parsing `d=YYYYMMDDHH[MMSS]` from the inventory, so a
single GRIB2 file carrying a whole month decodes correctly.
- The era5_backfill mix task enqueues month-tile batches directly.
The Era5FetchWorker now declares an Oban unique constraint on
{lat, lon, valid_time} for any job in available/scheduled/executing/
retryable, so two backfill runs targeting the same contact grid point
can no longer spawn parallel CDS requests for the same hour.
Because Oban OSS insert_all doesn't honor unique, ERA5 jobs are now
routed through Oban.insert/1 from ContactWeatherEnqueueWorker and the
era5_backfill mix task. Other worker types still use insert_all.
Enqueues ERA5 fetch jobs for contacts where HRRR is unavailable.
Deduplicates by rounded grid point and hour, skips existing profiles.
Usage: mix era5_backfill [limit]
