CDS rejects `reanalysis-era5-pressure-levels` requests that include
dewpoint_temperature because it's only a single-level 2m variable; no
pressure-level dewpoint exists in ERA5. The failure mode is a 2–3 second
`{"status": "failed"}` response with no message field, which is why
prod was burning through Era5MonthBatchWorker attempts without anything
landing in era5_profiles.
Switch the pressure-level moisture variable to specific_humidity (SPFH),
update the wgrib2 extraction regex, and convert SPFH → dewpoint °C via
Weather.ThetaE.dewpoint_from_spfh/2 in the profile builder. Surface
dewpoint keeps using the single-level 2m dewpoint_temperature, which is
valid and already working.
Per-point ERA5 fetches were tragically slow because every point-hour
triggered its own asynchronous CDS job (submit → poll → assemble →
download). For backfill this meant thousands of independent jobs
queued against Copernicus. The new path groups requests by calendar
month and a 2° × 2° lat/lon tile so one CDS cycle populates ~60k
profiles at once, and Oban uniqueness on (year, month, tile_lat,
tile_lon) collapses every duplicate enqueue.
- Era5BatchClient builds the monthly CDS requests, extracts every
(lat, lon, hour) from the GRIB2 blob with wgrib2, derives
refractivity params, and bulk-inserts in 2k-row chunks with
on_conflict: :nothing. fetch_month_into_db/1 short-circuits when
the month-tile already has any cached profile.
- Era5MonthBatchWorker runs the batch on the :era5 queue with a
generous backoff (10m → 1d) and the uniqueness key above.
- Era5FetchWorker is now a thin router: cache hit → :ok, cache miss
→ enqueue the month-batch for the point's tile-month and return.
No more per-point CDS calls.
- Wgrib2 grows extract_grid_messages/3 which preserves per-message
datetimes by parsing `d=YYYYMMDDHH[MMSS]` from the inventory, so a
single GRIB2 file carrying a whole month decodes correctly.
- The era5_backfill mix task enqueues month-tile batches directly.
