prop/lib/microwaveprop/workers
Graham McIntire 5cfb9e6c8e
fix(propagation): chain survives permanent step failures
Telemetry showed ~66 PropagationGridWorker exceptions per 6h with
55 ArgumentErrors and 11 TimeoutErrors, producing ~13 discarded
chain steps. Each discard broke the chain: subsequent forecast
hours were never enqueued, leaving the score store with huge gaps
(e.g. at 14:11 UTC the earliest available forecast was 18:00,
because f00-f05 all failed somewhere upstream and nothing ran
after them).

Three changes:

1. PropagationGridWorker: on the final attempt, still enqueue
   fh+1 even when this step failed. Oban discards the current
   job normally — but the rest of the chain keeps running, so
   one bad hour doesn't take out the remaining 12-18. The
   rescue is factored into a tested public helper.

2. HrrrClient.parse_idx: skip malformed idx lines instead of
   raising. NOAA S3 occasionally serves an HTML error page as
   the idx body, and the old strict String.to_integer path
   raised ArgumentError on the first non-numeric line and took
   down the chain step. This is the root cause of the 55
   ArgumentErrors.

3. JS renderTimeline: when no forecast hour is at-or-before
   wall-clock (all times are future — the gap scenario the
   fixes above are designed to prevent), stop labeling the
   earliest future slot "Now". Lets the user see honest
   "+Nh" offsets instead of a lie on the pill.
2026-04-19 09:26:20 -05:00
..
admin_task_worker.ex fix(admin_task_worker): unknown feature now returns error instead of falling through 2026-04-16 14:58:55 -05:00
backfill_enqueue_worker.ex feat(propagation): per-contact mechanism classification 2026-04-18 10:42:08 -05:00
canadian_sounding_fetch_worker.ex Ingest Canadian radiosondes via UWYO + plans for RDPS/HRDPS 2026-04-13 09:14:34 -05:00
common_volume_radar_worker.ex feat(telemetry): wide instrumentation + bump hrrr to 2 per pod 2026-04-18 16:33:34 -05:00
contact_import_worker.ex feat(import): async CSV import with progress-tracking schema 2026-04-17 09:31:22 -05:00
contact_position_backfill_worker.ex test: override stale distance_km when filling a missing pos 2026-04-16 14:09:45 -05:00
contact_weather_enqueue_worker.ex feat(contacts): widen sounding search, backfill on demand 2026-04-18 16:04:50 -05:00
gefs_fetch_worker.ex feat(gefs): score extended-horizon grid and seed cron 2026-04-18 14:41:41 -05:00
hrrr_fetch_worker.ex fix(workers): make weather/hrrr job uniqueness horizon indefinite 2026-04-17 10:15:30 -05:00
hrrr_native_grid_worker.ex refactor: normalize pos1/pos2 JSONB key to 'lon' everywhere 2026-04-17 09:10:32 -05:00
iemre_fetch_worker.ex Fix credo warnings: struct specs, length/1, and test patterns 2026-04-12 10:26:53 -05:00
ionosphere_fetch_worker.ex Ionosphere: GIRO ionosonde ingestion (foF2/foEs/hmF2/MUFD) 2026-04-15 14:37:43 -05:00
mechanism_classify_worker.ex feat(telemetry): broaden Instrument span coverage 2026-04-18 17:25:33 -05:00
mrms_fetch_worker.ex fix(propagation): hourly grid chain preempts MRMS/prune backlog 2026-04-19 08:42:22 -05:00
narr_fetch_worker.ex Rename ERA5 → NARR across the codebase 2026-04-16 09:22:23 -05:00
nexrad_worker.ex refactor: normalize pos1/pos2 JSONB key to 'lon' everywhere 2026-04-17 09:10:32 -05:00
propagation_grid_worker.ex fix(propagation): chain survives permanent step failures 2026-04-19 09:26:20 -05:00
propagation_prune_worker.ex fix(propagation): hourly grid chain preempts MRMS/prune backlog 2026-04-19 08:42:22 -05:00
rtma_fetch_worker.ex Add ERA5 reanalysis and RTMA data sources 2026-04-07 12:04:16 -05:00
solar_index_worker.ex Database performance fixes and async backfill enqueue 2026-04-04 19:19:18 -05:00
space_weather_fetch_worker.ex SpaceWeather: SWPC JSON ingestion (Kp, F10.7, GOES X-ray) 2026-04-15 14:55:24 -05:00
terrain_profile_worker.ex feat(telemetry): broaden Instrument span coverage 2026-04-18 17:25:33 -05:00
weather_fetch_worker.ex feat(contacts): widen sounding search, backfill on demand 2026-04-18 16:04:50 -05:00