prop/lib/microwaveprop/workers
Graham McIntire 7b78a2574c
fix(workers): 3 bug/perf fixes from codebase review
1. GridCache: auto-release fill lock when the claimer process crashes.
   claim_fill/1 + release_fill/1 go through the GenServer so the
   server can Process.monitor the caller and clean up the ETS entry
   on :DOWN. Clear/0 now resets both the data table and the lock
   table. Fixes a latent bug where a crashed fill leaked the lock
   indefinitely, preventing every subsequent /weather mount for that
   valid_time from claiming and leaving cache cold.

2. RadarFrameWorker: distinguish permanent vs transient fetch errors.
   404 from the IEM n0q archive is permanent (file will never exist)
   and marks contacts :unavailable as before. Any other error shape
   (5xx, timeout, transport failure) now returns {:error, reason}
   so Oban retries — previously those also pinned contacts at
   :unavailable after a transient outage.

3. AdminTaskWorker.native_derive: replace per-row Repo.update_all
   (N round-trips + N fsyncs) with one UPDATE ... FROM unnest(...)
   per 2000-row batch. For the 10k-profile budget this is one
   network round trip per chunk instead of 10k, and one fsync per
   chunk instead of 10k. Restructured the clause to separate
   derivation (pure) from persistence (I/O).

All three changes are test-covered (grid_cache_test auto-release
test, radar_frame_worker_test 5xx + transport tests, existing
admin_task_worker_test native_derive coverage exercises the new
bulk path). Also drops the scorer_diff no-op test that was
verifying the clause removed in 61da51c.
2026-04-21 09:53:13 -05:00
..
admin_task_worker.ex fix(workers): 3 bug/perf fixes from codebase review 2026-04-21 09:53:13 -05:00
backfill_enqueue_worker.ex refactor(contacts): rename propagation_mechanism_status -> mechanism_status 2026-04-21 09:28:37 -05:00
canadian_sounding_fetch_worker.ex Ingest Canadian radiosondes via UWYO + plans for RDPS/HRDPS 2026-04-13 09:14:34 -05:00
common_volume_radar_worker.ex feat(telemetry): wide instrumentation + bump hrrr to 2 per pod 2026-04-18 16:33:34 -05:00
contact_import_worker.ex feat(import): async CSV import with progress-tracking schema 2026-04-17 09:31:22 -05:00
contact_position_backfill_worker.ex test: override stale distance_km when filling a missing pos 2026-04-16 14:09:45 -05:00
contact_weather_enqueue_worker.ex refactor(contacts): rename propagation_mechanism_status -> mechanism_status 2026-04-21 09:28:37 -05:00
gefs_fetch_worker.ex feat(gefs): score extended-horizon grid and seed cron 2026-04-18 14:41:41 -05:00
hrrr_native_grid_worker.ex refactor: normalize pos1/pos2 JSONB key to 'lon' everywhere 2026-04-17 09:10:32 -05:00
iemre_fetch_worker.ex Fix credo warnings: struct specs, length/1, and test patterns 2026-04-12 10:26:53 -05:00
ionosphere_fetch_worker.ex Ionosphere: GIRO ionosonde ingestion (foF2/foEs/hmF2/MUFD) 2026-04-15 14:37:43 -05:00
mechanism_classify_worker.ex refactor(contacts): rename propagation_mechanism_status -> mechanism_status 2026-04-21 09:28:37 -05:00
narr_fetch_worker.ex Rename ERA5 → NARR across the codebase 2026-04-16 09:22:23 -05:00
nexrad_worker.ex refactor: normalize pos1/pos2 JSONB key to 'lon' everywhere 2026-04-17 09:10:32 -05:00
propagation_grid_worker.ex feat(propagation): Phase 3 Stream A cutover — Rust owns f00..f18 2026-04-19 18:13:41 -05:00
propagation_prune_worker.ex fix(propagation): hourly grid chain preempts MRMS/prune backlog 2026-04-19 08:42:22 -05:00
radar_frame_worker.ex fix(workers): 3 bug/perf fixes from codebase review 2026-04-21 09:53:13 -05:00
rtma_fetch_worker.ex Add ERA5 reanalysis and RTMA data sources 2026-04-07 12:04:16 -05:00
solar_index_worker.ex Database performance fixes and async backfill enqueue 2026-04-04 19:19:18 -05:00
space_weather_fetch_worker.ex SpaceWeather: SWPC JSON ingestion (Kp, F10.7, GOES X-ray) 2026-04-15 14:55:24 -05:00
terrain_profile_worker.ex feat(telemetry): broaden Instrument span coverage 2026-04-18 17:25:33 -05:00
weather_fetch_worker.ex feat(contacts): widen sounding search, backfill on demand 2026-04-18 16:04:50 -05:00