prop/lib/microwaveprop/workers
Graham McIntire 74f62834e3
perf(weather): asos_day — one job per unique (station, UTC date)
Replaces per-QSO-endpoint asos_batch granularity with per-station-day
granularity. Each contact in the same UTC day near the same station
now produces identical unique-args and collapses to ONE Oban job, so
the backlog shrinks as cross-contact duplicates dedup.

Each fetch also covers a full 24h window (vs the previous 4h) so one
IEM request returns ~24 hourly observations instead of ~1-2. The
effective bytes-per-request is 10x higher, amortizing the
IemRateLimiter gap + 429 retry tail across far more data.

Changes:
- Weather.station_day_covered?/2 + station_day_pairs_covered/1 — cheap
  UTC-day coverage checks for the enqueuer + worker skip paths.
- WeatherFetchWorker gains an "asos_day" clause that fetches the full
  UTC day for one station and upserts. Skips if the day is already
  covered in the DB.
- ContactWeatherEnqueueWorker.build_asos_jobs/3 now emits one asos_day
  job per (station_id, UTC date) pair. ±2h windows crossing midnight
  enumerate both dates.
- Microwaveprop.Weather.RebatchAsos.to_day_jobs/1 migrates any
  already-queued asos/asos_batch jobs into the new shape. Idempotent;
  supports dry_run.
- asos_batch worker clause retained for jobs already in flight at
  deploy time.

2835 tests + credo green.
2026-04-24 13:19:20 -05:00
..
admin_task_worker.ex fix(review): address code-reviewer findings 2026-04-21 10:09:46 -05:00
backfill_enqueue_worker.ex fix(hrrr): mark contacts :unavailable on failed fetch tasks 2026-04-22 17:46:51 -05:00
canadian_sounding_fetch_worker.ex fix(dialyzer): clear 125+ warnings under strict flags 2026-04-21 10:30:06 -05:00
common_volume_radar_worker.ex perf+hygiene: batch 1 of system-review fixes 2026-04-21 17:06:07 -05:00
contact_import_worker.ex perf+hygiene: batch 1 of system-review fixes 2026-04-21 17:06:07 -05:00
contact_position_backfill_worker.ex test: override stale distance_km when filling a missing pos 2026-04-16 14:09:45 -05:00
contact_weather_enqueue_worker.ex perf(weather): asos_day — one job per unique (station, UTC date) 2026-04-24 13:19:20 -05:00
gefs_fetch_worker.ex refactor(scores): rename file extension .ntms -> .prop with legacy reads 2026-04-21 15:54:12 -05:00
grid_cache_prune_worker.ex feat(workers): add GridCachePruneWorker 2026-04-21 13:19:52 -05:00
hrrr_native_grid_worker.ex refactor(db): use explicit on_conflict set clauses over :replace_all_except 2026-04-21 14:22:15 -05:00
iemre_fetch_worker.ex test: expand coverage and refactor to idiomatic style 2026-04-23 13:28:42 -05:00
ionosphere_fetch_worker.ex perf+hygiene: batch 1 of system-review fixes 2026-04-21 17:06:07 -05:00
mechanism_classify_worker.ex fix(workers): propagate transient errors so Oban retries + counters fire 2026-04-21 13:15:27 -05:00
narr_fetch_worker.ex fix(dialyzer): clear 125+ warnings under strict flags 2026-04-21 10:30:06 -05:00
nexrad_worker.ex refactor(db): use explicit on_conflict set clauses over :replace_all_except 2026-04-21 14:22:15 -05:00
propagation_grid_worker.ex fix(grid-tasks): reclaim orphan running rows on hourly seed 2026-04-23 12:57:39 -05:00
propagation_prune_worker.ex fix(propagation): hourly grid chain preempts MRMS/prune backlog 2026-04-19 08:42:22 -05:00
radar_frame_worker.ex refactor(db): use explicit on_conflict set clauses over :replace_all_except 2026-04-21 14:22:15 -05:00
rtma_fetch_worker.ex fix(workers): unique dedup on polling workers so queued dups don't stack 2026-04-21 14:12:43 -05:00
solar_index_worker.ex fix(dialyzer): clear 125+ warnings under strict flags 2026-04-21 10:30:06 -05:00
space_weather_fetch_worker.ex SpaceWeather: SWPC JSON ingestion (Kp, F10.7, GOES X-ray) 2026-04-15 14:55:24 -05:00
terrain_profile_worker.ex fix(dialyzer): clear 125+ warnings under strict flags 2026-04-21 10:30:06 -05:00
weather_fetch_worker.ex perf(weather): asos_day — one job per unique (station, UTC date) 2026-04-24 13:19:20 -05:00