prop/lib/microwaveprop/workers
Graham McIntire c073c6a95a
perf(weather): batch ASOS fetches into one multi-station request
IEM's ASOS CSV endpoint accepts multiple station= params in a single
request. Prior code enqueued one WeatherFetchWorker job per nearby
station per QSO endpoint — N jobs × 1 station each — which paid the
IemRateLimiter's 1500ms gap and the IEM 429-retry tail N times per
contact.

Changes:
- IemClient.asos_url/3 accepts a list of station codes.
- IemClient.fetch_asos_batch/3 fetches N stations in one call and
  returns rows grouped by station_code (with absent codes filled as
  empty lists so callers can stub them).
- parse_asos_csv/1 now exposes the station_code column it was
  previously discarding.
- WeatherFetchWorker gains an "asos_batch" fetch_type clause that
  unpacks rows per (station_id, station_code), upserting or stubbing
  each. The single-station "asos" clause stays for already-queued
  retryable jobs.
- ContactWeatherEnqueueWorker.build_asos_jobs/3 now emits one batch
  job per (lat, lon, 4h window) covering every uncovered nearby
  station (sorted for deterministic unique-args).

Expected effect on backfill: ~10-20x fewer IEM requests per contact
enrichment cycle, matching drop in 429 retry traffic.
2026-04-24 12:44:22 -05:00
..
admin_task_worker.ex fix(review): address code-reviewer findings 2026-04-21 10:09:46 -05:00
backfill_enqueue_worker.ex fix(hrrr): mark contacts :unavailable on failed fetch tasks 2026-04-22 17:46:51 -05:00
canadian_sounding_fetch_worker.ex fix(dialyzer): clear 125+ warnings under strict flags 2026-04-21 10:30:06 -05:00
common_volume_radar_worker.ex perf+hygiene: batch 1 of system-review fixes 2026-04-21 17:06:07 -05:00
contact_import_worker.ex perf+hygiene: batch 1 of system-review fixes 2026-04-21 17:06:07 -05:00
contact_position_backfill_worker.ex test: override stale distance_km when filling a missing pos 2026-04-16 14:09:45 -05:00
contact_weather_enqueue_worker.ex perf(weather): batch ASOS fetches into one multi-station request 2026-04-24 12:44:22 -05:00
gefs_fetch_worker.ex refactor(scores): rename file extension .ntms -> .prop with legacy reads 2026-04-21 15:54:12 -05:00
grid_cache_prune_worker.ex feat(workers): add GridCachePruneWorker 2026-04-21 13:19:52 -05:00
hrrr_native_grid_worker.ex refactor(db): use explicit on_conflict set clauses over :replace_all_except 2026-04-21 14:22:15 -05:00
iemre_fetch_worker.ex test: expand coverage and refactor to idiomatic style 2026-04-23 13:28:42 -05:00
ionosphere_fetch_worker.ex perf+hygiene: batch 1 of system-review fixes 2026-04-21 17:06:07 -05:00
mechanism_classify_worker.ex fix(workers): propagate transient errors so Oban retries + counters fire 2026-04-21 13:15:27 -05:00
narr_fetch_worker.ex fix(dialyzer): clear 125+ warnings under strict flags 2026-04-21 10:30:06 -05:00
nexrad_worker.ex refactor(db): use explicit on_conflict set clauses over :replace_all_except 2026-04-21 14:22:15 -05:00
propagation_grid_worker.ex fix(grid-tasks): reclaim orphan running rows on hourly seed 2026-04-23 12:57:39 -05:00
propagation_prune_worker.ex fix(propagation): hourly grid chain preempts MRMS/prune backlog 2026-04-19 08:42:22 -05:00
radar_frame_worker.ex refactor(db): use explicit on_conflict set clauses over :replace_all_except 2026-04-21 14:22:15 -05:00
rtma_fetch_worker.ex fix(workers): unique dedup on polling workers so queued dups don't stack 2026-04-21 14:12:43 -05:00
solar_index_worker.ex fix(dialyzer): clear 125+ warnings under strict flags 2026-04-21 10:30:06 -05:00
space_weather_fetch_worker.ex SpaceWeather: SWPC JSON ingestion (Kp, F10.7, GOES X-ray) 2026-04-15 14:55:24 -05:00
terrain_profile_worker.ex fix(dialyzer): clear 125+ warnings under strict flags 2026-04-21 10:30:06 -05:00
weather_fetch_worker.ex perf(weather): batch ASOS fetches into one multi-station request 2026-04-24 12:44:22 -05:00