Graham McIntire
74f62834e3
Replaces per-QSO-endpoint asos_batch granularity with per-station-day granularity. Each contact in the same UTC day near the same station now produces identical unique-args and collapses to ONE Oban job, so the backlog shrinks as cross-contact duplicates dedup. Each fetch also covers a full 24h window (vs the previous 4h) so one IEM request returns ~24 hourly observations instead of ~1-2. The effective bytes-per-request is 10x higher, amortizing the IemRateLimiter gap + 429 retry tail across far more data. Changes: - Weather.station_day_covered?/2 + station_day_pairs_covered/1 — cheap UTC-day coverage checks for the enqueuer + worker skip paths. - WeatherFetchWorker gains an "asos_day" clause that fetches the full UTC day for one station and upserts. Skips if the day is already covered in the DB. - ContactWeatherEnqueueWorker.build_asos_jobs/3 now emits one asos_day job per (station_id, UTC date) pair. ±2h windows crossing midnight enumerate both dates. - Microwaveprop.Weather.RebatchAsos.to_day_jobs/1 migrates any already-queued asos/asos_batch jobs into the new shape. Idempotent; supports dry_run. - asos_batch worker clause retained for jobs already in flight at deploy time. 2835 tests + credo green. |
||
|---|---|---|
| .. | ||
| fixtures | ||
| microwaveprop | ||
| microwaveprop_web | ||
| mix/tasks | ||
| support | ||
| test_helper.exs | ||