Graham McIntire
c073c6a95a
IEM's ASOS CSV endpoint accepts multiple station= params in a single request. Prior code enqueued one WeatherFetchWorker job per nearby station per QSO endpoint — N jobs × 1 station each — which paid the IemRateLimiter's 1500ms gap and the IEM 429-retry tail N times per contact. Changes: - IemClient.asos_url/3 accepts a list of station codes. - IemClient.fetch_asos_batch/3 fetches N stations in one call and returns rows grouped by station_code (with absent codes filled as empty lists so callers can stub them). - parse_asos_csv/1 now exposes the station_code column it was previously discarding. - WeatherFetchWorker gains an "asos_batch" fetch_type clause that unpacks rows per (station_id, station_code), upserting or stubbing each. The single-station "asos" clause stays for already-queued retryable jobs. - ContactWeatherEnqueueWorker.build_asos_jobs/3 now emits one batch job per (lat, lon, 4h window) covering every uncovered nearby station (sorted for deterministic unique-args). Expected effect on backfill: ~10-20x fewer IEM requests per contact enrichment cycle, matching drop in 429 retry traffic. |
||
|---|---|---|
| .. | ||
| microwaveprop | ||
| microwaveprop_web | ||
| mix/tasks | ||
| microwaveprop.ex | ||
| microwaveprop_web.ex | ||