Graham McIntire
23e4e3fef2
Two independent wins the latest prod telemetry pointed at: 1. Iowa Environmental Mesonet rate limiter. IEM throttles per source IP across all in-flight requests, so dropping the `:weather` Oban queue to 1/pod wasn't enough — workers were still issuing back-to- back requests inside each job and drawing a steady stream of HTTP 429s (1,396 retryable on the queue). A GenServer-based token bucket serialises acquire() with a 700ms min gap per pod; three pods give ~4 req/sec cluster-wide, well under the observed 429 threshold. Wrapped around every IemClient fetch. 2. Parallel HRRR surface + pressure grid fetch. The two products live in separate wrfsfcf / wrfprsf GRIB files, so their fetch → wgrib2 → decode pipelines are independent; running them sequentially was adding ~30s per forecast hour for no reason. Task.async the pair, merge at the end. Halves per-fh wall time and should unblock some of the missing-hour cases we saw on the 14:00Z chain. |
||
|---|---|---|
| .. | ||
| fixtures | ||
| microwaveprop | ||
| microwaveprop_web | ||
| mix/tasks | ||
| support | ||
| test_helper.exs | ||