Telemetry showed ~66 PropagationGridWorker exceptions per 6h with 55 ArgumentErrors and 11 TimeoutErrors, producing ~13 discarded chain steps. Each discard broke the chain: subsequent forecast hours were never enqueued, leaving the score store with huge gaps (e.g. at 14:11 UTC the earliest available forecast was 18:00, because f00-f05 all failed somewhere upstream and nothing ran after them). Three changes: 1. PropagationGridWorker: on the final attempt, still enqueue fh+1 even when this step failed. Oban discards the current job normally — but the rest of the chain keeps running, so one bad hour doesn't take out the remaining 12-18. The rescue is factored into a tested public helper. 2. HrrrClient.parse_idx: skip malformed idx lines instead of raising. NOAA S3 occasionally serves an HTML error page as the idx body, and the old strict String.to_integer path raised ArgumentError on the first non-numeric line and took down the chain step. This is the root cause of the 55 ArgumentErrors. 3. JS renderTimeline: when no forecast hour is at-or-before wall-clock (all times are future — the gap scenario the fixes above are designed to prevent), stop labeling the earliest future slot "Now". Lets the user see honest "+Nh" offsets instead of a lie on the pill. |
||
|---|---|---|
| .. | ||
| microwaveprop | ||
| microwaveprop_web | ||
| mix/tasks | ||
| microwaveprop.ex | ||
| microwaveprop_web.ex | ||