Era5SubmitWorker was counting era5_cds_jobs rows 1:1 against the CDS
150-job ceiling, but each row holds TWO CDS job IDs (single-level +
pressure-level). Prod hit 74 rows = 148 in-flight jobs and got stuck
in a reject→resubmit spiral. The cap guard now multiplies row count
by 2 to match what CDS sees.
Era5PollWorker @stuck_after_seconds goes from 4h to 18h. CDS
single-level routinely sits at 'accepted' for 12+h under load while
pressure finishes in ~90 min; the 4h threshold was firing on normal
slow runs and feeding the same spiral.
PipelineStatus now returns `label` + `detail` separately so the map
chip can render "Updating propagation" on one line and the
sub-detail ("HRRR run" / "ASOS nudge" / "+3h" / "now") on a second
line below it. running_label/1 is renamed to running_detail/1 and
returns just the sub-part.
|
||
|---|---|---|
| .. | ||
| admin | ||
| beacon_live | ||
| contact_live | ||
| user_management_live | ||
| about_live.ex | ||
| algo_live.ex | ||
| contact_map_live.ex | ||
| map_live.ex | ||
| path_live.ex | ||
| privacy_live.ex | ||
| rover_live.ex | ||
| status_live.ex | ||
| submit_live.ex | ||
| user_profile_live.ex | ||
| weather_map_live.ex | ||