Graham McIntire
fac604b289
Era5SubmitWorker was counting era5_cds_jobs rows 1:1 against the CDS
150-job ceiling, but each row holds TWO CDS job IDs (single-level +
pressure-level). Prod hit 74 rows = 148 in-flight jobs and got stuck
in a reject→resubmit spiral. The cap guard now multiplies row count
by 2 to match what CDS sees.
Era5PollWorker @stuck_after_seconds goes from 4h to 18h. CDS
single-level routinely sits at 'accepted' for 12+h under load while
pressure finishes in ~90 min; the 4h threshold was firing on normal
slow runs and feeding the same spiral.
PipelineStatus now returns `label` + `detail` separately so the map
chip can render "Updating propagation" on one line and the
sub-detail ("HRRR run" / "ASOS nudge" / "+3h" / "now") on a second
line below it. running_label/1 is renamed to running_detail/1 and
returns just the sub-part.
|
||
|---|---|---|
| .. | ||
| asos_nudge_test.exs | ||
| band_config_test.exs | ||
| duct_test.exs | ||
| grid_score_test.exs | ||
| grid_test.exs | ||
| inversion_test.exs | ||
| model_test.exs | ||
| pipeline_status_test.exs | ||
| recalibrator_test.exs | ||
| region_test.exs | ||
| score_cache_test.exs | ||
| scorer_diff_test.exs | ||
| scorer_test.exs | ||