Graham McIntire
fac604b289
Era5SubmitWorker was counting era5_cds_jobs rows 1:1 against the CDS
150-job ceiling, but each row holds TWO CDS job IDs (single-level +
pressure-level). Prod hit 74 rows = 148 in-flight jobs and got stuck
in a reject→resubmit spiral. The cap guard now multiplies row count
by 2 to match what CDS sees.
Era5PollWorker @stuck_after_seconds goes from 4h to 18h. CDS
single-level routinely sits at 'accepted' for 12+h under load while
pressure finishes in ~90 min; the 4h threshold was firing on normal
slow runs and feeding the same spiral.
PipelineStatus now returns `label` + `detail` separately so the map
chip can render "Updating propagation" on one line and the
sub-detail ("HRRR run" / "ASOS nudge" / "+3h" / "now") on a second
line below it. running_label/1 is renamed to running_detail/1 and
returns just the sub-part.
|
||
|---|---|---|
| .. | ||
| asos_nudge.ex | ||
| band_config.ex | ||
| duct.ex | ||
| freshness_monitor.ex | ||
| grid.ex | ||
| grid_score.ex | ||
| inversion.ex | ||
| pipeline_status.ex | ||
| rain_scatter.ex | ||
| recalibrator.ex | ||
| region.ex | ||
| score_cache.ex | ||
| scorer.ex | ||
| scorer_diff.ex | ||