No description
Graham McIntire
14a2284321
CDS returns status="rejected" (distinct from "failed") when a submit
is rate-limited past the 150-job per-user cap. The poll worker's
interpret_status_body treated this as an unknown status → generic
retryable error → 10 retries → discarded. 102 jobs hit this path
overnight.
Fix:
* Era5Client.check_status/1 now returns {:rejected, reason} (and
treats "dismissed" the same way — that's the status a manually
deleted job transitions to).
* Era5PollWorker handles :rejected exactly like :not_found: drop the
DB row, clean up the sibling CDS job, re-enqueue the submit, and
discard the current Oban job so it doesn't burn attempts retrying
a terminal state.
* Widened Era5SubmitWorker cap headroom from 10 → 30 (effective
ceiling 120 not 140). The race between concurrent workers all
passing the count check simultaneously meant we were reaching 141
in-flight against the 150 hard cap; 30 slots of headroom tolerates
a ~15-worker race before clipping CDS's ceiling.
Refactored handle_row/1 into handle_row + handle_non_both_done with a
leg_outcome/1 tag helper so the cross-product of leg statuses collapses
to a single pair match (credo complexity limit).
|
||
|---|---|---|
| .forgejo/workflows | ||
| .github | ||
| assets | ||
| bin | ||
| config | ||
| docs | ||
| k8s | ||
| lib | ||
| lib_ml | ||
| notebooks | ||
| priv | ||
| rel | ||
| scripts | ||
| test | ||
| vendor | ||
| .credo.exs | ||
| .dockerignore | ||
| .formatter.exs | ||
| .gitignore | ||
| .tool-versions | ||
| AGENTS.md | ||
| algo.md | ||
| app.json | ||
| CLAUDE.md | ||
| Dockerfile | ||
| elevation.md | ||
| mix.exs | ||
| mix.lock | ||
| prediction.md | ||
| README.md | ||
| tail_logs.sh | ||
| updates.md | ||
Microwaveprop
To start your Phoenix server:
- Run
mix setupto install and setup dependencies - Start Phoenix endpoint with
mix phx.serveror inside IEx withiex -S mix phx.server
Now you can visit localhost:4000 from your browser.
Ready to run in production? Please check our deployment guides.
Learn more
- Official website: https://www.phoenixframework.org/
- Guides: https://hexdocs.pm/phoenix/overview.html
- Docs: https://hexdocs.pm/phoenix
- Forum: https://elixirforum.com/c/phoenix-forum
- Source: https://github.com/phoenixframework/phoenix