Graham McIntire
022e6928bc
Replaces the serial claim→process loop with a JoinSet of N worker tasks, each running the same claim→fetch→decode→score→complete flow against grid_tasks. FOR UPDATE SKIP LOCKED already prevents contention between workers so no additional coordination is needed. talos5 is 4c/4t (i5-7260U); PROP_GRID_RS_PARALLELISM=4 turns the full f01..f18 chain from ~18 min serial into roughly ~5 min. Per-task peak memory after the streamed-bands refactor is ~250 Mi, so 4 concurrent is ~1 Gi — well under the 2 Gi container limit. Postgres pool sized to parallelism + 2 so NOTIFY and an opportunistic claim retry share the pool without blocking a worker transaction. |
||
|---|---|---|
| .. | ||
| deployment-backfill.yaml | ||
| deployment-grid-rs.yaml | ||
| deployment.yaml | ||
| flux.yaml | ||
| kustomization.yaml | ||
| metrics-service.yaml | ||
| namespace.yaml | ||
| rbac.yaml | ||
| service.yaml | ||