Graham McIntire
022e6928bc
Replaces the serial claim→process loop with a JoinSet of N worker tasks, each running the same claim→fetch→decode→score→complete flow against grid_tasks. FOR UPDATE SKIP LOCKED already prevents contention between workers so no additional coordination is needed. talos5 is 4c/4t (i5-7260U); PROP_GRID_RS_PARALLELISM=4 turns the full f01..f18 chain from ~18 min serial into roughly ~5 min. Per-task peak memory after the streamed-bands refactor is ~250 Mi, so 4 concurrent is ~1 Gi — well under the 2 Gi container limit. Postgres pool sized to parallelism + 2 so NOTIFY and an opportunistic claim retry share the pool without blocking a worker transaction. |
||
|---|---|---|
| .. | ||
| src | ||
| tests | ||
| .gitignore | ||
| Cargo.lock | ||
| Cargo.toml | ||
| Dockerfile | ||