fix(k8s): tier label isolates hot pods from backfill endpoints
Both the hot (`prop`) Deployment and the `prop-backfill` Deployment
labelled their pods with `app: prop` so both joined the same libcluster
Erlang cluster. The `prop` Service selector was `app: prop` too — which
meant the backfill pod (PHX_SERVER=false, no listener on :5000) was in
the Service endpoint pool.
With sessionAffinity: ClientIP, Cloudflared's source IP got hashed to
the backfill endpoint after a backfill pod roll and every request
from that client returned 502 (connection refused on :5000).
Fix: add `tier: hot` to hot-pod template labels; narrow the Service
selector to `{app: prop, tier: hot}`. Backfill keeps `tier: backfill`
and is cleanly excluded. libcluster selector (still `app: prop`) is
unchanged so backfill stays in the Erlang cluster for Oban queue
leader election and Oban.Pro's Smart engine.
This commit is contained in:
parent
8ab2d836fb
commit
6c73012d09
2 changed files with 10 additions and 0 deletions
|
|
@ -17,7 +17,12 @@ spec:
|
||||||
template:
|
template:
|
||||||
metadata:
|
metadata:
|
||||||
labels:
|
labels:
|
||||||
|
# `tier: hot` distinguishes these pods from prop-backfill (same
|
||||||
|
# app: prop label) so the Service selector can exclude backfill
|
||||||
|
# from its endpoint pool — backfill runs PHX_SERVER=false and
|
||||||
|
# doesn't listen on :5000.
|
||||||
app: prop
|
app: prop
|
||||||
|
tier: hot
|
||||||
spec:
|
spec:
|
||||||
tolerations:
|
tolerations:
|
||||||
- key: node-role.kubernetes.io/control-plane
|
- key: node-role.kubernetes.io/control-plane
|
||||||
|
|
|
||||||
|
|
@ -11,6 +11,11 @@ spec:
|
||||||
timeoutSeconds: 3600
|
timeoutSeconds: 3600
|
||||||
selector:
|
selector:
|
||||||
app: prop
|
app: prop
|
||||||
|
# Exclude prop-backfill (tier: backfill) — it runs PHX_SERVER=false
|
||||||
|
# and doesn't listen on :5000. A pre-tier-label deploy caused a
|
||||||
|
# Cloudflare 502 when ClientIP affinity pinned a source to the
|
||||||
|
# backfill endpoint.
|
||||||
|
tier: hot
|
||||||
ports:
|
ports:
|
||||||
- name: http
|
- name: http
|
||||||
port: 5000
|
port: 5000
|
||||||
|
|
|
||||||
Loading…
Add table
Reference in a new issue