Add zero-downtime deployment configuration

Prevent WebSocket disconnections during deployments by:

1. Rolling Update Strategy:
   - maxSurge: 1 (allow 1 extra pod during rollout)
   - maxUnavailable: 0 (keep all pods running)
   - minReadySeconds: 10 (wait before continuing rollout)

2. Graceful Shutdown:
   - terminationGracePeriodSeconds: 30 (allow Phoenix to drain connections)

3. PodDisruptionBudget:
   - minAvailable: 1 (ensure at least 1 pod always available)
   - Prevents all pods from being terminated simultaneously

4. Improved Health Checks:
   - Faster readiness probe (5s initial, 3s period)
   - More aggressive success/failure thresholds
   - New pods marked ready faster

This ensures new pods are fully ready and serving traffic before old
pods are terminated, maintaining WebSocket connections and preventing
'something went wrong' errors during deployments.
This commit is contained in:
Graham McIntire 2026-01-17 12:44:22 -06:00
parent 44c71bca96
commit 9c2d95b55a
No known key found for this signature in database
3 changed files with 26 additions and 2 deletions

View file

@ -6,6 +6,12 @@ metadata:
namespace: towerops
spec:
replicas: 2
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1 # Allow 1 extra pod during rollout
maxUnavailable: 0 # Keep all pods running during rollout
minReadySeconds: 10 # Wait 10s after pod is ready before continuing
selector:
matchLabels:
app: towerops
@ -14,6 +20,7 @@ spec:
labels:
app: towerops
spec:
terminationGracePeriodSeconds: 30 # Allow 30s for graceful shutdown
imagePullSecrets:
- name: gitlab-registry
securityContext:
@ -95,9 +102,14 @@ spec:
port: 4000
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 3
failureThreshold: 3
readinessProbe:
httpGet:
path: /health
port: 4000
initialDelaySeconds: 10
periodSeconds: 5
initialDelaySeconds: 5
periodSeconds: 3
timeoutSeconds: 2
successThreshold: 1
failureThreshold: 2

View file

@ -8,3 +8,4 @@ resources:
- service-headless.yaml
- certificate.yaml
- ingressroute.yaml
- poddisruptionbudget.yaml

View file

@ -0,0 +1,11 @@
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: towerops-pdb
namespace: towerops
spec:
minAvailable: 1 # Keep at least 1 pod running during voluntary disruptions
selector:
matchLabels:
app: towerops