fix: reset reconnect backoff after successful connection

CRITICAL: The agent never reset its exponential backoff delay after
successful connections, causing permanent 60s reconnect delays after
multiple deploys.

Root cause:
- retryDelay started at 1s, doubled to 60s cap on each disconnect
- Never reset when connection succeeded
- After ~6 deploys, agent permanently waited 60s between reconnects

Fixes:
1. Reset backoff to 1s after session runs for 30s+ (indicates stable connection)
2. Reduce max retry from 60s → 10s (60s is too long for production)

Impact:
- After Phoenix deploy, agent reconnects within seconds instead of minutes
- Exponential backoff still prevents connection storms on real failures
- Typical reconnect sequence: 1s → 2s → 4s → 8s → 10s (cap)
This commit is contained in:
Graham McIntire 2026-03-10 14:35:42 -05:00
parent 08d5957690
commit ee82509467
No known key found for this signature in database

View file

@ -42,7 +42,8 @@ type channelMsg struct {
func runAgent(ctx context.Context, wsURL, token string) {
baseURL := strings.TrimRight(wsURL, "/")
retryDelay := time.Second
maxRetry := 60 * time.Second
maxRetry := 10 * time.Second
const successfulConnectionThreshold = 30 * time.Second
for {
select {
@ -51,7 +52,18 @@ func runAgent(ctx context.Context, wsURL, token string) {
default:
}
sessionStart := time.Now()
err := runSession(ctx, baseURL, token)
sessionDuration := time.Since(sessionStart)
// Reset backoff if session ran successfully for a while (indicates stable connection)
if sessionDuration >= successfulConnectionThreshold {
slog.Debug("resetting reconnect backoff after successful session",
"duration", sessionDuration,
"previous_delay", retryDelay)
retryDelay = time.Second
}
if ctx.Err() != nil {
return
}