- agent.go: guard reader goroutine msgCh send with sessionCtx.Done()
to prevent permanent goroutine hang
- checks.go: cache regex.Compile (not MatchString) per HTTP check;
cache x509.SystemCertPool via sync.Once across SSL checks
- lldp.go: remove dead walkErrors variable
- ssh_test.go: reset global host key store to fix TOFU port collision
flakiness across tests
- .gitignore: add towerops-agent binary, .tool-versions
- Override wsHandshakeTimeout in 3 websocket tests that waited 30s each
- Make initialRetryDelay a package variable for faster retry tests
- Reduce time.Sleep calls across agent_test.go (1.5s->150ms, 3s->1s, etc.)
- Update Makefile with test-fast and test-short targets, add -count=1
- Update golang.org/x/crypto, x/net, x/sys to latest minors
- Fix TestWSDialDefaultPorts to use mocked dials instead of real TCP
Mark all critical and high severity issues as FIXED (11/11).
Defer JSON Marshal optimization as lower priority.
Document additional fixes made during testing phase.
Final fixes to pass all tests after audit improvements:
- Fixed reader goroutine context handling: added dedicated goroutine that
closes WebSocket connection on context cancellation, enabling fast shutdown
- Fixed HTTP client timeout: shared transports but per-check client with
custom timeout and redirect policy
- Fixed HTTP redirect handling: CheckRedirect function properly respects
FollowRedirects config
- Updated TestWSDialWriteHandshakeError: removed duplicate assertion, now
accepts both 'write handshake' and 'set deadline' errors
- Updated job executor tests: changed to expect error results on nil device
(executeSnmpJob, executeCredentialTest, executePingJob, executeMikrotikJob)
- Added missing 'strings' import to snmp_test.go
All 249 tests now pass with 97.6% coverage. go vet clean.
Remove duplicate assertion that expected only 'write handshake' error.
After SetDeadline error checking improvements, closed connections now
properly return 'set deadline' errors instead of proceeding to write.
Test now accepts both error types as valid.
When the WebSocket breaks (broken pipe), the main loop can get stuck
in handleMessage -> dispatchJob -> submit if the worker pool is full.
submit blocks on ctx.Done() but ctx is the parent context which only
cancels on SIGINT/SIGTERM, not on connection errors.
Create a session-scoped context that cancels when read/write errors
occur. Pass it to handleMessage so blocked submit calls unblock
immediately, allowing the main loop to receive the write error and
reconnect.
Previously the agent called os.Exit(0) when receiving a server restart
event, requiring Docker or manual intervention to bring it back. Now
it reconnects immediately with a fresh session, which is better for
both dev (no lost process) and production (faster recovery).
When the WebSocket connection drops, session cleanup waits for all
worker pools to drain. SNMP/MikroTik operations against unresponsive
devices can block for 30+ seconds per worker, preventing the agent
from reconnecting during that time.
Add stopWithTimeout(5s) so pool shutdown is bounded. If workers don't
finish within the timeout, they're abandoned and reconnection proceeds
immediately.
add SslCheckConfig protobuf message and executeSSLCheck function that
connects via crypto/tls, reads the peer certificate expiry, and returns
OK/WARNING/CRITICAL based on a configurable warning_days threshold.
CRITICAL: The agent never reset its exponential backoff delay after
successful connections, causing permanent 60s reconnect delays after
multiple deploys.
Root cause:
- retryDelay started at 1s, doubled to 60s cap on each disconnect
- Never reset when connection succeeded
- After ~6 deploys, agent permanently waited 60s between reconnects
Fixes:
1. Reset backoff to 1s after session runs for 30s+ (indicates stable connection)
2. Reduce max retry from 60s → 10s (60s is too long for production)
Impact:
- After Phoenix deploy, agent reconnects within seconds instead of minutes
- Exponential backoff still prevents connection storms on real failures
- Typical reconnect sequence: 1s → 2s → 4s → 8s → 10s (cap)
- Fix blocking MikroTik channel sends that could leak goroutines
- Default to 10s timeout when check TimeoutMs is 0
- Add payload size limit to check_jobs event
- Add 30s write deadline to WebSocket writes
- Use random ICMP sequence numbers to prevent ping ID collisions
- Atomic host key file writes via temp+rename
- Sanitize update URLs in log output
- Zero correct buffer in sendBinaryResult after MarshalAppend
- HTTP checks: use io.ReadAll with LimitReader instead of single Read()
to handle bodies spanning multiple TCP segments
- HTTP checks: enforce TLS 1.2 minimum, matching websocket and mikrotik
- Ping/credential jobs: use select/default for channel sends to prevent
goroutine leaks when result channel is full
- LLDP: check strconv.Atoi errors in IPv6 address parsing instead of
silently producing incorrect addresses
- SNMP: use comma-ok type assertions in snmpValueToString to prevent
panics from malformed device responses
- WebSocket: read handshake response line-by-line via bufio.Reader
instead of single conn.Read that could miss multi-segment responses
Adds LLDP (Link Layer Discovery Protocol) topology discovery capability
to the agent for local network neighbor detection.
New Features:
- LLDP-MIB walker for discovering network neighbors via SNMP
- New LLDP_TOPOLOGY job type for topology discovery jobs
- Automatic IPv4 and IPv6 management address parsing
- Bulk neighbor discovery with full port information
Protocol Buffers:
- Added LldpTopologyResult message for topology data
- Added LldpNeighbor message for individual neighbor info
- Extended JobType enum with LLDP_TOPOLOGY (value 5)
Implementation:
- lldp.go: LLDP discovery logic with SNMP walking
- agent.go: Job dispatch and result channel handling
- Updated handleMessage signature to support LLDP results
- Result sent to Phoenix via "lldp_topology_result" event
LLDP-MIB OIDs:
- 1.0.8802.1.1.2.1.3.3.0 (lldpLocSysName)
- 1.0.8802.1.1.2.1.3.7.1.4 (lldpLocPortDesc)
- 1.0.8802.1.1.2.1.4.1.1.7 (lldpRemPortId)
- 1.0.8802.1.1.2.1.4.1.1.8 (lldpRemPortDesc)
- 1.0.8802.1.1.2.1.4.1.1.9 (lldpRemSysName)
- 1.0.8802.1.1.2.1.4.2.1.3 (lldpRemManAddr)
This enables the Phoenix web application to dispatch LLDP discovery
jobs to agents for local neighbor detection, reducing SNMP load on the
Phoenix server and improving discovery performance.
If the initial connection attempt fails (e.g. server returns 403 over
IPv6), retry with tcp4 to force IPv4. This handles cases where DNS
returns AAAA records but the server isn't properly configured on IPv6.
