When the WebSocket connection drops, session cleanup waits for all
worker pools to drain. SNMP/MikroTik operations against unresponsive
devices can block for 30+ seconds per worker, preventing the agent
from reconnecting during that time.
Add stopWithTimeout(5s) so pool shutdown is bounded. If workers don't
finish within the timeout, they're abandoned and reconnection proceeds
immediately.
add SslCheckConfig protobuf message and executeSSLCheck function that
connects via crypto/tls, reads the peer certificate expiry, and returns
OK/WARNING/CRITICAL based on a configurable warning_days threshold.
CRITICAL: The agent never reset its exponential backoff delay after
successful connections, causing permanent 60s reconnect delays after
multiple deploys.
Root cause:
- retryDelay started at 1s, doubled to 60s cap on each disconnect
- Never reset when connection succeeded
- After ~6 deploys, agent permanently waited 60s between reconnects
Fixes:
1. Reset backoff to 1s after session runs for 30s+ (indicates stable connection)
2. Reduce max retry from 60s → 10s (60s is too long for production)
Impact:
- After Phoenix deploy, agent reconnects within seconds instead of minutes
- Exponential backoff still prevents connection storms on real failures
- Typical reconnect sequence: 1s → 2s → 4s → 8s → 10s (cap)
- Fix blocking MikroTik channel sends that could leak goroutines
- Default to 10s timeout when check TimeoutMs is 0
- Add payload size limit to check_jobs event
- Add 30s write deadline to WebSocket writes
- Use random ICMP sequence numbers to prevent ping ID collisions
- Atomic host key file writes via temp+rename
- Sanitize update URLs in log output
- Zero correct buffer in sendBinaryResult after MarshalAppend
- HTTP checks: use io.ReadAll with LimitReader instead of single Read()
to handle bodies spanning multiple TCP segments
- HTTP checks: enforce TLS 1.2 minimum, matching websocket and mikrotik
- Ping/credential jobs: use select/default for channel sends to prevent
goroutine leaks when result channel is full
- LLDP: check strconv.Atoi errors in IPv6 address parsing instead of
silently producing incorrect addresses
- SNMP: use comma-ok type assertions in snmpValueToString to prevent
panics from malformed device responses
- WebSocket: read handshake response line-by-line via bufio.Reader
instead of single conn.Read that could miss multi-segment responses
Adds LLDP (Link Layer Discovery Protocol) topology discovery capability
to the agent for local network neighbor detection.
New Features:
- LLDP-MIB walker for discovering network neighbors via SNMP
- New LLDP_TOPOLOGY job type for topology discovery jobs
- Automatic IPv4 and IPv6 management address parsing
- Bulk neighbor discovery with full port information
Protocol Buffers:
- Added LldpTopologyResult message for topology data
- Added LldpNeighbor message for individual neighbor info
- Extended JobType enum with LLDP_TOPOLOGY (value 5)
Implementation:
- lldp.go: LLDP discovery logic with SNMP walking
- agent.go: Job dispatch and result channel handling
- Updated handleMessage signature to support LLDP results
- Result sent to Phoenix via "lldp_topology_result" event
LLDP-MIB OIDs:
- 1.0.8802.1.1.2.1.3.3.0 (lldpLocSysName)
- 1.0.8802.1.1.2.1.3.7.1.4 (lldpLocPortDesc)
- 1.0.8802.1.1.2.1.4.1.1.7 (lldpRemPortId)
- 1.0.8802.1.1.2.1.4.1.1.8 (lldpRemPortDesc)
- 1.0.8802.1.1.2.1.4.1.1.9 (lldpRemSysName)
- 1.0.8802.1.1.2.1.4.2.1.3 (lldpRemManAddr)
This enables the Phoenix web application to dispatch LLDP discovery
jobs to agents for local neighbor detection, reducing SNMP load on the
Phoenix server and improving discovery performance.
If the initial connection attempt fails (e.g. server returns 403 over
IPv6), retry with tcp4 to force IPv4. This handles cases where DNS
returns AAAA records but the server isn't properly configured on IPv6.
sync.Once contains sync.noCopy and cannot be assigned by value.
Reset with a fresh sync.Once{} in defer instead of save/restore.
Also gitignore known_hosts.json runtime data.
Add Forgejo workflow and renovate.json config to automatically
check for gomod, Docker, and GitHub Actions dependency updates.
Also bumps golang.org/x/net v0.50.0 → v0.51.0.
- Add missing checkResultCh parameter to all handleMessage() calls in tests
- Add missing checkResultCh parameter to all dispatchJob() calls in tests
- Add checks worker pool to testPools() function
- Fix IPv6 address formatting in executeTCPCheck using net.JoinHostPort
- Add strconv import for port conversion
Extends agent to execute HTTP, TCP, and DNS service checks.
Agent Changes:
- Extended protobuf schema with Check, CheckResult, CheckList messages
- Added checks.go with HTTP/TCP/DNS executors (completely stateless)
- Added checks worker pool (50 workers)
- Handles check_jobs WebSocket event from Phoenix
- Reports check_result back to Phoenix
Check Executors:
- HTTP: Full request support with regex, SSL, headers
- TCP: Port connectivity with send/expect patterns
- DNS: Resolution for A, AAAA, CNAME, MX, TXT records
Architecture: Agent is stateless - receives jobs, executes, reports back.
Phoenix handles all scheduling and state tracking.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
The old code only tried unprivileged UDP ICMP (udp4), which relies on
the ping_group_range sysctl — not CAP_NET_RAW. Docker doesn't set that
sysctl, so pings always timed out and fell through to exec. Now tries
raw ICMP sockets (ip4:icmp) first, which properly uses CAP_NET_RAW.
Also sets cap_net_raw+ep on the agent binary in the Dockerfile.
Previously the agent proceeded to the main loop without confirming
the server accepted the channel join. A rejected join (e.g. invalid
token) left the agent running but silently non-functional. Now waits
for the join reply with a 10s timeout, returns error on rejection.
Reader goroutine starts before join send so the reply can be received.
Writer goroutine starts after join confirmation.
Previously, resetting slice length to 0 left previous protobuf data
(device IDs, OID values, credentials) in the backing array. Now zeros
the full capacity before returning to the pool.
handleMessage returns bool to signal session end instead of calling
osExit directly. Defers (pool stop, writeCh close, writerWg wait)
now run before exit, preventing in-flight result loss and panics
from writes to closed channels.
Fix 1: Limit inbound job payload size to 4MB to prevent OOM from malicious server
Fix 2: Worker pool submit accepts context and returns bool, prevents blocking event loop
Fix 3: Panic recovery in worker pool goroutines prevents permanent worker death
Fix 4: WebSocket handshake timeout (30s) prevents indefinite blocking on slow server
Fix 5: Writer goroutine propagates errors to main loop via writeErrCh
Use sync.Pool to reuse 4KB byte slices for protobuf marshaling via
MarshalAppend instead of allocating fresh buffers per result message.
Reduces GC pressure under high-throughput polling.
Accumulate SNMP results and flush in batches of 50 or every 100ms,
whichever comes first. Reduces per-message overhead when thousands
of devices produce results near-simultaneously. Also flushes
remaining results on disconnect/shutdown.
Build the entire frame (header + mask key + masked payload) in one
buffer and write it with a single syscall instead of two separate
writes. Removes the failOnNthWrite test that tested the old
two-write implementation detail.
Wrap the WebSocket connection in an 8KB bufio.Reader so that small
reads (2-byte frame headers, length extensions) are served from
buffer instead of making individual syscalls.
Raise write channel from 500 to 10000, SNMP/monitoring result
channels from 1000 to 10000, MikroTik/credential channels to 5000.
Prevents silent data loss when polling thousands of devices.
