musl libc has weak fork-safety guarantees in multi-threaded processes.
When Tokio's runtime is running and we fork() for SNMP operations, musl's
internal state can be inconsistent in the child, causing SIGSEGV on every
SNMP walk to Ubiquiti devices. glibc handles this via pthread_atfork.
- Dockerfile: Alpine → rust:1.93-bookworm build + debian:12-slim runtime
- CI: Remove ensure-netsnmp-base/manifest jobs, simplify build deps
- Delete Dockerfile.netsnmp and netsnmp-base.yml workflow
- Add NULL pointer guards in snmp_helper.c for fork-safety
- Add SNMP exception type handling (NoSuchObject/Instance/EndOfMibView)
- Add crash reproduction tests
In Fork mode the parent never calls snmp_init_library(), so
pthread_once remains untriggered. The child called init_snmp()
directly, then snmp_open_session() triggered pthread_once calling
init_snmp() a second time. This double-initialization of libnetsnmp
caused signal 11 in every child process.
Let snmp_open_session() handle initialization via pthread_once,
which correctly initializes exactly once and sets numeric OID format.
Net-SNMP was crashing (signal 11) on every operation despite fork isolation.
The issue was that init_snmp() tries to load MIB files by default, which can
cause crashes if MIB files are missing, corrupted, or incompatible.
Since we only use numeric OIDs (e.g., "1.3.6.1.2.1.1.1.0"), we don't need
MIB name resolution at all. This change explicitly disables MIB loading in
the child process by:
1. Setting MIBS and MIBDIRS environment variables to empty
2. Calling netsnmp_set_mib_directory("") to disable MIB directory search
3. These must happen BEFORE calling init_snmp()
This should fix the persistent SIGSEGV crashes in SNMP operations.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Each SNMP GET/WALK now runs in a forked child process. If libnetsnmp
triggers a SIGSEGV or other fatal signal, only the child dies - the
parent agent logs the crash and continues operating for all other devices.
Key changes:
- C helper: snmp_get_isolated() and snmp_walk_isolated() using
fork+pipe pattern with 60s alarm watchdog and mutex-serialized forks
- Rust: IsolationMode enum (Fork/Direct) controlled by
TOWEROPS_SNMP_ISOLATION env var, defaults to Fork
- New CrashRecovered error variant with signal info and logging
- Device poller logs crash recovery events at error level
- Startup logs active isolation mode
Set TOWEROPS_SNMP_ISOLATION=direct to disable isolation for debugging.
