graham/towerops
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
towerops/priv
History
Graham McIntire cedbc5abd1
fix: resolve critical data loss and protobuf schema bugs (Phase 1) 
This commit addresses Phase 1 (CRITICAL - Data Loss Prevention) issues
from the comprehensive bugs and inconsistencies analysis.

CRITICAL FIXES:

1. Protobuf Schema Inconsistencies
   - Added missing job_id field (field 5) to SnmpResult message
   - Fixed AgentError message structure (job_id, message, timestamp)
   - Removed obsolete transport field from SnmpDevice struct
   - Regenerated protobuf code with protoc-gen-elixir
   - Prevents crashes when agent sends SNMP results or errors

2. Interface/Sensor Discovery Timeout Data Loss
   - Changed timeout behavior to return error instead of empty list
   - Prevents deletion of ALL existing interfaces/sensors on slow networks
   - Discovery fails gracefully instead of destroying historical data
   - Addresses commit 7a57f7c sensor discovery pipeline data loss issue

3. Polling Offset Schedule Mismatch
   - Fixed schedule_next_poll() to use offset only (not interval + offset)
   - Maintains consistent polling intervals across all executions
   - Prevents load bunching that offset was designed to prevent
   - Example: 300s interval with offset=94s now consistently polls every 5m

4. Always Reschedule Race Condition
   - Added should_continue_polling?() and should_continue_monitoring?()
   - Only reschedule if device is still enabled and should be polled by Phoenix
   - Prevents zombie jobs from continuing to poll disabled/deleted/reassigned devices
   - Stops race condition where stop_polling() is bypassed by in-flight jobs

Files Changed:
- priv/proto/agent.proto
- lib/towerops/proto/agent.pb.ex (regenerated)
- lib/towerops_web/channels/agent_channel.ex
- lib/towerops/snmp/discovery.ex
- lib/towerops/workers/device_poller_worker.ex
- lib/towerops/workers/device_monitor_worker.ex
- test/towerops/workers/device_poller_worker_test.exs

Test Results: All passing
- device_poller_worker_test.exs: 8 tests, 0 failures
- device_monitor_worker_test.exs: 3 tests, 0 failures
- discovery_test.exs: 13 tests, 0 failures

Remaining Phases:
- Phase 2 (HIGH): 8 data integrity issues
- Phase 3 (MEDIUM): 10 reliability issues
- Phase 4 (LOW): 3 cleanup issues

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 09:16:03 -06:00
..
gettext feat: migrate user settings and agent management to gettext 2026-02-02 13:20:54 -06:00
mibs mib update from librenms 2026-01-28 13:38:31 -06:00
profiles fix airfiber identification 2026-02-05 14:08:34 -06:00
proto fix: resolve critical data loss and protobuf schema bugs (Phase 1) 2026-02-09 09:16:03 -06:00
repo fix: resolve critical sensor discovery pipeline data loss 2026-02-09 08:32:24 -06:00
static security: add session inactivity timeout and comprehensive security documentation 2026-02-08 10:47:44 -06:00