Graham McIntire
ee8a3220c4
Replaced GenServer-based periodic workers with Oban Cron jobs to improve
pod rollover resilience and simplify architecture.
Worker Changes:
- NeighborCleanupWorker: GenServer → Oban Cron (hourly)
- Cleans stale neighbors, ARP entries, and MAC addresses
- Runs every hour via Oban.Plugins.Cron
- StaleAgentWorker: GenServer → Oban Cron (every minute)
- Detects agents that haven't checked in for 10+ minutes
- Refactored to reduce nesting (extracted helper functions)
- Removed stateful tracking (now stateless, re-evaluates each run)
- AgentLatencyEvaluator: GenServer → Oban Cron (every 5 minutes)
- Latency-based agent reassignment with 20% threshold
- Removed trigger_evaluation/0 (no longer needed)
- JobHealthCheckWorker: NEW Oban Cron worker (every 10 minutes)
- Safety net to recover missing monitor/poller jobs
- Auto-creates jobs for devices with monitoring/SNMP enabled
Infrastructure Changes:
- Removed Monitoring.Supervisor (no longer needed)
- Updated application.ex to remove GenServer workers from supervision tree
- Added Oban.Plugins.Cron to dev.exs and runtime.exs
- All workers now run cluster-wide via PostgreSQL-backed coordination
Test Updates:
- Updated all worker tests to call perform(%Oban.Job{args: %{}})
- Removed GenServer lifecycle tests (start_link, send messages, etc.)
- Removed async sleep calls (no longer needed)
Benefits:
- Better pod rollover resilience (Cron jobs run cluster-wide)
- Simpler architecture (no GenServers for periodic tasks)
- Better observability (all jobs visible in Oban dashboard)
- Safety net for missing jobs (JobHealthCheckWorker)
- Stateless workers (easier to reason about and test)
Documentation:
- Updated CLAUDE.md with Background Job Architecture section
- Documented job types, queues, and resilience features
All tests passing (3,686 tests, 0 failures).
|
||
|---|---|---|
| .. | ||
| mix/tasks | ||
| snmpkit | ||
| towerops | ||
| towerops_web | ||
| snmp_lib.ex | ||
| snmp_mgr.ex | ||
| snmpkit.ex | ||
| towerops.ex | ||
| towerops_web.ex | ||