docs: update CLAUDE.md to reflect Oban worker architecture

Changes:
- Replace Exq references with Oban in LiveDashboard metrics section
- Add new 'Background Job Architecture' section documenting Oban workers
- Document DevicePollerWorker, DeviceMonitorWorker, DiscoveryWorker
- Update SNMP Polling Architecture to reference new workers
- Document automatic job lifecycle management
- Clarify direct Oban worker pattern (no GenServer/coordinator layer)

This ensures documentation matches the refactored architecture.
This commit is contained in:
Graham McIntire 2026-01-24 16:42:17 -06:00
parent 92cd812806
commit 5109f7b6a1
No known key found for this signature in database

View file

@ -172,10 +172,10 @@ LiveDashboard is available at `/dashboard` (requires authentication in productio
- **Phoenix Metrics**: Request duration, route timings, channel/socket metrics
- **Database Metrics**: Query time, decode time, queue time, idle time
- **VM Metrics**: Memory usage, run queue lengths
- **Exq/Background Jobs**:
- Queue sizes (default, discovery, polling, monitoring, maintenance)
- Busy worker processes
- Total worker processes
- **Oban/Background Jobs**:
- Queue sizes (default, discovery, pollers, monitors, maintenance)
- Currently executing jobs
- Available jobs waiting to be processed
- **Redis/Valkey**:
- Connected clients
- Memory usage (MB)
@ -186,7 +186,7 @@ Metrics are collected every 10 seconds via `telemetry_poller` and published usin
To view metrics:
1. Navigate to `/dashboard` (or `/dev/dashboard` in development)
2. Click the "Metrics" tab
3. Scroll to see Exq and Redis sections
3. Scroll to see Oban and Redis sections
## Project-Specific Constraints
@ -277,21 +277,62 @@ curl https://towerops.net/api/v1/sites \
- Sites: GET /api/v1/sites, POST /api/v1/sites, GET /api/v1/sites/:id, PATCH /api/v1/sites/:id, DELETE /api/v1/sites/:id
- Devices: GET /api/v1/devices, POST /api/v1/devices, GET /api/v1/devices/:id, PATCH /api/v1/devices/:id, DELETE /api/v1/devices/:id
### Background Job Architecture (Oban)
The application uses **Oban** (PostgreSQL-backed job queue) for all background processing. Jobs are distributed across the cluster and automatically retried on failure.
**Oban Queues:**
- `default` (10 workers) - General background tasks
- `discovery` (10 workers) - SNMP discovery jobs
- `pollers` (50 workers) - SNMP polling (one job per device)
- `monitors` (50 workers) - Device health monitoring (one job per device)
- `maintenance` (5 workers) - Cleanup and housekeeping tasks
**Key Oban Workers:**
1. **`DevicePollerWorker`** - SNMP polling for time-series data
- Runs every 60 seconds per device (configurable via `check_interval_seconds`)
- Collects: sensor readings, interface stats, neighbor topology, ARP/MAC tables
- Uses Oban `unique` constraint to prevent duplicate jobs
- Self-schedules next poll after completion
- Broadcasts PubSub events for real-time UI updates
2. **`DeviceMonitorWorker`** - Health monitoring via ping
- Runs every 30 seconds per device
- Performs ICMP ping checks to detect device up/down status
- Creates alerts on status changes
- Self-schedules next check after completion
3. **`DiscoveryWorker`** - One-time SNMP discovery
- Triggered manually or on device creation
- Collects: system info, interfaces, sensors, neighbors
- Creates/updates SNMPDevice record with full topology
4. **`NeighborCleanupWorker`** - Maintenance task
- Runs periodically to clean up stale neighbor data
- Deletes neighbors older than 24 hours
**Automatic Job Management:**
- Jobs are automatically started when `monitoring_enabled` or `snmp_enabled` is set to `true`
- Jobs are automatically stopped when disabled or device is deleted
- All job lifecycle managed in `Devices` context (create_device, update_device, delete_device)
### SNMP Polling Architecture
The application has two separate SNMP collection mechanisms:
**1. Discovery (`Towerops.Snmp.Discovery`)** - One-time or manual collection
- Triggered manually by user or on a schedule
- Triggered manually by user or via `DiscoveryWorker` Oban job
- Collects: system info, device identification, interfaces, sensors, **neighbors**
- Creates/updates the device record and all associated data
- Typically runs when equipment is first added or when user clicks "Rediscover"
**2. Polling (`Towerops.Snmp.PollerWorker`)** - Continuous time-series collection
**2. Polling (`DevicePollerWorker`)** - Continuous time-series collection
- Runs automatically every 60 seconds (configurable per equipment)
- Collects: sensor readings, interface statistics, **neighbor topology**
- Updates time-series data without recreating device records
- Keeps neighbor data fresh by polling LLDP/CDP every minute
- **Architecture**: Direct Oban worker (no GenServer or coordinator layer)
**Neighbor Data Lifecycle:**
- Discovery saves initial neighbors with 5-minute stale threshold