Completes full migration from device-level polling to per-check polling.
Migration Changes:
- Removed DevicePollerWorker scheduling from device lifecycle
- Checks automatically scheduled when created
- Discovery creates and schedules checks atomically
New Functions:
- Monitoring.stop_device_checks/1: Cancel all check jobs for device
- Monitoring.disable_device_checks/1: Disable checks when SNMP off
Discovery Integration:
- All create_*_check functions now schedule checks after creation
JobHealthCheckWorker Updates:
- Removed DevicePollerWorker recovery logic
- Only recovers DeviceMonitorWorker jobs now
Test Updates:
- Marked DevicePollerWorkerTest as skipped (deprecated)
- Updated JobHealthCheckWorkerTest expectations
All 6953 tests passing, 0 failures
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit implements the unified checks architecture that consolidates
SNMP monitoring with HTTP/TCP/DNS checks under a single "check" abstraction,
enabling consistent graphing, alerting, and management across all check types.
Database Changes:
- Add source_type and source_id to checks table for tracking auto-discovered
vs manually created checks
- Add value field to check_results for storing numeric sensor readings
- Maintain backward compatibility with existing check_results data
New SNMP Executors:
- SnmpSensorExecutor: Poll sensor OIDs and return formatted values with
status determination (OK/WARNING/CRITICAL based on limits)
- SnmpInterfaceExecutor: Poll interface stats (bandwidth, packets, errors)
- SnmpProcessorExecutor: Poll CPU/processor usage
- SnmpStorageExecutor: Poll disk/memory usage with percentage calculations
Check Execution Worker:
- CheckExecutorWorker: Unified Oban worker that dispatches to appropriate
executor based on check_type (snmp_sensor, snmp_interface, http, tcp, etc.)
- Self-schedules next execution with distributed polling offsets
- Records results in check_results TimescaleDB hypertable
- Updates check state (OK/WARNING/CRITICAL/UNKNOWN)
Discovery Integration:
- Auto-creates checks during SNMP discovery for sensors, interfaces,
processors, and storage
- Links checks to source entities via source_id for data lookup
- Enables/disables checks based on discovery results
UI Enhancements:
- Checks tab on device detail page with grouped display
- FormComponent for adding manual HTTP/TCP/DNS checks
- Empty state with "Run Discovery" prompt
- Check status badges and last checked times
Graphing:
- Update GraphLive to accept check_id parameter
- Query check_results table for time-series data
- Support all check types (SNMP, HTTP response times, etc.)
Testing:
- Comprehensive test suite for SnmpSensorExecutor (5 tests)
- Test suite for CheckExecutorWorker (7 tests)
- Test coverage for discovery check creation (6 tests)
- Remove deprecated monitoring_test.exs testing old API
Bug Fixes:
- Fix SNMP executors reading credentials from correct Device schema fields
(device.snmp_version instead of device.snmp_device.version)
- Update agent channel test to query MonitoringCheck table directly
Code Quality:
- Extract add_snmp_credentials helper to reduce cyclomatic complexity
- Use map-based lookups for sensor formatting and check type grouping
- Apply pattern matching in dispatcher to reduce complexity
- All credo checks passing with no issues
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- create MonitoringCheck schema for the monitoring_checks table, fixing
silent data loss where ping results were dropped by the wrong schema
- update agent channel and device monitor worker to use new schema
- fix Stats queries to use MonitoringCheck instead of Check schema
- add list_online_cloud_pollers and list_cloud_polled_devices queries
- add CloudLatencyProbeWorker to dispatch cross-agent latency probes
every 8 hours via PubSub to all online cloud pollers
- scope AgentLatencyEvaluator to cloud pollers only with cloud_only
filter, lower min_checks (5), wider time window (48h), and
device-level assignments to avoid changing site/org defaults
- update cron schedule: probes at 0/8/16h, evaluation at 0:30/8:30/16:30
- refactor monitoring.ex and http_executor.ex to fix credo complexity
- Add 'Checks' tab link to device navigation
- Display empty state when no checks configured
- Add tests for tab visibility and empty state
- Fix unused variable warnings in monitoring context
- Remove unused CheckResult alias and module attributes
- Add stub for legacy get_latency_data for backward compatibility
Tests: 2 new tests added, all passing
Following TDD: RED-GREEN cycle verified for each feature
Enhanced AgentChannel logging to track device IDs across job list updates.
Added tests verifying server correctly excludes deleted devices from job lists.
Root cause: Bug is in Go agent (appends jobs without clearing), not Phoenix server.
Build a rich network topology from SNMP polling data using evidence-based
confidence scoring. LLDP/CDP neighbors, MAC address tables, and ARP data
are combined to infer device links with weighted confidence merging.
- Add DeviceLink and DeviceLinkEvidence schemas for persistent topology
- Implement evidence collectors: LLDP (0.95), CDP (0.95), MAC (0.7), ARP (0.6)
- Add device role inference from sysObjectID/sysDescr patterns
- Hook topology inference into DevicePollerWorker pipeline
- Add stale link cleanup (24h mark stale, 72h delete) via NeighborCleanupWorker
- Update NetworkMapLive with "Added" vs "All Devices" tabs
- Add connected devices section to device detail page
- Add device role selector to device edit form
- Update Cytoscape.js with role-based node shapes/colors and confidence edges
Infers device roles from LLDP/CDP capabilities, interface count,
vendor/manufacturer, and system description. Only overwrites
inferred roles, never manual overrides. Wired into process_device
pipeline.
Orchestrates evidence collection from LLDP, CDP, MAC, and ARP
data. Groups evidence by remote device, resolves to managed
devices, upserts links with merged confidence. Broadcasts
topology changes via PubSub. Skips self-links.
LLDP/CDP evidence (0.95 confidence) from snmp_neighbors.
MAC evidence (0.7) cross-references MAC tables with device
interface MACs. ARP evidence (0.6) matches by IP and MAC.
Self-links excluded. Unmatched entries tracked as discovered.
The plug was attempting to register a before_send callback after
blocking a banned IP and sending a 403 response. This caused
Plug.Conn.AlreadySentError in production.
Fixed by checking conn.halted before attempting to register the
404 tracking callback. If the request was already blocked and sent,
we skip the callback registration.
Also updated the test to verify proper behavior without the
workaround for AlreadySentError.
Ecto schemas for persistent topology model. DeviceLink tracks
connections between devices with confidence scores. DeviceLinkEvidence
records why each link is believed to exist. Device schema gets
device_role and device_role_source fields.
Filter port_died and write_failed/epipe errors from Honeybadger reports,
email notifications, and logs. Reorder supervision tree so Oban drains
before its dependencies shut down, and add 40s shutdown grace period.
Phase 3 audit found screenos.yaml detection profile was not registered
in the Juniper vendor module. Added to profile_names list.
ScreenOS is Juniper's legacy firewall operating system (pre-Junos).
Coverage: 5 Juniper profiles now registered
- juniper-mss
- juniperex2500os
- junos
- junose
- screenos (newly added)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Always broadcast from update_device_status/2 so the device index
LiveView refreshes even when status hasn't changed. Add a 15-second
tick timer to keep "Last Checked" timestamps fresh between polls.
Scope alert PubSub topics to organization for defense-in-depth.
Add a Honeybadger NoticeFilter that sends an email with the full
error class, message, stacktrace, and server info to graham@mcintire.me
whenever an exception is reported. The notice passes through unchanged
so Honeybadger still receives it normally.
Replace individual Repo.insert() with Repo.insert_all() for sensor readings,
interface stats, processor readings, and storage readings in agent channel and
device poller worker. Add partial/composite database indexes for common query
patterns. Dashboard now uses GROUP BY aggregation instead of loading all devices.
DeviceLive.Show only reloads data for the active tab on PubSub events.
DeviceLive.Index debounces status changes and skips quota queries on updates.
Agent polling preloads filtered to monitored sensors/interfaces only.
Changed expect() to stub() in all discover_wireless_sensors/1 tests
to handle multiple SNMP get calls:
- Static sensor fetch calls Client.get() for CPU
- ePMP mode detection calls Client.get() for wirelessInterfaceMode
- Each sensor value fetch calls Client.get() (RSSI, SNR, Frequency, Clients)
Using stub() allows unlimited calls vs expect() which enforces
exactly one call, fixing Mox.UnexpectedCallError in all tests.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Update AgentLive.Index to properly handle browser back/forward buttons
by syncing the setup token modal to URL parameters.
Changes:
- Update handle_params/3 to read modal param and set modal state
- Update show_setup handler to use push_patch with ?modal=setup
- Update close_token_modal handler to use push_patch (remove modal param)
- Update handle_agent_creation_success to use push_patch after creating agent
Behavior:
- Browser back button now closes the setup modal without navigating away
- URL reflects current modal state (?modal=setup)
- Bookmarkable URLs restore exact modal state (can share agent setup URL)
- After creating agent, modal opens via URL state
Follows idiomatic Phoenix LiveView pattern using push_patch/2 and
handle_params/3 as documented in CLAUDE.md.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Implemented all critical and nice-to-have Arista sensor improvements identified
in LibreNMS parity audit. Upgrades Arista EOS support from 85% to 100% parity.
Files Changed:
- lib/towerops/snmp/profiles/vendors/arista.ex (enhanced)
Added 4 major enhancements:
1. DOM power conversion (watts→dBm): Detects optical power sensors via regex,
converts using formula dBm = 10*log10(watts*1000), preserves original value
in metadata
2. Arista threshold discovery: Walks ARISTA-ENTITY-SENSOR-MIB threshold table
(OID 1.3.6.1.4.1.30065.3.3.1.1.1), applies 4 threshold types (low_critical,
low_warn, high_warn, high_critical), converts thresholds to dBm for optical
sensors
3. Smart grouping: Organizes sensors by SFPs, PSUs, Platform (chipsets), Power
Connectors, System for better UX
4. Description cleanup: Removes redundant "sensor" text, simplifies PSU naming,
cleans whitespace
- lib/towerops/snmp/profiles/dynamic.ex (integration)
Added apply_vendor_post_processing/3 to call Arista enhancements after sensor
discovery. Applies to both arista_eos and arista-mos profiles.
- test/towerops/snmp/profiles/vendors/arista_test.exs (comprehensive tests)
Added 23 new tests covering:
- DOM conversion (6 tests): Rx/Tx power, Xcvr power, non-DOM sensors, edge cases
- Threshold discovery (4 tests): Basic thresholds, dBm conversion, no thresholds,
SNMP failures
- Smart grouping (6 tests): SFPs, Xcvr, PSUs, power connectors, platform, system
- Description cleanup (5 tests): Trailing sensor, duplicate Sensor strings, PSU
naming, hotspot, whitespace
- End-to-end post-processing (1 test): All enhancements in correct order
All tests passing (30/30 in arista_test.exs, 6367/6367 total).
Result: Arista EOS support upgraded from 85% to 100% LibreNMS parity. All critical
gaps closed: optical power now displayed correctly in dBm, alerting enabled via
thresholds, sensors organized for better UX, descriptions cleaned up.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
normalize OIDs by stripping leading dots when grouping for deduplication.
gosnmp returns OIDs with leading dots but YAML profiles and vendor modules
define them without, causing the same sensor to appear twice (e.g. MikroTik
DHCP lease count). also removed redundant DHCP Leases entry from routeros
wireless_oid_defs since the YAML profile already covers it.
- use 64-bit HC counters (ifHCInOctets/ifHCOutOctets) instead of 32-bit
counters that wrap every ~34s on gigabit links
- fix negative integer parsing in replay adapter so sensor scale values
like -3 produce correct divisors (1000) instead of falling through to 1
- replace walking entire Cisco enterprise tree with targeted subtrees to
avoid timeouts; add Juniper, HP/H3C, Fortinet, Cambium vendor OIDs
- add missing discovery OIDs: ENTITY-STATE-MIB, HOST-RESOURCES-MIB
tables, and UCD-SNMP-MIB Linux CPU stats fallback
Some MikroTik devices return the version OID as a list of integers
[7, 20, 6] instead of a binary string "7.20.6". Added pattern match
to detect_version/1 to handle this case and convert to dotted string.
Fixes warning: "Failed to detect version: {:ok, [7, 20, 6]}"
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
strip leading dots from OID keys returned by agent so they match
sensor OIDs stored without leading dots. fixes ubiquiti airos
wireless sensors (clients, ccq, frequency, etc) not being polled.
also fix flaky sensor_change_detector tests using global pubsub topic.
extract sensor change detection from DevicePollerWorker into shared
SensorChangeDetector module. add sensor_value_changed event for non-%
sensors when value differs from last_value. agent channel now detects
changes and updates last_value, matching phoenix poller behavior.
YAML parses bare numbers like `index: 0` as integers. coerce
index_template to string before calling String.replace in both
build_table_sensor and build_state_sensor.
newly discovered devices sat idle with no metric data until the
next scheduling cycle. now the channel sends a poll job (and ping
job if monitoring enabled) right after discovery succeeds, using
send(self(), ...) so discovery DB writes commit first.
also reduces k8s replicas from 3 to 2.
gosnmp returns OIDs with leading dots (e.g., ".1.3.6.1.2.1.1.1.0")
but the Replay adapter did exact-match lookups without the dot.
Walks already normalized both sides, so interfaces worked. GETs
did not, breaking all system OID resolution (sysDescr, sysName, etc).
- increase timing threshold in session activity test (flaky at 50ms)
- show org name instead of agent ID in admin agents list
- fix agent live tests wiping auth session with init_test_session
- add resilient error handling in FourOhFourTracker for missing redis
the version regex only allowed [a-zA-Z0-9.] in semver suffixes,
which rejected hyphens from git-describe output (e.g. 0.1.0-5-gabcdef0).
this caused every heartbeat to fail validation, preventing last_seen_at
from updating, which led to agent status showing "warning" and eventual
channel disconnect after 360s.
also accept bare git SHAs and short identifiers like "dev".
Replace phx-click JS.navigate with real <.link navigate> elements
on table rows so users can Ctrl+Click to open in new tabs, use
right-click context menus, and navigate via keyboard. Adds row_link
attr to <.table> core component and applies it to all agent tables.
Converts device list custom table from phx-click to inner links.
Broadcast org-scoped PubSub events from Devices context mutations
(create, update, delete, status change). Convert DeviceLive.Index
from assigns to streams with flat row items. Subscribe on mount so
the device list updates automatically across browser tabs.
POST /api/v1/webhooks/agent-release triggers all connected agents
to self-update via their existing WebSocket channels. Authenticated
with a shared secret (AGENT_WEBHOOK_SECRET env var).
- Add ReleaseChecker.invalidate_cache/0 for fresh GitHub fetch
- Add Agents.list_updatable_agents/0 (enabled + seen < 10min)
- Add Agents.broadcast_mass_update/0 orchestration function
- Add WebhookAuth plug with timing-safe secret comparison
- Add AgentReleaseWebhookController with :webhook pipeline
- Configure AGENT_WEBHOOK_SECRET in dev/test/runtime configs
Add server-initiated restart and self-update capabilities for remote
agents. Restart broadcasts on the lifecycle PubSub topic causing the
agent to exit cleanly for Docker to restart. Self-update fetches the
latest release from GitHub, finds the matching binary for the agent's
architecture, and sends the download URL with SHA256 checksum.
- Add restart_agent/1 and update_agent/3 to Agents context
- Handle :restart_requested and :update_requested in AgentChannel
- Add superuser-only Restart and Update buttons on agent show page
- Add ReleaseChecker module for GitHub Releases API with 5-min cache
- Add arch field to AgentHeartbeat protobuf for architecture detection
- Store arch in agent metadata from heartbeat
The admin agents page links to /agents/:id but the show and edit
pages were scoped to the current user's organization, causing 404s
for agents belonging to other orgs.
Broadcast agent lifecycle events (created/deleted) on a separate
"admin:agents" topic so the admin page updates dynamically when
agents are added or removed from any organization.
Superuser-only page showing all agents across all organizations
and cloud pollers in a single table with live-ticking timestamps
and real-time status updates via PubSub.
