Completes Phase 5 of the unified checks system by enabling automatic check
creation during SNMP discovery and providing a backfill tool for existing devices.
Discovery Integration:
- create_checks_from_discovery/2: Creates checks for all discovered entities
(sensors, interfaces, processors, storage) after SNMP discovery completes
- Links each check to its source entity via source_id for data lookup
- Returns detailed results with counts per type and error tracking
- Logs creation results with total counts and any failures
Helper Functions:
- create_sensor_check/2: Creates snmp_sensor checks with config
- create_interface_check/2: Creates snmp_interface checks
- create_processor_check/2: Creates snmp_processor checks
- create_storage_check/2: Creates snmp_storage checks
All checks configured with:
- 60-second poll intervals (default)
- Auto-discovery source type
- Enabled by default
- Type-specific configuration in JSONB config field
Backfill Tool:
- mix backfill.checks: Creates checks for existing SNMP-enabled devices
- Supports --dry-run mode for preview
- Shows counts per device (sensors, interfaces, processors, storage)
- Reports success/failure counts and detailed errors
- Safely handles devices without discovery data
- Reduced nesting depth using cond and helper functions for credo compliance
This enables the unified monitoring system to work with both new discoveries
and existing devices, ensuring all SNMP data can be monitored through the
unified checks interface.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit implements the unified checks architecture that consolidates
SNMP monitoring with HTTP/TCP/DNS checks under a single "check" abstraction,
enabling consistent graphing, alerting, and management across all check types.
Database Changes:
- Add source_type and source_id to checks table for tracking auto-discovered
vs manually created checks
- Add value field to check_results for storing numeric sensor readings
- Maintain backward compatibility with existing check_results data
New SNMP Executors:
- SnmpSensorExecutor: Poll sensor OIDs and return formatted values with
status determination (OK/WARNING/CRITICAL based on limits)
- SnmpInterfaceExecutor: Poll interface stats (bandwidth, packets, errors)
- SnmpProcessorExecutor: Poll CPU/processor usage
- SnmpStorageExecutor: Poll disk/memory usage with percentage calculations
Check Execution Worker:
- CheckExecutorWorker: Unified Oban worker that dispatches to appropriate
executor based on check_type (snmp_sensor, snmp_interface, http, tcp, etc.)
- Self-schedules next execution with distributed polling offsets
- Records results in check_results TimescaleDB hypertable
- Updates check state (OK/WARNING/CRITICAL/UNKNOWN)
Discovery Integration:
- Auto-creates checks during SNMP discovery for sensors, interfaces,
processors, and storage
- Links checks to source entities via source_id for data lookup
- Enables/disables checks based on discovery results
UI Enhancements:
- Checks tab on device detail page with grouped display
- FormComponent for adding manual HTTP/TCP/DNS checks
- Empty state with "Run Discovery" prompt
- Check status badges and last checked times
Graphing:
- Update GraphLive to accept check_id parameter
- Query check_results table for time-series data
- Support all check types (SNMP, HTTP response times, etc.)
Testing:
- Comprehensive test suite for SnmpSensorExecutor (5 tests)
- Test suite for CheckExecutorWorker (7 tests)
- Test coverage for discovery check creation (6 tests)
- Remove deprecated monitoring_test.exs testing old API
Bug Fixes:
- Fix SNMP executors reading credentials from correct Device schema fields
(device.snmp_version instead of device.snmp_device.version)
- Update agent channel test to query MonitoringCheck table directly
Code Quality:
- Extract add_snmp_credentials helper to reduce cyclomatic complexity
- Use map-based lookups for sensor formatting and check type grouping
- Apply pattern matching in dispatcher to reduce complexity
- All credo checks passing with no issues
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- create MonitoringCheck schema for the monitoring_checks table, fixing
silent data loss where ping results were dropped by the wrong schema
- update agent channel and device monitor worker to use new schema
- fix Stats queries to use MonitoringCheck instead of Check schema
- add list_online_cloud_pollers and list_cloud_polled_devices queries
- add CloudLatencyProbeWorker to dispatch cross-agent latency probes
every 8 hours via PubSub to all online cloud pollers
- scope AgentLatencyEvaluator to cloud pollers only with cloud_only
filter, lower min_checks (5), wider time window (48h), and
device-level assignments to avoid changing site/org defaults
- update cron schedule: probes at 0/8/16h, evaluation at 0:30/8:30/16:30
- refactor monitoring.ex and http_executor.ex to fix credo complexity
- Add 'Checks' tab link to device navigation
- Display empty state when no checks configured
- Add tests for tab visibility and empty state
- Fix unused variable warnings in monitoring context
- Remove unused CheckResult alias and module attributes
- Add stub for legacy get_latency_data for backward compatibility
Tests: 2 new tests added, all passing
Following TDD: RED-GREEN cycle verified for each feature
Enhanced AgentChannel logging to track device IDs across job list updates.
Added tests verifying server correctly excludes deleted devices from job lists.
Root cause: Bug is in Go agent (appends jobs without clearing), not Phoenix server.
Implements Icinga2-style generic check system for comprehensive monitoring.
Phoenix Backend:
- Generic checks table with JSONB config for all check types
- TimescaleDB check_results hypertable for time-series data
- Check executors (HTTP, TCP, DNS)
- CheckWorker with self-scheduling and state transitions
- Extended alerts system for check-based alerting
Check Types: HTTP/HTTPS, TCP, DNS
Architecture: Icinga2 Checkable pattern, soft/hard states, flapping detection
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Build a rich network topology from SNMP polling data using evidence-based
confidence scoring. LLDP/CDP neighbors, MAC address tables, and ARP data
are combined to infer device links with weighted confidence merging.
- Add DeviceLink and DeviceLinkEvidence schemas for persistent topology
- Implement evidence collectors: LLDP (0.95), CDP (0.95), MAC (0.7), ARP (0.6)
- Add device role inference from sysObjectID/sysDescr patterns
- Hook topology inference into DevicePollerWorker pipeline
- Add stale link cleanup (24h mark stale, 72h delete) via NeighborCleanupWorker
- Update NetworkMapLive with "Added" vs "All Devices" tabs
- Add connected devices section to device detail page
- Add device role selector to device edit form
- Update Cytoscape.js with role-based node shapes/colors and confidence edges
push_patch without replace: true created a history entry loop:
/devices/:id → /devices/:id?tab=overview, so hitting back would
immediately redirect forward again, trapping the user.
Infers device roles from LLDP/CDP capabilities, interface count,
vendor/manufacturer, and system description. Only overwrites
inferred roles, never manual overrides. Wired into process_device
pipeline.
Orchestrates evidence collection from LLDP, CDP, MAC, and ARP
data. Groups evidence by remote device, resolves to managed
devices, upserts links with merged confidence. Broadcasts
topology changes via PubSub. Skips self-links.
LLDP/CDP evidence (0.95 confidence) from snmp_neighbors.
MAC evidence (0.7) cross-references MAC tables with device
interface MACs. ARP evidence (0.6) matches by IP and MAC.
Self-links excluded. Unmatched entries tracked as discovered.
The plug was attempting to register a before_send callback after
blocking a banned IP and sending a 403 response. This caused
Plug.Conn.AlreadySentError in production.
Fixed by checking conn.halted before attempting to register the
404 tracking callback. If the request was already blocked and sent,
we skip the callback registration.
Also updated the test to verify proper behavior without the
workaround for AlreadySentError.
Ecto schemas for persistent topology model. DeviceLink tracks
connections between devices with confidence scores. DeviceLinkEvidence
records why each link is believed to exist. Device schema gets
device_role and device_role_source fields.
Filter port_died and write_failed/epipe errors from Honeybadger reports,
email notifications, and logs. Reorder supervision tree so Oban drains
before its dependencies shut down, and add 40s shutdown grace period.
Phase 3 audit found screenos.yaml detection profile was not registered
in the Juniper vendor module. Added to profile_names list.
ScreenOS is Juniper's legacy firewall operating system (pre-Junos).
Coverage: 5 Juniper profiles now registered
- juniper-mss
- juniperex2500os
- junos
- junose
- screenos (newly added)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Always broadcast from update_device_status/2 so the device index
LiveView refreshes even when status hasn't changed. Add a 15-second
tick timer to keep "Last Checked" timestamps fresh between polls.
Scope alert PubSub topics to organization for defense-in-depth.
Add a Honeybadger NoticeFilter that sends an email with the full
error class, message, stacktrace, and server info to graham@mcintire.me
whenever an exception is reported. The notice passes through unchanged
so Honeybadger still receives it normally.
Replace individual Repo.insert() with Repo.insert_all() for sensor readings,
interface stats, processor readings, and storage readings in agent channel and
device poller worker. Add partial/composite database indexes for common query
patterns. Dashboard now uses GROUP BY aggregation instead of loading all devices.
DeviceLive.Show only reloads data for the active tab on PubSub events.
DeviceLive.Index debounces status changes and skips quota queries on updates.
Agent polling preloads filtered to monitored sensors/interfaces only.
Update AgentLive.Index to properly handle browser back/forward buttons
by syncing the setup token modal to URL parameters.
Changes:
- Update handle_params/3 to read modal param and set modal state
- Update show_setup handler to use push_patch with ?modal=setup
- Update close_token_modal handler to use push_patch (remove modal param)
- Update handle_agent_creation_success to use push_patch after creating agent
Behavior:
- Browser back button now closes the setup modal without navigating away
- URL reflects current modal state (?modal=setup)
- Bookmarkable URLs restore exact modal state (can share agent setup URL)
- After creating agent, modal opens via URL state
Follows idiomatic Phoenix LiveView pattern using push_patch/2 and
handle_params/3 as documented in CLAUDE.md.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Update UserSettingsLive to properly handle browser back/forward buttons
by syncing all modal state to URL parameters.
Changes:
- Update handle_params/3 to read modal param and set modal states
- Add build_settings_path/2 helper to construct URLs with preserved params
- Update all modal show/close handlers to use push_patch instead of assign:
- add_token modal
- token_created modal (shown after creating token)
- revoke_all modal
- add_device modal (TOTP)
- device_qr modal (shown after creating device)
- recovery_codes modal
Behavior:
- Browser back button now closes modals without navigating away
- URL reflects current modal state (?modal=add_token, etc.)
- Tab and page params preserved when opening/closing modals
- Bookmarkable URLs restore exact modal state
Follows idiomatic Phoenix LiveView pattern using push_patch/2 and
handle_params/3 as documented in CLAUDE.md.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Implemented all critical and nice-to-have Arista sensor improvements identified
in LibreNMS parity audit. Upgrades Arista EOS support from 85% to 100% parity.
Files Changed:
- lib/towerops/snmp/profiles/vendors/arista.ex (enhanced)
Added 4 major enhancements:
1. DOM power conversion (watts→dBm): Detects optical power sensors via regex,
converts using formula dBm = 10*log10(watts*1000), preserves original value
in metadata
2. Arista threshold discovery: Walks ARISTA-ENTITY-SENSOR-MIB threshold table
(OID 1.3.6.1.4.1.30065.3.3.1.1.1), applies 4 threshold types (low_critical,
low_warn, high_warn, high_critical), converts thresholds to dBm for optical
sensors
3. Smart grouping: Organizes sensors by SFPs, PSUs, Platform (chipsets), Power
Connectors, System for better UX
4. Description cleanup: Removes redundant "sensor" text, simplifies PSU naming,
cleans whitespace
- lib/towerops/snmp/profiles/dynamic.ex (integration)
Added apply_vendor_post_processing/3 to call Arista enhancements after sensor
discovery. Applies to both arista_eos and arista-mos profiles.
- test/towerops/snmp/profiles/vendors/arista_test.exs (comprehensive tests)
Added 23 new tests covering:
- DOM conversion (6 tests): Rx/Tx power, Xcvr power, non-DOM sensors, edge cases
- Threshold discovery (4 tests): Basic thresholds, dBm conversion, no thresholds,
SNMP failures
- Smart grouping (6 tests): SFPs, Xcvr, PSUs, power connectors, platform, system
- Description cleanup (5 tests): Trailing sensor, duplicate Sensor strings, PSU
naming, hotspot, whitespace
- End-to-end post-processing (1 test): All enhancements in correct order
All tests passing (30/30 in arista_test.exs, 6367/6367 total).
Result: Arista EOS support upgraded from 85% to 100% LibreNMS parity. All critical
gaps closed: optical power now displayed correctly in dBm, alerting enabled via
thresholds, sensors organized for better UX, descriptions cleaned up.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Some devices (e.g., AirOS 8.7.14) return the literal string "null"
for unavailable metrics like UCD CPU stats. Added explicit handling
in parse_integer/1 and parse_float/1 to return nil for "null" strings,
preventing ArgumentError when attempting binary_to_integer conversion.
normalize OIDs by stripping leading dots when grouping for deduplication.
gosnmp returns OIDs with leading dots but YAML profiles and vendor modules
define them without, causing the same sensor to appear twice (e.g. MikroTik
DHCP lease count). also removed redundant DHCP Leases entry from routeros
wireless_oid_defs since the YAML profile already covers it.
- use 64-bit HC counters (ifHCInOctets/ifHCOutOctets) instead of 32-bit
counters that wrap every ~34s on gigabit links
- fix negative integer parsing in replay adapter so sensor scale values
like -3 produce correct divisors (1000) instead of falling through to 1
- replace walking entire Cisco enterprise tree with targeted subtrees to
avoid timeouts; add Juniper, HP/H3C, Fortinet, Cambium vendor OIDs
- add missing discovery OIDs: ENTITY-STATE-MIB, HOST-RESOURCES-MIB
tables, and UCD-SNMP-MIB Linux CPU stats fallback
Some MikroTik devices return the version OID as a list of integers
[7, 20, 6] instead of a binary string "7.20.6". Added pattern match
to detect_version/1 to handle this case and convert to dotted string.
Fixes warning: "Failed to detect version: {:ok, [7, 20, 6]}"
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
green for online, yellow for unknown, red for offline. previously
nodes were colored by device type which didn't convey health state.
also remove agent last_seen_at display from device detail page since
the offline badge already communicates agent health.
strip leading dots from OID keys returned by agent so they match
sensor OIDs stored without leading dots. fixes ubiquiti airos
wireless sensors (clients, ccq, frequency, etc) not being polled.
also fix flaky sensor_change_detector tests using global pubsub topic.
extract sensor change detection from DevicePollerWorker into shared
SensorChangeDetector module. add sensor_value_changed event for non-%
sensors when value differs from last_value. agent channel now detects
changes and updates last_value, matching phoenix poller behavior.
live poll mode used get_device_assignment which only checks direct
device-level assignments. devices with agent assigned at site or
org level fell through to direct Phoenix SNMP. replaced with
get_effective_agent_token which walks the full cascade.
division by sensor_divisor produced full float precision values
like 34.09999999. apply Float.round/2 in both the phoenix poller
and agent channel paths.
store_monitoring_check saved ping results from agents but never
updated device status. DeviceMonitorWorker skips agent-assigned
devices, so nothing ever moved them from :unknown to :up/:down.
added update_device_status_from_check to derive status from check
result, update the device, create alerts on transitions, and
broadcast via PubSub — matching DeviceMonitorWorker behavior.
YAML parses bare numbers like `index: 0` as integers. coerce
index_template to string before calling String.replace in both
build_table_sensor and build_state_sensor.
newly discovered devices sat idle with no metric data until the
next scheduling cycle. now the channel sends a poll job (and ping
job if monitoring enabled) right after discovery succeeds, using
send(self(), ...) so discovery DB writes commit first.
also reduces k8s replicas from 3 to 2.
gosnmp returns OIDs with leading dots (e.g., ".1.3.6.1.2.1.1.1.0")
but the Replay adapter did exact-match lookups without the dot.
Walks already normalized both sides, so interfaces worked. GETs
did not, breaking all system OID resolution (sysDescr, sysName, etc).
- increase timing threshold in session activity test (flaky at 50ms)
- show org name instead of agent ID in admin agents list
- fix agent live tests wiping auth session with init_test_session
- add resilient error handling in FourOhFourTracker for missing redis
