LLDP/CDP evidence (0.95 confidence) from snmp_neighbors.
MAC evidence (0.7) cross-references MAC tables with device
interface MACs. ARP evidence (0.6) matches by IP and MAC.
Self-links excluded. Unmatched entries tracked as discovered.
The plug was attempting to register a before_send callback after
blocking a banned IP and sending a 403 response. This caused
Plug.Conn.AlreadySentError in production.
Fixed by checking conn.halted before attempting to register the
404 tracking callback. If the request was already blocked and sent,
we skip the callback registration.
Also updated the test to verify proper behavior without the
workaround for AlreadySentError.
Ecto schemas for persistent topology model. DeviceLink tracks
connections between devices with confidence scores. DeviceLinkEvidence
records why each link is believed to exist. Device schema gets
device_role and device_role_source fields.
Filter port_died and write_failed/epipe errors from Honeybadger reports,
email notifications, and logs. Reorder supervision tree so Oban drains
before its dependencies shut down, and add 40s shutdown grace period.
Phase 3 audit found screenos.yaml detection profile was not registered
in the Juniper vendor module. Added to profile_names list.
ScreenOS is Juniper's legacy firewall operating system (pre-Junos).
Coverage: 5 Juniper profiles now registered
- juniper-mss
- juniperex2500os
- junos
- junose
- screenos (newly added)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Always broadcast from update_device_status/2 so the device index
LiveView refreshes even when status hasn't changed. Add a 15-second
tick timer to keep "Last Checked" timestamps fresh between polls.
Scope alert PubSub topics to organization for defense-in-depth.
Add a Honeybadger NoticeFilter that sends an email with the full
error class, message, stacktrace, and server info to graham@mcintire.me
whenever an exception is reported. The notice passes through unchanged
so Honeybadger still receives it normally.
Replace individual Repo.insert() with Repo.insert_all() for sensor readings,
interface stats, processor readings, and storage readings in agent channel and
device poller worker. Add partial/composite database indexes for common query
patterns. Dashboard now uses GROUP BY aggregation instead of loading all devices.
DeviceLive.Show only reloads data for the active tab on PubSub events.
DeviceLive.Index debounces status changes and skips quota queries on updates.
Agent polling preloads filtered to monitored sensors/interfaces only.
Changed expect() to stub() in all discover_wireless_sensors/1 tests
to handle multiple SNMP get calls:
- Static sensor fetch calls Client.get() for CPU
- ePMP mode detection calls Client.get() for wirelessInterfaceMode
- Each sensor value fetch calls Client.get() (RSSI, SNR, Frequency, Clients)
Using stub() allows unlimited calls vs expect() which enforces
exactly one call, fixing Mox.UnexpectedCallError in all tests.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Update AgentLive.Index to properly handle browser back/forward buttons
by syncing the setup token modal to URL parameters.
Changes:
- Update handle_params/3 to read modal param and set modal state
- Update show_setup handler to use push_patch with ?modal=setup
- Update close_token_modal handler to use push_patch (remove modal param)
- Update handle_agent_creation_success to use push_patch after creating agent
Behavior:
- Browser back button now closes the setup modal without navigating away
- URL reflects current modal state (?modal=setup)
- Bookmarkable URLs restore exact modal state (can share agent setup URL)
- After creating agent, modal opens via URL state
Follows idiomatic Phoenix LiveView pattern using push_patch/2 and
handle_params/3 as documented in CLAUDE.md.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Implemented all critical and nice-to-have Arista sensor improvements identified
in LibreNMS parity audit. Upgrades Arista EOS support from 85% to 100% parity.
Files Changed:
- lib/towerops/snmp/profiles/vendors/arista.ex (enhanced)
Added 4 major enhancements:
1. DOM power conversion (watts→dBm): Detects optical power sensors via regex,
converts using formula dBm = 10*log10(watts*1000), preserves original value
in metadata
2. Arista threshold discovery: Walks ARISTA-ENTITY-SENSOR-MIB threshold table
(OID 1.3.6.1.4.1.30065.3.3.1.1.1), applies 4 threshold types (low_critical,
low_warn, high_warn, high_critical), converts thresholds to dBm for optical
sensors
3. Smart grouping: Organizes sensors by SFPs, PSUs, Platform (chipsets), Power
Connectors, System for better UX
4. Description cleanup: Removes redundant "sensor" text, simplifies PSU naming,
cleans whitespace
- lib/towerops/snmp/profiles/dynamic.ex (integration)
Added apply_vendor_post_processing/3 to call Arista enhancements after sensor
discovery. Applies to both arista_eos and arista-mos profiles.
- test/towerops/snmp/profiles/vendors/arista_test.exs (comprehensive tests)
Added 23 new tests covering:
- DOM conversion (6 tests): Rx/Tx power, Xcvr power, non-DOM sensors, edge cases
- Threshold discovery (4 tests): Basic thresholds, dBm conversion, no thresholds,
SNMP failures
- Smart grouping (6 tests): SFPs, Xcvr, PSUs, power connectors, platform, system
- Description cleanup (5 tests): Trailing sensor, duplicate Sensor strings, PSU
naming, hotspot, whitespace
- End-to-end post-processing (1 test): All enhancements in correct order
All tests passing (30/30 in arista_test.exs, 6367/6367 total).
Result: Arista EOS support upgraded from 85% to 100% LibreNMS parity. All critical
gaps closed: optical power now displayed correctly in dBm, alerting enabled via
thresholds, sensors organized for better UX, descriptions cleaned up.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
normalize OIDs by stripping leading dots when grouping for deduplication.
gosnmp returns OIDs with leading dots but YAML profiles and vendor modules
define them without, causing the same sensor to appear twice (e.g. MikroTik
DHCP lease count). also removed redundant DHCP Leases entry from routeros
wireless_oid_defs since the YAML profile already covers it.
- use 64-bit HC counters (ifHCInOctets/ifHCOutOctets) instead of 32-bit
counters that wrap every ~34s on gigabit links
- fix negative integer parsing in replay adapter so sensor scale values
like -3 produce correct divisors (1000) instead of falling through to 1
- replace walking entire Cisco enterprise tree with targeted subtrees to
avoid timeouts; add Juniper, HP/H3C, Fortinet, Cambium vendor OIDs
- add missing discovery OIDs: ENTITY-STATE-MIB, HOST-RESOURCES-MIB
tables, and UCD-SNMP-MIB Linux CPU stats fallback
Some MikroTik devices return the version OID as a list of integers
[7, 20, 6] instead of a binary string "7.20.6". Added pattern match
to detect_version/1 to handle this case and convert to dotted string.
Fixes warning: "Failed to detect version: {:ok, [7, 20, 6]}"
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
strip leading dots from OID keys returned by agent so they match
sensor OIDs stored without leading dots. fixes ubiquiti airos
wireless sensors (clients, ccq, frequency, etc) not being polled.
also fix flaky sensor_change_detector tests using global pubsub topic.
extract sensor change detection from DevicePollerWorker into shared
SensorChangeDetector module. add sensor_value_changed event for non-%
sensors when value differs from last_value. agent channel now detects
changes and updates last_value, matching phoenix poller behavior.
YAML parses bare numbers like `index: 0` as integers. coerce
index_template to string before calling String.replace in both
build_table_sensor and build_state_sensor.
newly discovered devices sat idle with no metric data until the
next scheduling cycle. now the channel sends a poll job (and ping
job if monitoring enabled) right after discovery succeeds, using
send(self(), ...) so discovery DB writes commit first.
also reduces k8s replicas from 3 to 2.
gosnmp returns OIDs with leading dots (e.g., ".1.3.6.1.2.1.1.1.0")
but the Replay adapter did exact-match lookups without the dot.
Walks already normalized both sides, so interfaces worked. GETs
did not, breaking all system OID resolution (sysDescr, sysName, etc).
- increase timing threshold in session activity test (flaky at 50ms)
- show org name instead of agent ID in admin agents list
- fix agent live tests wiping auth session with init_test_session
- add resilient error handling in FourOhFourTracker for missing redis
the version regex only allowed [a-zA-Z0-9.] in semver suffixes,
which rejected hyphens from git-describe output (e.g. 0.1.0-5-gabcdef0).
this caused every heartbeat to fail validation, preventing last_seen_at
from updating, which led to agent status showing "warning" and eventual
channel disconnect after 360s.
also accept bare git SHAs and short identifiers like "dev".
Replace phx-click JS.navigate with real <.link navigate> elements
on table rows so users can Ctrl+Click to open in new tabs, use
right-click context menus, and navigate via keyboard. Adds row_link
attr to <.table> core component and applies it to all agent tables.
Converts device list custom table from phx-click to inner links.
Broadcast org-scoped PubSub events from Devices context mutations
(create, update, delete, status change). Convert DeviceLive.Index
from assigns to streams with flat row items. Subscribe on mount so
the device list updates automatically across browser tabs.
POST /api/v1/webhooks/agent-release triggers all connected agents
to self-update via their existing WebSocket channels. Authenticated
with a shared secret (AGENT_WEBHOOK_SECRET env var).
- Add ReleaseChecker.invalidate_cache/0 for fresh GitHub fetch
- Add Agents.list_updatable_agents/0 (enabled + seen < 10min)
- Add Agents.broadcast_mass_update/0 orchestration function
- Add WebhookAuth plug with timing-safe secret comparison
- Add AgentReleaseWebhookController with :webhook pipeline
- Configure AGENT_WEBHOOK_SECRET in dev/test/runtime configs
Add server-initiated restart and self-update capabilities for remote
agents. Restart broadcasts on the lifecycle PubSub topic causing the
agent to exit cleanly for Docker to restart. Self-update fetches the
latest release from GitHub, finds the matching binary for the agent's
architecture, and sends the download URL with SHA256 checksum.
- Add restart_agent/1 and update_agent/3 to Agents context
- Handle :restart_requested and :update_requested in AgentChannel
- Add superuser-only Restart and Update buttons on agent show page
- Add ReleaseChecker module for GitHub Releases API with 5-min cache
- Add arch field to AgentHeartbeat protobuf for architecture detection
- Store arch in agent metadata from heartbeat
The admin agents page links to /agents/:id but the show and edit
pages were scoped to the current user's organization, causing 404s
for agents belonging to other orgs.
Broadcast agent lifecycle events (created/deleted) on a separate
"admin:agents" topic so the admin page updates dynamically when
agents are added or removed from any organization.
Superuser-only page showing all agents across all organizations
and cloud pollers in a single table with live-ticking timestamps
and real-time status updates via PubSub.
- Add Towerops.TaskSupervisor to supervision tree, replace bare spawn/1
in agent channel with supervised tasks for polling data processing
- Change DiscoveryWorker from max_attempts: 1 to 3 with retryable error
classification (transient errors retry, permanent errors discard)
- Name all 9 parallel polling tasks and log timeouts/crashes instead of
silently dropping results via Stream.run()
- Add tests for TaskSupervisor presence and error classification
This commit addresses Phase 1 (CRITICAL - Data Loss Prevention) issues
from the comprehensive bugs and inconsistencies analysis.
CRITICAL FIXES:
1. Protobuf Schema Inconsistencies
- Added missing job_id field (field 5) to SnmpResult message
- Fixed AgentError message structure (job_id, message, timestamp)
- Removed obsolete transport field from SnmpDevice struct
- Regenerated protobuf code with protoc-gen-elixir
- Prevents crashes when agent sends SNMP results or errors
2. Interface/Sensor Discovery Timeout Data Loss
- Changed timeout behavior to return error instead of empty list
- Prevents deletion of ALL existing interfaces/sensors on slow networks
- Discovery fails gracefully instead of destroying historical data
- Addresses commit 7a57f7c sensor discovery pipeline data loss issue
3. Polling Offset Schedule Mismatch
- Fixed schedule_next_poll() to use offset only (not interval + offset)
- Maintains consistent polling intervals across all executions
- Prevents load bunching that offset was designed to prevent
- Example: 300s interval with offset=94s now consistently polls every 5m
4. Always Reschedule Race Condition
- Added should_continue_polling?() and should_continue_monitoring?()
- Only reschedule if device is still enabled and should be polled by Phoenix
- Prevents zombie jobs from continuing to poll disabled/deleted/reassigned devices
- Stops race condition where stop_polling() is bypassed by in-flight jobs
Files Changed:
- priv/proto/agent.proto
- lib/towerops/proto/agent.pb.ex (regenerated)
- lib/towerops_web/channels/agent_channel.ex
- lib/towerops/snmp/discovery.ex
- lib/towerops/workers/device_poller_worker.ex
- lib/towerops/workers/device_monitor_worker.ex
- test/towerops/workers/device_poller_worker_test.exs
Test Results: All passing
- device_poller_worker_test.exs: 8 tests, 0 failures
- device_monitor_worker_test.exs: 3 tests, 0 failures
- discovery_test.exs: 13 tests, 0 failures
Remaining Phases:
- Phase 2 (HIGH): 8 data integrity issues
- Phase 3 (MEDIUM): 10 reliability issues
- Phase 4 (LOW): 3 cleanup issues
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
