Commit graph

970 commits

Author SHA1 Message Date
17c5b681a3
back up to 2 replicas 2026-02-11 16:04:47 -06:00
47fcbeb483
fix: agent discovery and polling accuracy improvements
- use 64-bit HC counters (ifHCInOctets/ifHCOutOctets) instead of 32-bit
  counters that wrap every ~34s on gigabit links
- fix negative integer parsing in replay adapter so sensor scale values
  like -3 produce correct divisors (1000) instead of falling through to 1
- replace walking entire Cisco enterprise tree with targeted subtrees to
  avoid timeouts; add Juniper, HP/H3C, Fortinet, Cambium vendor OIDs
- add missing discovery OIDs: ENTITY-STATE-MIB, HOST-RESOURCES-MIB
  tables, and UCD-SNMP-MIB Linux CPU stats fallback
2026-02-11 15:58:31 -06:00
863f92a94a
fix: handle RouterOS version returned as list of integers
Some MikroTik devices return the version OID as a list of integers
[7, 20, 6] instead of a binary string "7.20.6". Added pattern match
to detect_version/1 to handle this case and convert to dotted string.

Fixes warning: "Failed to detect version: {:ok, [7, 20, 6]}"

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-11 15:49:57 -06:00
26bb3d5ca1
color network map nodes by device status
green for online, yellow for unknown, red for offline. previously
nodes were colored by device type which didn't convey health state.
also remove agent last_seen_at display from device detail page since
the offline badge already communicates agent health.
2026-02-11 15:41:50 -06:00
92b4f0081a
reduce to one replica in prod for now 2026-02-11 15:38:07 -06:00
475dea639a
remove agent last_seen_at display from device detail page
the offline badge already communicates agent health status,
so the relative timestamp is redundant.
2026-02-11 15:35:59 -06:00
edb330e4de
normalize OIDs to fix agent-polled sensor matching
strip leading dots from OID keys returned by agent so they match
sensor OIDs stored without leading dots. fixes ubiquiti airos
wireless sensors (clients, ccq, frequency, etc) not being polled.
also fix flaky sensor_change_detector tests using global pubsub topic.
2026-02-11 15:33:08 -06:00
abbe117f5e
add sensor value change events for all sensor types
extract sensor change detection from DevicePollerWorker into shared
SensorChangeDetector module. add sensor_value_changed event for non-%
sensors when value differs from last_value. agent channel now detects
changes and updates last_value, matching phoenix poller behavior.
2026-02-11 15:06:55 -06:00
45357e334a
fix live polling to use effective agent from site/org cascade
live poll mode used get_device_assignment which only checks direct
device-level assignments. devices with agent assigned at site or
org level fell through to direct Phoenix SNMP. replaced with
get_effective_agent_token which walks the full cascade.
2026-02-11 14:12:42 -06:00
6d597f7e76
round sensor readings to 1 decimal place
division by sensor_divisor produced full float precision values
like 34.09999999. apply Float.round/2 in both the phoenix poller
and agent channel paths.
2026-02-11 14:09:22 -06:00
2875866c07
fix agent-polled devices stuck in unknown status
store_monitoring_check saved ping results from agents but never
updated device status. DeviceMonitorWorker skips agent-assigned
devices, so nothing ever moved them from :unknown to :up/:down.

added update_device_status_from_check to derive status from check
result, update the device, create alerts on transitions, and
broadcast via PubSub — matching DeviceMonitorWorker behavior.
2026-02-11 14:03:04 -06:00
cb3f2edf50
polling fix 2026-02-11 13:54:23 -06:00
4e290b1938
fix String.replace crash when YAML index is numeric
YAML parses bare numbers like `index: 0` as integers. coerce
index_template to string before calling String.replace in both
build_table_sensor and build_state_sensor.
2026-02-11 13:27:46 -06:00
9e36d5030a
poll device immediately after discovery completes
newly discovered devices sat idle with no metric data until the
next scheduling cycle. now the channel sends a poll job (and ping
job if monitoring enabled) right after discovery succeeds, using
send(self(), ...) so discovery DB writes commit first.

also reduces k8s replicas from 3 to 2.
2026-02-11 13:24:55 -06:00
10d1157ec3
fix agent discovery system OIDs returning no_such_name
gosnmp returns OIDs with leading dots (e.g., ".1.3.6.1.2.1.1.1.0")
but the Replay adapter did exact-match lookups without the dot.
Walks already normalized both sides, so interfaces worked. GETs
did not, breaking all system OID resolution (sysDescr, sysName, etc).
2026-02-11 12:46:44 -06:00
d36fcbe628
fix raw discovery data timestamp to use user's timezone 2026-02-11 12:41:31 -06:00
1a31fd00b6
fix nil sysDescr crashing vendor identification
String.contains?/2 crashes on nil input. default nil sysDescr
and sysObjectID to empty string in find_vendor_match.
2026-02-11 12:28:58 -06:00
c9f154f8cd
fix 5 failing tests
- increase timing threshold in session activity test (flaky at 50ms)
- show org name instead of agent ID in admin agents list
- fix agent live tests wiping auth session with init_test_session
- add resilient error handling in FourOhFourTracker for missing redis
2026-02-11 12:21:06 -06:00
92eb583a8b
fix siteless devices not inheriting org default agent
devices assigned directly to an org (no site) were excluded from
apply_agent_to_all_equipment due to inner join through site table.
also fix fallback agent resolution to check device.organization
when device.site is nil.
2026-02-11 12:20:53 -06:00
bd24dfcbed
add diagnostic logging for agent result handling 2026-02-11 12:00:04 -06:00
e4623f0c3d
show agent ID instead of org name in admin agents list
Makes it easier to identify and reference specific agents in the admin view
2026-02-11 11:56:09 -06:00
f53332672b
add IP allowlist link to superadmin nav, rename whitelist/blacklist to allowlist/denylist 2026-02-11 11:40:44 -06:00
f2cd3488ee
fix heartbeat validation rejecting git-describe version strings
the version regex only allowed [a-zA-Z0-9.] in semver suffixes,
which rejected hyphens from git-describe output (e.g. 0.1.0-5-gabcdef0).
this caused every heartbeat to fail validation, preventing last_seen_at
from updating, which led to agent status showing "warning" and eventual
channel disconnect after 360s.

also accept bare git SHAs and short identifiers like "dev".
2026-02-11 11:31:19 -06:00
c5d8d73a0c
add row_link to table component for proper anchor tags
Replace phx-click JS.navigate with real <.link navigate> elements
on table rows so users can Ctrl+Click to open in new tabs, use
right-click context menus, and navigate via keyboard. Adds row_link
attr to <.table> core component and applies it to all agent tables.
Converts device list custom table from phx-click to inner links.
2026-02-11 08:46:03 -06:00
77cf289058
Add real-time device list with PubSub and LiveView streams
Broadcast org-scoped PubSub events from Devices context mutations
(create, update, delete, status change). Convert DeviceLive.Index
from assigns to streams with flat row items. Subscribe on mount so
the device list updates automatically across browser tabs.
2026-02-10 17:38:58 -06:00
138151ade6
Replace conditionals with functional pattern matching
Refactor cond/if/nested-case blocks into idiomatic multi-clause
functions across permissions, organizations, credential resolver,
SNMP client, and monitoring modules. Extract shared log_snmp_error
helper and dispatch_snmp for SNMP adapter routing.
2026-02-10 17:11:37 -06:00
4d73e77f3a
Add 404-based brute force protection system
Implements automated detection and blocking of IPs exhibiting scanning behavior (5+ unique 404s within 1 minute).

Key features:
- Progressive ban escalation (5 min → 1 hour → permanent)
- CIDR range and exact IP whitelisting
- Redis-backed 404 path tracking with 60s TTL
- Cloudflare WAF integration for permanent bans
- Admin UI for whitelist and blocked IP management
- Oban cron jobs for cleanup (expired bans, stale violations)
- ETS caching for whitelist performance

Components:
- Plug: ToweropsWeb.Plugs.BruteForceProtection
- Context: Towerops.Security.BruteForce
- Schemas: IpBlock, IpWhitelist
- Workers: CloudflareBanWorker, ExpiredBanCleanupWorker, StaleViolationCleanupWorker
- Admin UI: /admin/security (superuser only)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-10 16:31:48 -06:00
c7df6a8569
Add CI-triggered mass agent update webhook
POST /api/v1/webhooks/agent-release triggers all connected agents
to self-update via their existing WebSocket channels. Authenticated
with a shared secret (AGENT_WEBHOOK_SECRET env var).

- Add ReleaseChecker.invalidate_cache/0 for fresh GitHub fetch
- Add Agents.list_updatable_agents/0 (enabled + seen < 10min)
- Add Agents.broadcast_mass_update/0 orchestration function
- Add WebhookAuth plug with timing-safe secret comparison
- Add AgentReleaseWebhookController with :webhook pipeline
- Configure AGENT_WEBHOOK_SECRET in dev/test/runtime configs
2026-02-10 13:40:32 -06:00
e8abd8fd60
Update changelog for agent self-update feature
Added entry for agent self-update capability within Docker.
2026-02-10 13:28:48 -06:00
ecd9f78ba8
Add backward compatibility for PING results from older agents
Older agents send PING job results as SnmpResult messages instead of
MonitoringCheck messages, causing FunctionClauseError when
process_job_result/3 tried to pattern match on job_type: :PING.

Added handler that:
- Catches PING results in SnmpResult format
- Logs warning that agent should be upgraded
- Creates monitoring check with success status
- Stores result via store_monitoring_check

This prevents crashes while maintaining backward compatibility until
all production agents are upgraded to use MonitoringCheck format.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-10 13:18:40 -06:00
fbce65070a
Add agent restart and self-update commands
Add server-initiated restart and self-update capabilities for remote
agents. Restart broadcasts on the lifecycle PubSub topic causing the
agent to exit cleanly for Docker to restart. Self-update fetches the
latest release from GitHub, finds the matching binary for the agent's
architecture, and sends the download URL with SHA256 checksum.

- Add restart_agent/1 and update_agent/3 to Agents context
- Handle :restart_requested and :update_requested in AgentChannel
- Add superuser-only Restart and Update buttons on agent show page
- Add ReleaseChecker module for GitHub Releases API with 5-min cache
- Add arch field to AgentHeartbeat protobuf for architecture detection
- Store arch in agent metadata from heartbeat
2026-02-10 13:06:10 -06:00
c066399e18
Allow superadmins to view and edit any agent regardless of org
The admin agents page links to /agents/:id but the show and edit
pages were scoped to the current user's organization, causing 404s
for agents belonging to other orgs.
2026-02-10 12:19:20 -06:00
c1fca6276a
Add real-time updates to admin agents page via PubSub
Broadcast agent lifecycle events (created/deleted) on a separate
"admin:agents" topic so the admin page updates dynamically when
agents are added or removed from any organization.
2026-02-10 11:20:51 -06:00
c6caa81850
Group cloud pollers and org agents separately on admin agents page
Cloud pollers render in their own table at the top, org agents
below with the organization name shown under each agent name.
2026-02-10 11:07:41 -06:00
6a66203e48
Add link to admin agents page from admin dashboard 2026-02-10 11:04:55 -06:00
183f2a89ac
Add admin agents page at /admin/agents
Superuser-only page showing all agents across all organizations
and cloud pollers in a single table with live-ticking timestamps
and real-time status updates via PubSub.
2026-02-10 10:56:10 -06:00
5c59777baa
Live-ticking relative timestamps on agents page 2026-02-10 10:24:21 -06:00
481fc6d4a2
Add absolute format to timestamp component to fix render flash 2026-02-10 10:19:54 -06:00
d1cd567cd5
Show full timestamp below relative time in Last Seen column 2026-02-10 10:07:44 -06:00
e9afac3e9d
Show full timestamp below relative time on Last Seen card 2026-02-10 09:55:34 -06:00
6724a0a4bc
clean up agent view again 2026-02-10 09:51:28 -06:00
07a9366892
clean up agent view 2026-02-10 09:48:09 -06:00
ba56dee1ca
remove date helper 2026-02-10 09:15:41 -06:00
fce1a8b229
Show full version timestamp with friendly date on agents page 2026-02-10 09:04:56 -06:00
94405f0716
fix agents updates 2026-02-10 08:08:34 -06:00
144e04b998
change metadata version to version 2026-02-09 17:39:33 -06:00
95eafd0c47
routeros fixes and latency rounding 2026-02-09 17:30:06 -06:00
4cfab3b1ee
Reliability improvements: agent channel, polling, discovery, batch ops
- Add Task.Supervisor for supervised background tasks in agent channel
- Discovery worker retries transient failures (max 3 attempts)
- Log silent polling timeouts with named task identification
- Track errors in batch ARP/MAC upserts instead of silently ignoring
- Discovery gracefully degrades on interface/sensor timeout (partial results)
- Synchronous job dispatch on agent join to prevent race condition
- Surface SNMP result, monitoring check, and backup processing errors
- Atomic delete-stale + upsert for neighbors/ARP/MAC (transaction)
- Discovery transaction timeout (60s) with duration logging
- Heartbeat timeout detection (5min) closes stale agent channels
- Live poll timeout (60s) notifies waiting LiveView on expiry
- Disconnect agent channel when token is disabled or deleted
- Debounce rapid assignment changes to prevent redundant job dispatches
- Sensor index deduplication via YAML index template substitution
2026-02-09 16:28:12 -06:00
3b8e1ea639
Reliability improvements: Task.Supervisor, discovery retries, polling visibility
- Add Towerops.TaskSupervisor to supervision tree, replace bare spawn/1
  in agent channel with supervised tasks for polling data processing
- Change DiscoveryWorker from max_attempts: 1 to 3 with retryable error
  classification (transient errors retry, permanent errors discard)
- Name all 9 parallel polling tasks and log timeouts/crashes instead of
  silently dropping results via Stream.run()
- Add tests for TaskSupervisor presence and error classification
2026-02-09 16:05:30 -06:00
2911a3c17d
test fixes 2026-02-09 15:30:52 -06:00