Commit graph

75 commits

Author SHA1 Message Date
c36b1873eb
routeros fix 2026-01-25 16:59:32 -06:00
33c5b977b0
ssl ecto 2026-01-25 15:53:28 -06:00
984b1b38c7
add pg_stat_statements 2026-01-25 15:44:59 -06:00
6e86f54aa0
mikrotik sensor fixes 2026-01-25 13:24:28 -06:00
a23296ea34
fix: add missing sensor units to existing sensors
Sets sensor_unit for all sensors that are missing it, fixing sensors
created before the discovery code was updated to set units properly.

Updates:
- Temperature sensors: set to '°C'
- Voltage sensors: set to 'V'
- Current sensors: set to 'mA'
- Power sensors: set to 'W'
- dBm sensors: set to 'dBm'

This ensures the device detail page displays units correctly for all
sensor types (e.g., "27 °C" instead of "27" for temperature).

The discovery code already sets these units for new sensors, so this
migration only affects existing data.

Related to:
- lib/towerops/snmp/profiles/vendors/routeros.ex:690 (°C)
- lib/towerops/snmp/profiles/vendors/routeros.ex:709 (V)
- lib/towerops/snmp/profiles/base.ex:1473-1480 (sensor_type_to_unit)

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-25 12:38:23 -06:00
68a4ebf601
fix: correct misclassified MikroTik voltage sensors
Fixes voltage sensors that were incorrectly classified as temperature
sensors due to MikroTik devices reporting wrong mtxrGaugeUnit values.

Problem:
- Sensors with names like "psu1-voltage" and "psu2-voltage" were
  classified as temperature sensors with divisor=1
- This caused voltage readings to display as 469V instead of 46.9V
- The RouterOS profile already has name-based override logic for new
  discoveries, but existing sensors needed fixing

Solution:
- Migration identifies sensors by:
  - OID pattern: 1.3.6.1.4.1.14988.1.1.3.100.1.3.%
  - Type mismatch: sensor_type = 'temperature' but name contains 'voltage'
  - Incorrect divisor: sensor_divisor = 1
- Updates sensor metadata: type, unit, divisor
- Corrects historical sensor_reading values by dividing by 10

Future discoveries are already protected by the name-based override
logic in lib/towerops/snmp/profiles/vendors/routeros.ex:620-630

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-25 12:30:26 -06:00
8158042220
fix: correct Mikrotik voltage sensor divisor from 1 to 10
Mikrotik/RouterOS voltage sensors were showing incorrect values (469.0
instead of 46.9) because existing sensors in the database have
sensor_divisor=1 instead of the correct value of 10.

The RouterOS profile code was fixed in a previous commit to set
divisor=10 for new sensors, but existing sensors need to be updated.

Migration changes:
- Updates sensor_divisor from 1 to 10 for Mikrotik voltage sensors
- Identifies Mikrotik sensors by OID prefix: 1.3.6.1.4.1.14988.1.1.3.100.1.3
- Divides all historical sensor_reading values by 10 to correct the data
- Both up and down migrations included for full reversibility

After this migration:
- Voltage will show as 46.9V instead of 469.0V
- Historical graphs will show correct voltage values
2026-01-25 12:11:48 -06:00
29593ac734
refactor: migrate from etcd to Oban for distributed job coordination
Replaces etcd-based distributed locking with Oban's PostgreSQL-backed job queue.
This simplifies the architecture by eliminating the need for a separate etcd cluster
while providing better reliability and observability.

Changes:
- Add Oban dependency and migration (oban_jobs table)
- Create DevicePollerCoordinator and DeviceMonitorCoordinator Oban workers
- Remove EtcdCoordinator and EtcdLock modules
- Update application supervisor to start Oban
- Configure Oban with pollers (50 workers) and monitors (50 workers) queues
- Remove etcd StatefulSet from Kubernetes manifests
- Update monitoring supervisor documentation

Benefits:
- Simpler architecture (no etcd cluster to manage)
- PostgreSQL-based (uses existing database)
- Built-in uniqueness prevents duplicate jobs cluster-wide
- Better observability with Oban Web UI
- Automatic job rescue on node crashes
- Easier local development (no etcd required)

What was removed:
- etcd StatefulSet (3 pods)
- EtcdCoordinator module (320 lines)
- EtcdLock module (158 lines)
- eetcd dependency

All 3,686 tests passing.
2026-01-24 16:12:27 -06:00
e6054129b2
remote agent/cloud improvements 2026-01-24 09:16:41 -06:00
f084e09ea1
add multiple cloud agents 2026-01-23 18:04:01 -06:00
80d1864700
mac discovery 2026-01-23 09:22:08 -06:00
7bd3b1f9ec
arp 2026-01-23 08:40:57 -06:00
f402dcb7af
fix mikrotik sensors 2026-01-22 18:07:20 -06:00
96fc488023
sanitize snmp data and fix some cpu things 2026-01-22 16:53:01 -06:00
17f4558702
implement timescaledb improvements 2026-01-22 13:24:14 -06:00
7656ac7212
import yamls 2026-01-21 14:30:04 -06:00
1fc066d95b
feat: add memory pool and storage discovery (Phase 1.5)
- Add snmp_memory_pools table and MemoryPool schema
- Add snmp_storage table and Storage schema
- Add discover_memory_pools/1 for HOST-RESOURCES-MIB RAM/swap
- Add discover_storage/1 for HOST-RESOURCES-MIB disk storage
- Add comprehensive tests for schema validation and discovery
- Completes Phase 1 of discovery improvements
2026-01-21 10:57:08 -06:00
45ddcdac80
feat: add physical inventory discovery (Phase 1.4)
- Create snmp_physical_entities table with migration
- Add PhysicalEntity schema with entity_class validation
- Support parent/child hierarchy via parent_entity_id
- Implement discover_physical_entities/1 in Base profile
- Discover chassis, modules, PSUs, fans from ENTITY-MIB
- Map entity class integers to string names
- Add 10 schema tests and 4 discovery tests
- Fix flaky AlertNotifier test with unique names and mailbox clearing
2026-01-21 10:46:27 -06:00
1cf6f327ea
feat: add IP address discovery (Phase 1.3)
- Create snmp_ip_addresses table with migration
- Add IpAddress schema with ip_type validation (ipv4/ipv6)
- Add prefix_length validation based on ip_type
- Implement discover_ip_addresses/1 in Base profile
- Extract IP addresses from IP-MIB ipAdEntTable
- Support multiple IPs per interface with subnet masks
- Add 12 schema tests and 4 discovery tests
2026-01-21 10:36:19 -06:00
579a7bac21
add VLAN schema, migration, and discovery 2026-01-21 10:31:21 -06:00
c3e26a44d6
add state sensor schema, migration, and discovery 2026-01-21 10:25:01 -06:00
b4f8b40b7f
Include MIB files in Docker image instead of using PVC
- Remove PVC volume mount from k8s/deployment.yaml
- Delete k8s/mib-pvc.yaml (no longer needed)
- Update .gitignore to commit MIB files to git
- Add mix import_mibs task to copy MIBs from LibreNMS
- Add all MIB files from priv/mibs/ to git

This fixes the multi-attach volume error in Kubernetes where new pods
couldn't start because the RWO (ReadWriteOnce) PVC was attached to the
old pod. MIBs are now part of the Docker image and can be used by all
pods simultaneously.
2026-01-19 14:01:03 -06:00
f497de4474
Add ICMP monitoring support to Phoenix agent channel
- Add monitoring_enabled and check_interval_seconds fields to Device and SnmpDevice protobuf messages
- Add MonitoringCheck protobuf message for ping results
- Update AgentChannel to handle monitoring_check events from agents
- Configure monitoring settings in job payloads sent to agents
2026-01-19 13:46:20 -06:00
fe7a44e5e2
test improvements 2026-01-19 12:08:48 -06:00
49bc320a89
better mib handling 2026-01-19 09:01:56 -06:00
9f59e7661a
add universoal import 2026-01-18 16:59:34 -06:00
1f123bbeb9
add honeybadger and snmp overhaul 2026-01-18 16:16:08 -06:00
94311a48aa
discovery fix and favicon 2026-01-18 13:47:19 -06:00
6706613f24
add texas footer 2026-01-18 13:15:27 -06:00
1c400259fc
fix: use microsecond precision for ICMP latency measurements
- Changed ping module to use microsecond timer resolution instead of millisecond
- Prevents 0ms readings for very fast pings (localhost, local network)
- Updated response_time_ms to float type to store decimal precision
- Migration converts existing integer values to float
2026-01-18 12:52:29 -06:00
5b1b9a08c7
improve profile import 2026-01-18 11:46:56 -06:00
f941e13f0d
add user profile settings with name, avatar, and timezone
- Add migration for name, avatar_url, timezone fields
- Update User schema with profile_changeset
- Add Accounts context functions for profile updates
- Reorganize settings page into sections:
  * Personal Information (name, avatar, timezone)
  * Account (email, password)
  * API (tokens)
  * Notifications (mobile devices)
  * Security (passkeys)
- Avatar URL accepts external URLs (Gravatar, etc)
- Timezone selector with common zones
2026-01-18 10:32:23 -06:00
56093bb493
refactor: use API token auth for profile imports instead of session cookies
Changes profile import endpoint to use standard API token authentication:

API Token Changes:
- Add user_id to api_tokens table (tracks who created the token)
- Update ApiTokens.verify_token/1 to return user along with org_id
- Update ApiAuth plug to assign current_user from token

Profile Import Changes:
- Move endpoint from /api/v1/admin/profiles/import to /api/v1/profiles/import
- Check user.is_superuser in controller instead of using RequireSuperuser plug
- Use api_v1 pipeline (Bearer token auth) instead of browser session
- Update documentation to show API token usage

Security:
- Only API tokens created by superusers can import profiles
- Returns 403 Forbidden if token user is not a superuser
- Logs import attempts with user email for audit trail

This provides a consistent API experience using Bearer tokens
instead of requiring browser session cookies.
2026-01-18 09:30:21 -06:00
b83484acf8
feat: poll actual firmware version and serial from SNMP
Enhance dynamic profile to poll OS definitions from SNMP:
- Add serial_number field to snmp_devices table
- Update Dynamic profile to query profile_os_definitions
- Poll SNMP OIDs for version and serial fields
- Display actual firmware version instead of profile name

For ePMP devices, this will now query:
- CAMBIUM-PMP80211-MIB::cambiumCurrentuImageVersion.0 for firmware
- CAMBIUM-PMP80211-MIB::cambiumEPMPMSN.0 for serial number

The Operating System field will now show actual version like
'Cambium 4.7.1' instead of 'Cambium epmp'.
2026-01-17 18:25:52 -06:00
007adcc489
feat: add database-driven SNMP device profiles
Implement dynamic device profile system using LibreNMS YAML definitions:

- Create 7 database tables for device profiles, detection rules, sensors, processors, memory pools
- Add DeviceProfiles context module with profile matching logic
- Add YAML importer to parse LibreNMS os_detection and os_discovery files
- Add mix task for importing profiles from LibreNMS repository
- Create Dynamic profile module to interpret database definitions at runtime
- Update discovery.ex to check database profiles before hard-coded modules
- Fix Redis PubSub configuration to support unnamed nodes for Mix tasks

Imported 5 Ubiquiti/Cambium profiles: epmp, airos-af, airos-af-ltu, airos-af60, unifi

The system now supports 671+ device profiles from LibreNMS without requiring
code changes for each device type.
2026-01-17 18:13:53 -06:00
412086d7ec
Fix alerts page by migrating old equipment_* alert types to device_*
The Alert schema was updated to use :device_down and :device_up enum values,
but existing database records still had 'equipment_down' and 'equipment_up' values.

Added migration to update all existing alert records to use the new naming
convention.
2026-01-17 17:15:18 -06:00
c4760ca0dc
Add API v1 endpoints with organization-scoped API tokens
Created API token system for programmatic access:
- API tokens table with organization scoping
- ApiTokens context for token management
- ApiAuth plug for Bearer token authentication
- Tokens shown once on creation, then hashed for security

Implemented RESTful API v1 endpoints:
- SitesController: CRUD operations for sites
- DevicesController: CRUD operations for devices
- All operations scoped to authenticated organization
- Proper authorization checks via site ownership

Technical details:
- Tokens prefixed with "towerops_" for easy identification
- SHA-256 hashing for token storage
- Last used timestamp tracking (async update)
- Optional token expiration support
- Standard JSON error responses (40x status codes)

Routes:
- /api/v1/sites (GET, POST, PATCH, DELETE)
- /api/v1/devices (GET, POST, PATCH, DELETE)

Authentication:
- Authorization: Bearer towerops_xxxxx header required
- Returns 401 for invalid/expired tokens
- Returns 403 for unauthorized resource access
2026-01-17 16:53:31 -06:00
560370cb0b
feat: allow device name to be populated from SNMP sysName
- Make device name field optional when SNMP is enabled (ICMP Only mode still requires name)
- Add database migration to make name column nullable
- Add conditional validation: name required only when snmp_enabled is false
- Update SNMP discovery to populate device name from sysName when empty
- Add helper text explaining SNMP name population behavior in form
- Add comprehensive tests for device name SNMP population
- Update test mocks to account for NetSnmp profile sensor discovery
2026-01-17 16:28:43 -06:00
a810e75fc4
rename equipment to device 2026-01-17 14:48:46 -06:00
ba1e8933e4
Add hierarchical SNMP configuration with org/site/equipment fallback
Implements a hierarchical SNMP community configuration system where settings
can be defined at the organization level (global default), overridden at the
site level, or overridden at the equipment level.

Changes:
- Migration: Add snmp_version and snmp_community to organizations and sites
- Organization schema: Add SNMP fields with default version "2c"
- Site schema: Add SNMP fields that override organization defaults
- Equipment context: Add get_snmp_config/1 with hierarchical fallback logic
- Discovery: Use Equipment.get_snmp_config instead of equipment.snmp_community
- PollerWorker: Use Equipment.get_snmp_config instead of equipment.snmp_community

The fallback order is:
1. Equipment-level (highest priority)
2. Site-level
3. Organization-level (default)

All 785 tests passing.
2026-01-17 11:47:17 -06:00
eb91798729
agent improvements and agent config 2026-01-16 17:26:51 -06:00
dcd4588304
fix neighbors 2026-01-16 16:09:15 -06:00
6627235981
add tests and add snmp neighbor discovery 2026-01-16 13:44:29 -06:00
3539e36f12
updates 2026-01-16 13:15:59 -06:00
c7f02eac24
add api for mobile login 2026-01-15 15:36:14 -06:00
ccae030569
Remove duplicate agent index migration
Removed 20260115010756_add_agent_query_composite_indexes.exs which
was a duplicate of 20260115010646_add_composite_indexes_for_agent_queries.exs.

Both had already run but the duplicate one used create_if_not_exists
so it didn't fail. Cleaned up to avoid confusion.
2026-01-15 07:29:49 -06:00
54b08bfb18 Add database indexes and optimize SNMP polling
- Added partial indexes for alerts and SNMP equipment queries
- Fixed missing agent_assignments association in Equipment schema
- Fixed missing preloads to avoid N+1 queries
- Optimized Rust agent SNMP credential field access
2026-01-15 07:28:14 -06:00
96706b2cf8
Deduplicate agent status helper functions
Extracted 5 shared helper functions into ToweropsWeb.AgentLive.Helpers:
- agent_status/1 - Determines online/warning/offline/never status
- status_badge_class/1 - Returns Tailwind CSS classes for status badges
- status_dot_class/1 - Returns CSS classes for animated status dots
- format_last_seen/1 - Formats DateTime as human-readable time ago
- format_uptime/1 - Formats uptime seconds as days/hours/minutes

Removed duplicate code from:
- lib/towerops_web/live/agent_live/index.ex
- lib/towerops_web/live/agent_live/show.ex

Both modules now import the shared helpers, following DRY principle.
This reduces code duplication and makes status formatting consistent
across all agent-related pages.
2026-01-14 19:12:06 -06:00
a2d96f8e6e
Implement hierarchical agent assignment for SNMP polling
Add three-level agent assignment hierarchy (Equipment > Site > Organization)
allowing flexible agent deployment strategies. Agents now receive only equipment
assigned to them through direct assignment or inheritance from site/organization
defaults.

Key changes:
- Add agent_token_id to Sites table with migration
- Implement get_effective_agent_token/1 for hierarchical resolution
- Add list_agent_polling_targets/1 to return polling targets per agent
- Update API config endpoint to use hierarchical polling targets
- Add agent assignment UI to equipment, site, and organization forms
- Show agent source (direct/site/org/none) in equipment form
- Add equipment count column to agent list
- Update terminology from "poll from server" to "cloud polling"

Tests:
- Add 8 comprehensive tests for list_agent_polling_targets/1
- Add end-to-end test for hierarchical config endpoint
- All 775 tests passing
2026-01-14 08:38:50 -06:00
b8bd952681
agent token changes 2026-01-13 13:43:43 -06:00