Commit graph

73 commits

Author SHA1 Message Date
0995abbccb Add mandatory TOTP MFA 2026-01-28 15:09:38 -06:00
234ff7cbd2 add upsert support for geoip imports to prevent duplicates 2026-01-28 13:37:02 -06:00
d1fd1d4a62 remove Mix.env and support ip database import 2026-01-28 13:08:06 -06:00
220f1edce3 cookie improvements 2026-01-28 12:30:28 -06:00
d34c72a0ab track more snmp things 2026-01-28 09:18:31 -06:00
5fb92dd961
add device/org limit 2026-01-27 14:14:25 -06:00
3bb89670bd
enable agent debugging 2026-01-27 13:25:04 -06:00
bb86dec559
add oban pro 2026-01-26 17:07:42 -06:00
756a6b4cd4
poller agent fix and trigger device rediscovery 2026-01-26 14:51:32 -06:00
99414459d3
device reorder 2026-01-26 10:22:12 -06:00
33c5b977b0
ssl ecto 2026-01-25 15:53:28 -06:00
984b1b38c7
add pg_stat_statements 2026-01-25 15:44:59 -06:00
a23296ea34
fix: add missing sensor units to existing sensors
Sets sensor_unit for all sensors that are missing it, fixing sensors
created before the discovery code was updated to set units properly.

Updates:
- Temperature sensors: set to '°C'
- Voltage sensors: set to 'V'
- Current sensors: set to 'mA'
- Power sensors: set to 'W'
- dBm sensors: set to 'dBm'

This ensures the device detail page displays units correctly for all
sensor types (e.g., "27 °C" instead of "27" for temperature).

The discovery code already sets these units for new sensors, so this
migration only affects existing data.

Related to:
- lib/towerops/snmp/profiles/vendors/routeros.ex:690 (°C)
- lib/towerops/snmp/profiles/vendors/routeros.ex:709 (V)
- lib/towerops/snmp/profiles/base.ex:1473-1480 (sensor_type_to_unit)

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-25 12:38:23 -06:00
68a4ebf601
fix: correct misclassified MikroTik voltage sensors
Fixes voltage sensors that were incorrectly classified as temperature
sensors due to MikroTik devices reporting wrong mtxrGaugeUnit values.

Problem:
- Sensors with names like "psu1-voltage" and "psu2-voltage" were
  classified as temperature sensors with divisor=1
- This caused voltage readings to display as 469V instead of 46.9V
- The RouterOS profile already has name-based override logic for new
  discoveries, but existing sensors needed fixing

Solution:
- Migration identifies sensors by:
  - OID pattern: 1.3.6.1.4.1.14988.1.1.3.100.1.3.%
  - Type mismatch: sensor_type = 'temperature' but name contains 'voltage'
  - Incorrect divisor: sensor_divisor = 1
- Updates sensor metadata: type, unit, divisor
- Corrects historical sensor_reading values by dividing by 10

Future discoveries are already protected by the name-based override
logic in lib/towerops/snmp/profiles/vendors/routeros.ex:620-630

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-25 12:30:26 -06:00
8158042220
fix: correct Mikrotik voltage sensor divisor from 1 to 10
Mikrotik/RouterOS voltage sensors were showing incorrect values (469.0
instead of 46.9) because existing sensors in the database have
sensor_divisor=1 instead of the correct value of 10.

The RouterOS profile code was fixed in a previous commit to set
divisor=10 for new sensors, but existing sensors need to be updated.

Migration changes:
- Updates sensor_divisor from 1 to 10 for Mikrotik voltage sensors
- Identifies Mikrotik sensors by OID prefix: 1.3.6.1.4.1.14988.1.1.3.100.1.3
- Divides all historical sensor_reading values by 10 to correct the data
- Both up and down migrations included for full reversibility

After this migration:
- Voltage will show as 46.9V instead of 469.0V
- Historical graphs will show correct voltage values
2026-01-25 12:11:48 -06:00
29593ac734
refactor: migrate from etcd to Oban for distributed job coordination
Replaces etcd-based distributed locking with Oban's PostgreSQL-backed job queue.
This simplifies the architecture by eliminating the need for a separate etcd cluster
while providing better reliability and observability.

Changes:
- Add Oban dependency and migration (oban_jobs table)
- Create DevicePollerCoordinator and DeviceMonitorCoordinator Oban workers
- Remove EtcdCoordinator and EtcdLock modules
- Update application supervisor to start Oban
- Configure Oban with pollers (50 workers) and monitors (50 workers) queues
- Remove etcd StatefulSet from Kubernetes manifests
- Update monitoring supervisor documentation

Benefits:
- Simpler architecture (no etcd cluster to manage)
- PostgreSQL-based (uses existing database)
- Built-in uniqueness prevents duplicate jobs cluster-wide
- Better observability with Oban Web UI
- Automatic job rescue on node crashes
- Easier local development (no etcd required)

What was removed:
- etcd StatefulSet (3 pods)
- EtcdCoordinator module (320 lines)
- EtcdLock module (158 lines)
- eetcd dependency

All 3,686 tests passing.
2026-01-24 16:12:27 -06:00
e6054129b2
remote agent/cloud improvements 2026-01-24 09:16:41 -06:00
f084e09ea1
add multiple cloud agents 2026-01-23 18:04:01 -06:00
80d1864700
mac discovery 2026-01-23 09:22:08 -06:00
7bd3b1f9ec
arp 2026-01-23 08:40:57 -06:00
f402dcb7af
fix mikrotik sensors 2026-01-22 18:07:20 -06:00
96fc488023
sanitize snmp data and fix some cpu things 2026-01-22 16:53:01 -06:00
17f4558702
implement timescaledb improvements 2026-01-22 13:24:14 -06:00
1fc066d95b
feat: add memory pool and storage discovery (Phase 1.5)
- Add snmp_memory_pools table and MemoryPool schema
- Add snmp_storage table and Storage schema
- Add discover_memory_pools/1 for HOST-RESOURCES-MIB RAM/swap
- Add discover_storage/1 for HOST-RESOURCES-MIB disk storage
- Add comprehensive tests for schema validation and discovery
- Completes Phase 1 of discovery improvements
2026-01-21 10:57:08 -06:00
45ddcdac80
feat: add physical inventory discovery (Phase 1.4)
- Create snmp_physical_entities table with migration
- Add PhysicalEntity schema with entity_class validation
- Support parent/child hierarchy via parent_entity_id
- Implement discover_physical_entities/1 in Base profile
- Discover chassis, modules, PSUs, fans from ENTITY-MIB
- Map entity class integers to string names
- Add 10 schema tests and 4 discovery tests
- Fix flaky AlertNotifier test with unique names and mailbox clearing
2026-01-21 10:46:27 -06:00
1cf6f327ea
feat: add IP address discovery (Phase 1.3)
- Create snmp_ip_addresses table with migration
- Add IpAddress schema with ip_type validation (ipv4/ipv6)
- Add prefix_length validation based on ip_type
- Implement discover_ip_addresses/1 in Base profile
- Extract IP addresses from IP-MIB ipAdEntTable
- Support multiple IPs per interface with subnet masks
- Add 12 schema tests and 4 discovery tests
2026-01-21 10:36:19 -06:00
579a7bac21
add VLAN schema, migration, and discovery 2026-01-21 10:31:21 -06:00
c3e26a44d6
add state sensor schema, migration, and discovery 2026-01-21 10:25:01 -06:00
fe7a44e5e2
test improvements 2026-01-19 12:08:48 -06:00
9f59e7661a
add universoal import 2026-01-18 16:59:34 -06:00
1f123bbeb9
add honeybadger and snmp overhaul 2026-01-18 16:16:08 -06:00
94311a48aa
discovery fix and favicon 2026-01-18 13:47:19 -06:00
1c400259fc
fix: use microsecond precision for ICMP latency measurements
- Changed ping module to use microsecond timer resolution instead of millisecond
- Prevents 0ms readings for very fast pings (localhost, local network)
- Updated response_time_ms to float type to store decimal precision
- Migration converts existing integer values to float
2026-01-18 12:52:29 -06:00
5b1b9a08c7
improve profile import 2026-01-18 11:46:56 -06:00
f941e13f0d
add user profile settings with name, avatar, and timezone
- Add migration for name, avatar_url, timezone fields
- Update User schema with profile_changeset
- Add Accounts context functions for profile updates
- Reorganize settings page into sections:
  * Personal Information (name, avatar, timezone)
  * Account (email, password)
  * API (tokens)
  * Notifications (mobile devices)
  * Security (passkeys)
- Avatar URL accepts external URLs (Gravatar, etc)
- Timezone selector with common zones
2026-01-18 10:32:23 -06:00
56093bb493
refactor: use API token auth for profile imports instead of session cookies
Changes profile import endpoint to use standard API token authentication:

API Token Changes:
- Add user_id to api_tokens table (tracks who created the token)
- Update ApiTokens.verify_token/1 to return user along with org_id
- Update ApiAuth plug to assign current_user from token

Profile Import Changes:
- Move endpoint from /api/v1/admin/profiles/import to /api/v1/profiles/import
- Check user.is_superuser in controller instead of using RequireSuperuser plug
- Use api_v1 pipeline (Bearer token auth) instead of browser session
- Update documentation to show API token usage

Security:
- Only API tokens created by superusers can import profiles
- Returns 403 Forbidden if token user is not a superuser
- Logs import attempts with user email for audit trail

This provides a consistent API experience using Bearer tokens
instead of requiring browser session cookies.
2026-01-18 09:30:21 -06:00
b83484acf8
feat: poll actual firmware version and serial from SNMP
Enhance dynamic profile to poll OS definitions from SNMP:
- Add serial_number field to snmp_devices table
- Update Dynamic profile to query profile_os_definitions
- Poll SNMP OIDs for version and serial fields
- Display actual firmware version instead of profile name

For ePMP devices, this will now query:
- CAMBIUM-PMP80211-MIB::cambiumCurrentuImageVersion.0 for firmware
- CAMBIUM-PMP80211-MIB::cambiumEPMPMSN.0 for serial number

The Operating System field will now show actual version like
'Cambium 4.7.1' instead of 'Cambium epmp'.
2026-01-17 18:25:52 -06:00
007adcc489
feat: add database-driven SNMP device profiles
Implement dynamic device profile system using LibreNMS YAML definitions:

- Create 7 database tables for device profiles, detection rules, sensors, processors, memory pools
- Add DeviceProfiles context module with profile matching logic
- Add YAML importer to parse LibreNMS os_detection and os_discovery files
- Add mix task for importing profiles from LibreNMS repository
- Create Dynamic profile module to interpret database definitions at runtime
- Update discovery.ex to check database profiles before hard-coded modules
- Fix Redis PubSub configuration to support unnamed nodes for Mix tasks

Imported 5 Ubiquiti/Cambium profiles: epmp, airos-af, airos-af-ltu, airos-af60, unifi

The system now supports 671+ device profiles from LibreNMS without requiring
code changes for each device type.
2026-01-17 18:13:53 -06:00
412086d7ec
Fix alerts page by migrating old equipment_* alert types to device_*
The Alert schema was updated to use :device_down and :device_up enum values,
but existing database records still had 'equipment_down' and 'equipment_up' values.

Added migration to update all existing alert records to use the new naming
convention.
2026-01-17 17:15:18 -06:00
c4760ca0dc
Add API v1 endpoints with organization-scoped API tokens
Created API token system for programmatic access:
- API tokens table with organization scoping
- ApiTokens context for token management
- ApiAuth plug for Bearer token authentication
- Tokens shown once on creation, then hashed for security

Implemented RESTful API v1 endpoints:
- SitesController: CRUD operations for sites
- DevicesController: CRUD operations for devices
- All operations scoped to authenticated organization
- Proper authorization checks via site ownership

Technical details:
- Tokens prefixed with "towerops_" for easy identification
- SHA-256 hashing for token storage
- Last used timestamp tracking (async update)
- Optional token expiration support
- Standard JSON error responses (40x status codes)

Routes:
- /api/v1/sites (GET, POST, PATCH, DELETE)
- /api/v1/devices (GET, POST, PATCH, DELETE)

Authentication:
- Authorization: Bearer towerops_xxxxx header required
- Returns 401 for invalid/expired tokens
- Returns 403 for unauthorized resource access
2026-01-17 16:53:31 -06:00
560370cb0b
feat: allow device name to be populated from SNMP sysName
- Make device name field optional when SNMP is enabled (ICMP Only mode still requires name)
- Add database migration to make name column nullable
- Add conditional validation: name required only when snmp_enabled is false
- Update SNMP discovery to populate device name from sysName when empty
- Add helper text explaining SNMP name population behavior in form
- Add comprehensive tests for device name SNMP population
- Update test mocks to account for NetSnmp profile sensor discovery
2026-01-17 16:28:43 -06:00
a810e75fc4
rename equipment to device 2026-01-17 14:48:46 -06:00
ba1e8933e4
Add hierarchical SNMP configuration with org/site/equipment fallback
Implements a hierarchical SNMP community configuration system where settings
can be defined at the organization level (global default), overridden at the
site level, or overridden at the equipment level.

Changes:
- Migration: Add snmp_version and snmp_community to organizations and sites
- Organization schema: Add SNMP fields with default version "2c"
- Site schema: Add SNMP fields that override organization defaults
- Equipment context: Add get_snmp_config/1 with hierarchical fallback logic
- Discovery: Use Equipment.get_snmp_config instead of equipment.snmp_community
- PollerWorker: Use Equipment.get_snmp_config instead of equipment.snmp_community

The fallback order is:
1. Equipment-level (highest priority)
2. Site-level
3. Organization-level (default)

All 785 tests passing.
2026-01-17 11:47:17 -06:00
dcd4588304
fix neighbors 2026-01-16 16:09:15 -06:00
6627235981
add tests and add snmp neighbor discovery 2026-01-16 13:44:29 -06:00
3539e36f12
updates 2026-01-16 13:15:59 -06:00
c7f02eac24
add api for mobile login 2026-01-15 15:36:14 -06:00
ccae030569
Remove duplicate agent index migration
Removed 20260115010756_add_agent_query_composite_indexes.exs which
was a duplicate of 20260115010646_add_composite_indexes_for_agent_queries.exs.

Both had already run but the duplicate one used create_if_not_exists
so it didn't fail. Cleaned up to avoid confusion.
2026-01-15 07:29:49 -06:00
54b08bfb18 Add database indexes and optimize SNMP polling
- Added partial indexes for alerts and SNMP equipment queries
- Fixed missing agent_assignments association in Equipment schema
- Fixed missing preloads to avoid N+1 queries
- Optimized Rust agent SNMP credential field access
2026-01-15 07:28:14 -06:00
96706b2cf8
Deduplicate agent status helper functions
Extracted 5 shared helper functions into ToweropsWeb.AgentLive.Helpers:
- agent_status/1 - Determines online/warning/offline/never status
- status_badge_class/1 - Returns Tailwind CSS classes for status badges
- status_dot_class/1 - Returns CSS classes for animated status dots
- format_last_seen/1 - Formats DateTime as human-readable time ago
- format_uptime/1 - Formats uptime seconds as days/hours/minutes

Removed duplicate code from:
- lib/towerops_web/live/agent_live/index.ex
- lib/towerops_web/live/agent_live/show.ex

Both modules now import the shared helpers, following DRY principle.
This reduces code duplication and makes status formatting consistent
across all agent-related pages.
2026-01-14 19:12:06 -06:00