Created API token system for programmatic access:
- API tokens table with organization scoping
- ApiTokens context for token management
- ApiAuth plug for Bearer token authentication
- Tokens shown once on creation, then hashed for security
Implemented RESTful API v1 endpoints:
- SitesController: CRUD operations for sites
- DevicesController: CRUD operations for devices
- All operations scoped to authenticated organization
- Proper authorization checks via site ownership
Technical details:
- Tokens prefixed with "towerops_" for easy identification
- SHA-256 hashing for token storage
- Last used timestamp tracking (async update)
- Optional token expiration support
- Standard JSON error responses (40x status codes)
Routes:
- /api/v1/sites (GET, POST, PATCH, DELETE)
- /api/v1/devices (GET, POST, PATCH, DELETE)
Authentication:
- Authorization: Bearer towerops_xxxxx header required
- Returns 401 for invalid/expired tokens
- Returns 403 for unauthorized resource access
- Make device name field optional when SNMP is enabled (ICMP Only mode still requires name)
- Add database migration to make name column nullable
- Add conditional validation: name required only when snmp_enabled is false
- Update SNMP discovery to populate device name from sysName when empty
- Add helper text explaining SNMP name population behavior in form
- Add comprehensive tests for device name SNMP population
- Update test mocks to account for NetSnmp profile sensor discovery
Implements a hierarchical SNMP community configuration system where settings
can be defined at the organization level (global default), overridden at the
site level, or overridden at the equipment level.
Changes:
- Migration: Add snmp_version and snmp_community to organizations and sites
- Organization schema: Add SNMP fields with default version "2c"
- Site schema: Add SNMP fields that override organization defaults
- Equipment context: Add get_snmp_config/1 with hierarchical fallback logic
- Discovery: Use Equipment.get_snmp_config instead of equipment.snmp_community
- PollerWorker: Use Equipment.get_snmp_config instead of equipment.snmp_community
The fallback order is:
1. Equipment-level (highest priority)
2. Site-level
3. Organization-level (default)
All 785 tests passing.
Removed 20260115010756_add_agent_query_composite_indexes.exs which
was a duplicate of 20260115010646_add_composite_indexes_for_agent_queries.exs.
Both had already run but the duplicate one used create_if_not_exists
so it didn't fail. Cleaned up to avoid confusion.
Extracted 5 shared helper functions into ToweropsWeb.AgentLive.Helpers:
- agent_status/1 - Determines online/warning/offline/never status
- status_badge_class/1 - Returns Tailwind CSS classes for status badges
- status_dot_class/1 - Returns CSS classes for animated status dots
- format_last_seen/1 - Formats DateTime as human-readable time ago
- format_uptime/1 - Formats uptime seconds as days/hours/minutes
Removed duplicate code from:
- lib/towerops_web/live/agent_live/index.ex
- lib/towerops_web/live/agent_live/show.ex
Both modules now import the shared helpers, following DRY principle.
This reduces code duplication and makes status formatting consistent
across all agent-related pages.
Add three-level agent assignment hierarchy (Equipment > Site > Organization)
allowing flexible agent deployment strategies. Agents now receive only equipment
assigned to them through direct assignment or inheritance from site/organization
defaults.
Key changes:
- Add agent_token_id to Sites table with migration
- Implement get_effective_agent_token/1 for hierarchical resolution
- Add list_agent_polling_targets/1 to return polling targets per agent
- Update API config endpoint to use hierarchical polling targets
- Add agent assignment UI to equipment, site, and organization forms
- Show agent source (direct/site/org/none) in equipment form
- Add equipment count column to agent list
- Update terminology from "poll from server" to "cloud polling"
Tests:
- Add 8 comprehensive tests for list_agent_polling_targets/1
- Add end-to-end test for hierarchical config endpoint
- All 775 tests passing
- Add MIB files from LibreNMS in priv/mibs/ for reference
- Create MibParser module to validate OIDs against official MIB definitions
- Add MIB validation tests to ensure hardcoded OIDs match MIB specs
- Refactor SNMP tests to be generic/behavior-focused instead of vendor-specific
- Remove vendor-specific test files (cisco_test, net_snmp_test)
- All 104 tests passing with automated OID validation
Implement comprehensive admin interface allowing designated superusers to view all users and organizations, impersonate users for debugging, and perform administrative operations. All superuser actions are tracked in audit logs for compliance.
Features:
- Superuser authentication with dedicated admin routes at /admin
- User impersonation with session state preservation
- Admin dashboard with system statistics
- User and organization management interfaces
- Comprehensive audit logging with IP tracking
- Visual impersonation banner with exit capability
- Security controls preventing self-impersonation and superuser-to-superuser impersonation
Database:
- Add is_superuser boolean field to users table
- Create audit_logs table for tracking sensitive operations
- Set graham@mcintire.me as initial superuser
- Add equipment_events table and Event schema for tracking device changes
- Implement automatic change detection for interface attributes during SNMP polling
- Detects operational status changes (up/down)
- Detects admin status changes
- Detects speed changes with warning severity for speed drops
- Detects MAC address changes
- Add Logs tab to equipment detail page with event display
- Fix temperature sensor display bug (was showing 4.3C instead of 43C due to double division)
- Remove response time tracking from monitoring checks and UI
- Move Rediscover Device button to equipment edit page
This fixes critical issues where SNMP sensor readings, interface stats, and
monitoring checks were not being saved to the database due to schema mismatches.
Database schema fixes:
- Recreate snmp_sensor_readings table with binary_id primary key
- Recreate snmp_interface_stats table with binary_id primary key
- Recreate monitoring_checks table with binary_id primary key
The original migrations used default integer primary keys, but the schemas
expected binary_id (UUID). This caused Ecto to generate UUID strings that
Postgrex tried to encode as binaries, resulting in silent insert failures.
SNMP interface stats fixes:
- Fix field name mismatch in get_interface_stats (if_in_octets vs in_octets)
- Remove unnecessary tuple handling code (Client already unwraps SNMPKit tuples)
- Clean up decode_snmp_value function and improve error messages
Monitoring check fixes:
- Fix response_time type from float to integer in Poller.check_device
- Add error logging to catch and report check creation failures
These changes enable:
- SNMP sensor readings to be collected and stored (disk usage, memory, CPU, etc.)
- Interface statistics to be collected (Counter64 octets, errors, discards)
- Equipment availability metrics to update on the dashboard
Prevents duplicate polling when multiple pods are running by using
database-based coordination.
Changes:
- Added last_snmp_poll_at timestamp to equipment table
- Added Equipment.update_snmp_poll_time/1 function
- Updated PollerWorker to check last_snmp_poll_at before polling
- Polls are skipped if equipment was polled within (interval - 5s)
- Updates timestamp before polling (optimistic locking)
This ensures that when K8s scales up/down or pods restart, only one
pod polls each piece of equipment at a time, preventing wasteful
duplicate SNMP queries and database writes.
The 5-second grace period accounts for clock drift and processing delays
between pods.
Storage sensors (disk/memory usage) need to fetch two OIDs to calculate
percentages: the 'used' value and the 'size' value. Previously, only the
'used' OID was stored and polled, resulting in meaningless raw values.
Changes:
- Added metadata JSONB field to snmp_sensors table
- Updated Sensor schema to include metadata field
- Updated MikroTik profile to store size_oid in metadata for storage sensors
- Updated PollerWorker to handle percentage calculation sensors:
- poll_simple_sensor: Standard OID fetch with divisor
- poll_percentage_sensor: Fetches both used and size OIDs, calculates percentage
Storage sensors now properly calculate and store percentage values during polling.
NOTE: Existing MikroTik devices need to re-run SNMP discovery to populate
the metadata field for storage sensors.
- Convert snmp_sensor_readings to standard table
- Convert snmp_interface_stats to standard table
- Disable snmp_sensor_aggregates (continuous aggregates)
- Disable snmp_interface_aggregates (continuous aggregates)
All TimescaleDB commercial features (hypertables, continuous aggregates,
retention/compression policies) are disabled. Using standard PostgreSQL
tables with proper indexing instead.
Commented out retention and compression policies which require
TimescaleDB commercial license. Apache-licensed version only supports
basic hypertable functionality.
Monitoring Infrastructure:
- lib/towerops/monitoring/ping.ex - OS-agnostic ping functionality that works on macOS, Linux, and Windows
- lib/towerops/monitoring/equipment_monitor.ex - GenServer that monitors individual equipment at configurable intervals
- lib/towerops/monitoring/supervisor.ex - DynamicSupervisor with Registry for managing monitor workers
- lib/towerops/monitoring.ex - Context for creating and querying monitoring checks
Database:
- monitoring_checks table stores historical ping results with status, response time, and timestamps
- Indexed on (equipment_id, checked_at) for efficient querying
Real-Time Updates:
- PubSub broadcasting on equipment:#{id} topic when status changes
- Equipment show page subscribes to updates and refreshes automatically
- Manual "Check Now" button to trigger immediate checks
Application Integration:
- Monitoring.Supervisor added to application supervision tree
- All monitored equipment starts monitoring automatically on app startup
- Recent checks displayed in Equipment show page
How It Works
1. On application start, a monitor GenServer is spawned for each equipment with monitoring_enabled: true
2. Each monitor pings its equipment at the configured interval (default 5 minutes)
3. Results are saved to monitoring_checks table
4. Equipment status is updated if it changes (up ↔ down)
5. Status changes are broadcast via PubSub to all connected LiveViews
6. The Equipment show page updates in real-time when status changes
- Sites schema with hierarchical parent-child relationships
- CRUD operations for sites
- Sites context with helper functions
- LiveView pages:
- /orgs/:slug/sites - List all sites
- /orgs/:slug/sites/new - Create new site
- /orgs/:slug/sites/:id - View site details
- /orgs/:slug/sites/:id/edit - Edit site
- Support for site locations and descriptions
- Site tree builder for hierarchy visualization
2. Equipment Management
- Equipment schema with monitoring fields
- IP address validation (IPv4 & IPv6)
- Equipment status tracking (up/down/unknown)
- Customizable check intervals per equipment
- Equipment context with CRUD operations
- LiveView pages:
- /orgs/:slug/equipment - List all equipment
- /orgs/:slug/equipment/new - Add equipment
- /orgs/:slug/equipment/:id - View equipment details
- /orgs/:slug/equipment/:id/edit - Edit equipment
- Equipment can be added from site pages
- Status badges and last checked timestamps
3. Database Schema
- sites table with self-referencing parent_site_id
- equipment table with status tracking
- All migrations run successfully
- Proper indexes on foreign keys and status
4. Features Implemented
- ✅ IP address validation using :inet.parse_address
- ✅ Site hierarchy with parent-child relationships
- ✅ Equipment linked to sites
- ✅ Monitoring enabled/disabled per equipment
- ✅ Customizable check intervals (30s - 3600s)
- ✅ Status tracking (up/down/unknown)
- ✅ Timestamps for last check and last status change
- ✅ Organization-scoped data (users only see their org's data)
