Commit graph

117 commits

Author SHA1 Message Date
5a3ede1f71 feat: config change → performance correlation system (Roadmap #2)
- New config_change_events table linking backup diffs to time windows
- ConfigChanges context with record/list functions and date filters
- Auto-detect changed sections from MikroTik config diffs
- Agent channel emits change events after backup creation
- Correlator engine: 2h before / 4h after QoE metric comparison
- Creates suspect_config_change insights on significant degradation
- Config Timeline LiveView with chart data, change list, detail view
- Device page: Config Timeline tab + Recent Config Changes card
- Dashboard: Recent Config Changes section with impact indicators
- Fix pre-existing integrations_live.html.heex scope bug
2026-02-13 17:56:55 -06:00
20f5f48372
feat: command center dashboard with unified insights and contextual enrichment
Transform the dashboard into a single-pane-of-glass command center that
blends operational metrics with business context from Gaiia subscriber data.

- Extend insight schema with multi-source support (snmp, gaiia, system)
- Add Oban workers for automated insight generation (device health, system, gaiia)
- New Dashboard context with health score computation and site summaries
- Rewrite dashboard LiveView with health overview, alert/insight feeds, site grid
- Add source filter pills for insights with URL state management
- Add metrics bar to site detail pages (device count, alerts, subscribers, MRR)
- Add subscriber/MRR context to alert list site links
- Add subscriber badges to device list site headers
- Real-time PubSub updates for alerts on dashboard
2026-02-13 14:52:49 -06:00
934ee5bb86
add gaiia outage impact analysis (stage 5)
Computes subscriber impact when device alerts fire. Three analysis
layers: device-level (inventory item -> direct subscriber) and
site-level (network site -> all subscribers at tower). Impact data
stored as JSONB on alert records and displayed as badges on alert list.
2026-02-13 11:21:59 -06:00
3cd2303b6f
add gaiia integration stages 1-2: sync engine and entity mapping UI
Stage 1 - Foundation:
- GraphQL client with Relay cursor pagination and rate limiting
- 4 Ecto schemas: accounts, network sites, inventory items, billing subscriptions
- Gaiia context with upsert operations preserving user mappings
- Sync module orchestrating full organization sync
- Oban cron worker running every 15 minutes
- Migration creating 4 tables with org-scoped unique constraints

Stage 2 - Entity Mapping:
- GaiiaMappingLive with two tabs (Network Sites / Inventory Items)
- Match suggestion engine (IP/name matching with confidence levels)
- Inline search and link/unlink UI following Preseem devices pattern
- Filter by all/mapped/unmapped with URL-driven state
- Amber suggestion badges for potential matches
2026-02-13 10:31:32 -06:00
860fb7c3b6
Merge remote-tracking branch 'origin/main' into feature/preseem-integration
# Conflicts:
#	lib/towerops_web/live/device_live/show.ex
#	lib/towerops_web/live/device_live/show.html.heex
2026-02-13 09:11:45 -06:00
9a19b9b0a5
add preseem insights schema, generation, and queries 2026-02-13 08:53:01 -06:00
df2ca42404
add preseem device baselines and fleet profiles schemas 2026-02-13 08:31:15 -06:00
5118b7eb56
add preseem subscriber metrics schema and migration 2026-02-12 17:12:53 -06:00
26e19b00f7
add preseem sync logs schema and migration 2026-02-12 17:12:19 -06:00
5f17d5aed1
add preseem access points schema and migration 2026-02-12 17:11:36 -06:00
d7741cdc0e
feat: implement unified checks system (Phase 1-4)
This commit implements the unified checks architecture that consolidates
SNMP monitoring with HTTP/TCP/DNS checks under a single "check" abstraction,
enabling consistent graphing, alerting, and management across all check types.

Database Changes:
- Add source_type and source_id to checks table for tracking auto-discovered
  vs manually created checks
- Add value field to check_results for storing numeric sensor readings
- Maintain backward compatibility with existing check_results data

New SNMP Executors:
- SnmpSensorExecutor: Poll sensor OIDs and return formatted values with
  status determination (OK/WARNING/CRITICAL based on limits)
- SnmpInterfaceExecutor: Poll interface stats (bandwidth, packets, errors)
- SnmpProcessorExecutor: Poll CPU/processor usage
- SnmpStorageExecutor: Poll disk/memory usage with percentage calculations

Check Execution Worker:
- CheckExecutorWorker: Unified Oban worker that dispatches to appropriate
  executor based on check_type (snmp_sensor, snmp_interface, http, tcp, etc.)
- Self-schedules next execution with distributed polling offsets
- Records results in check_results TimescaleDB hypertable
- Updates check state (OK/WARNING/CRITICAL/UNKNOWN)

Discovery Integration:
- Auto-creates checks during SNMP discovery for sensors, interfaces,
  processors, and storage
- Links checks to source entities via source_id for data lookup
- Enables/disables checks based on discovery results

UI Enhancements:
- Checks tab on device detail page with grouped display
- FormComponent for adding manual HTTP/TCP/DNS checks
- Empty state with "Run Discovery" prompt
- Check status badges and last checked times

Graphing:
- Update GraphLive to accept check_id parameter
- Query check_results table for time-series data
- Support all check types (SNMP, HTTP response times, etc.)

Testing:
- Comprehensive test suite for SnmpSensorExecutor (5 tests)
- Test suite for CheckExecutorWorker (7 tests)
- Test coverage for discovery check creation (6 tests)
- Remove deprecated monitoring_test.exs testing old API

Bug Fixes:
- Fix SNMP executors reading credentials from correct Device schema fields
  (device.snmp_version instead of device.snmp_device.version)
- Update agent channel test to query MonitoringCheck table directly

Code Quality:
- Extract add_snmp_credentials helper to reduce cyclomatic complexity
- Use map-based lookups for sensor formatting and check type grouping
- Apply pattern matching in dispatcher to reduce complexity
- All credo checks passing with no issues

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-12 16:58:40 -06:00
39f0ffe253
add integrations schema with encrypted credentials 2026-02-12 16:54:14 -06:00
291ecd1054
feat: add full-featured monitoring platform (HTTP/TCP/DNS checks)
Implements Icinga2-style generic check system for comprehensive monitoring.

Phoenix Backend:
- Generic checks table with JSONB config for all check types
- TimescaleDB check_results hypertable for time-series data
- Check executors (HTTP, TCP, DNS)
- CheckWorker with self-scheduling and state transitions
- Extended alerts system for check-based alerting

Check Types: HTTP/HTTPS, TCP, DNS
Architecture: Icinga2 Checkable pattern, soft/hard states, flapping detection

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-12 14:05:31 -06:00
00c366ac9a
feat: add LLDP, CDP, MAC, and ARP evidence collectors
LLDP/CDP evidence (0.95 confidence) from snmp_neighbors.
MAC evidence (0.7) cross-references MAC tables with device
interface MACs. ARP evidence (0.6) matches by IP and MAC.
Self-links excluded. Unmatched entries tracked as discovered.
2026-02-12 12:55:39 -06:00
3abcbebdd9
feat: add topology inference database tables
Add device_role/device_role_source to devices table.
Create device_links and device_link_evidence tables for
persistent topology model.
2026-02-12 12:45:36 -06:00
aa1bac0d46
perf: batch inserts, selective LiveView reloading, and database indexes for scale
Replace individual Repo.insert() with Repo.insert_all() for sensor readings,
interface stats, processor readings, and storage readings in agent channel and
device poller worker. Add partial/composite database indexes for common query
patterns. Dashboard now uses GROUP BY aggregation instead of loading all devices.
DeviceLive.Show only reloads data for the active tab on PubSub events.
DeviceLive.Index debounces status changes and skips quota queries on updates.
Agent polling preloads filtered to monitored sensors/interfaces only.
2026-02-12 10:16:10 -06:00
4d73e77f3a
Add 404-based brute force protection system
Implements automated detection and blocking of IPs exhibiting scanning behavior (5+ unique 404s within 1 minute).

Key features:
- Progressive ban escalation (5 min → 1 hour → permanent)
- CIDR range and exact IP whitelisting
- Redis-backed 404 path tracking with 60s TTL
- Cloudflare WAF integration for permanent bans
- Admin UI for whitelist and blocked IP management
- Oban cron jobs for cleanup (expired bans, stale violations)
- ETS caching for whitelist performance

Components:
- Plug: ToweropsWeb.Plugs.BruteForceProtection
- Context: Towerops.Security.BruteForce
- Schemas: IpBlock, IpWhitelist
- Workers: CloudflareBanWorker, ExpiredBanCleanupWorker, StaleViolationCleanupWorker
- Admin UI: /admin/security (superuser only)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-10 16:31:48 -06:00
623fe07081
wip 2026-02-09 12:44:37 -06:00
7a57f7c9d2
fix: resolve critical sensor discovery pipeline data loss
Critical fixes to sensor discovery and display pipeline:

- Fix float value rejection: Change is_integer guards to is_number in
  vendor.ex, dynamic.ex to accept decimal sensor values (temperature,
  counters, etc.)
- Fix scale factor calculation: Rewrite base.ex divisor calculation
  using pure integer arithmetic to prevent rounding errors
- Add missing sensor type support: Current, power, fan, load, and
  signal quality sensor categories now filtered and assigned in UI
- Refactor resolve_snmp_community to reduce cyclomatic complexity

Impact:
- Sensors with float values (DHCP leases, connection counts, precise
  temperatures) no longer silently dropped before database
- ENTITY-SENSOR-MIB divisors calculated correctly without float errors
- UI coverage increased from ~35% to ~60% of discovered sensor types

Files modified:
- lib/towerops/snmp/profiles/vendors/vendor.ex
- lib/towerops/snmp/profiles/dynamic.ex
- lib/towerops/snmp/profiles/base.ex
- lib/towerops_web/live/device_live/show.ex
- lib/towerops/devices.ex

Documentation: SENSOR_PIPELINE_FIXES.md
2026-02-09 08:32:24 -06:00
dbb2412e89
tcp snmp test 2026-02-04 17:28:14 -06:00
fc66109f19
Add default organization feature
Features:
- Users can set a default organization in org settings
- First organization created is automatically set as default
- Default org selector only shown if user has 2+ organizations
- Migration auto-sets first org as default for existing users

Database changes:
- Added is_default boolean field to organization_memberships
- Added partial index for efficient default org lookups
- Migration sets oldest org as default for each existing user

UI changes:
- Organization settings page shows default org toggle
- Green badge displayed when org is already default
- Button to set org as default if it's not current default

Backend:
- Organizations.set_default_organization/2 manages defaults
- Organizations.get_default_membership/1 retrieves default
- Auto-set first org as default in create_organization/3
2026-02-04 16:09:43 -06:00
8382b5df6a
make devices site optional and many test fix 2026-02-04 15:05:42 -06:00
1a054fd598
make sites optional 2026-02-04 13:05:32 -06:00
c93144cc37
snmp v3 support 2026-02-04 12:02:38 -06:00
c198fdeaa1
add snmp port 2026-02-03 12:51:50 -06:00
5b84844e97
doc update 2026-02-03 09:07:06 -06:00
e181c17f78
fix: resolve MikroTik sensor values and empty SNMP communities
Fixes three critical issues affecting agent-based SNMP polling:

1. MikroTik gauge sensors incorrectly showing 10x values (223°C instead of 22.3°C)
   - Root cause: get_int_value() rejected zero values, causing mtxrGaugeUnit
     to be treated as nil and bypassing divisor logic
   - Fixed: Accept zero values and apply correct divisor=10 for temp/voltage/current/power

2. Empty SNMP community strings sent to agents causing authentication failures
   - Root cause: Devices with source="device" but null community weren't falling
     back to organization/site inheritance
   - Fixed: Enhanced resolve_snmp_community() to fall back to inheritance chain
     even when source="device" but community is empty

3. Continuous MikroTik API commands sent every 60s instead of once per 24h
   - Root cause: MikroTik jobs built on every poll cycle, not just discovery
   - Fixed: Only build MikroTik jobs during discovery phase

Migrations update existing data to fix incorrect divisor values and community
source tracking. Added comprehensive debug logging to diagnose future issues.
2026-02-02 17:12:36 -06:00
aaf8c38b78
handle mikrotik ssh 2026-02-02 16:42:18 -06:00
949cfaf4e0
add default org 2026-02-02 10:45:37 -06:00
f938b263cd
Mikrotik backup and gettext start 2026-02-02 09:11:22 -06:00
23af86ba73
add mikrotik device handling 2026-02-01 16:42:39 -06:00
a66aec675e Merge branch 'gmcintire/user-first-last-name' into 'main'
Replace single name field with first_name and last_name

See merge request towerops/towerops!3
2026-02-01 15:00:24 -06:00
e46ecbfca2 Replace single name field with first_name and last_name 2026-02-01 15:00:23 -06:00
f0d6ae42d0
feat: update UserAuth to redirect to sudo verify page 2026-02-01 14:58:07 -06:00
ada851e7ed
feat: add Accounts.grant_sudo_mode/1 function 2026-02-01 14:02:41 -06:00
004189aaf9
Add firmware tracking database schema
- Create firmware_releases table to store latest available firmware versions
- Create device_firmware_history table to track version changes over time
- Add FirmwareRelease and DeviceFirmwareHistory Ecto schemas with validations
- Add comprehensive tests for both schemas (23 tests, 100% coverage)

firmware_releases stores one record per vendor/product_line with latest version.
device_firmware_history is append-only audit log of device firmware changes.

Part of firmware version tracking system for MikroTik (extensible to other vendors).
2026-02-01 10:13:16 -06:00
2fab08f5f8
forgot password flow 2026-01-31 17:03:22 -06:00
dff9c26905
fix TOTP enrollment with recovery codes 2026-01-31 14:54:44 -06:00
12a777d13d fix routeros temperature divisor 2026-01-31 12:19:53 -06:00
3c00dcf37c remove passkeys for now 2026-01-31 09:50:35 -06:00
22ae257b60 refactor 2026-01-31 09:35:07 -06:00
6ca22c5dd0 enhance user auditing 2026-01-30 17:38:07 -06:00
55e9397d59 user session tracking 2026-01-29 14:14:21 -06:00
c0736d4c2e gdpr consent tracking 2026-01-29 11:12:35 -06:00
0995abbccb Add mandatory TOTP MFA 2026-01-28 15:09:38 -06:00
234ff7cbd2 add upsert support for geoip imports to prevent duplicates 2026-01-28 13:37:02 -06:00
d1fd1d4a62 remove Mix.env and support ip database import 2026-01-28 13:08:06 -06:00
220f1edce3 cookie improvements 2026-01-28 12:30:28 -06:00
d34c72a0ab track more snmp things 2026-01-28 09:18:31 -06:00
5fb92dd961
add device/org limit 2026-01-27 14:14:25 -06:00