Commit graph

312 commits

Author SHA1 Message Date
520f7b0c54 Add subscriber-to-AP matching via ARP + Gaiia integration
Matches Gaiia subscriber accounts to TowerOps devices using:
- ARP table IP matching (high confidence)
- ARP table MAC matching (high confidence)
- Site CIDR fallback (medium confidence)

Enables subscriber impact display on alerts and device pages.
2026-02-16 11:07:20 -06:00
2735c1a032 Add onboarding flow for new organizations 2026-02-16 10:14:45 -06:00
5b786aac08 Add maintenance windows and weather overlay (backend)
Maintenance Windows:
- maintenance_windows table with org/site/device scoping
- Full CRUD context (Towerops.Maintenance)
- device_in_maintenance?/1 checks device → site → org hierarchy
- Alert suppression in DeviceMonitorWorker (skips alerts during maintenance)
- maintenance_badge/1 helper for UI display

Weather Overlay:
- weather_observations + weather_alerts tables
- OpenWeatherMap free API client (lat/lon based)
- WeatherSyncWorker (Oban, 15min interval, respects rate limits)
- Observation parsing from OWM response format
- Correlation helpers: severe_weather_near?/2, weather_at/2
- Severity classification tuned for WISP ops (wind, rain, ice thresholds)
- Per-org latest observations query for dashboard
- 30-day automatic cleanup

Both features are backend-only — no UI yet.
2026-02-16 10:01:19 -06:00
b4c0407ee0 Add granular org roles: executive, technician
New role system:
- owner: full access + financials
- admin: full access + financials (except org deletion)
- executive: read-only + full financial visibility (MRR, revenue)
- technician: field ops (devices, sites, alerts) without financials
- member: legacy alias for technician (migrated)
- viewer: read-only, no financials

Financial data (MRR, revenue, billing) gated behind can_view_financials
assign in all LiveView templates: dashboard, alerts, trace, site show.

Includes migration to rename existing member roles to technician.
2026-02-15 17:40:53 -06:00
de075ea8d9
Store billing subscriber/MRR data on integrations for dashboard
Each billing sync (Gaiia, Splynx, VISP, Sonar) already computes
subscriber counts and MRR but only stored them in sync messages.
Now persists them on the integrations table so the dashboard can
aggregate across all billing providers instead of only querying
Gaiia network sites (which were never populated).

Also fixes SnmpKit Config environment detection to use Mix.env()
as fallback when MIX_ENV env var is not set.
2026-02-15 16:51:33 -06:00
c790794191 Fix tests, credo issues, and Gaiia sync bugs
- Fix CLOAK_KEY placeholder in nix shell (invalid base64)
- Fix error HTML test assertion to match actual template
- Fix Gaiia sync: read 'subnet' field instead of 'block' for IP blocks
- Fix Gaiia sync: extract nested status name from subscription objects
- Fix security headers tests for environment detection
- Fix mobile auth tests to use raw_token
- Fix LiveView test assertions to match actual template content
- Fix SNMP config test expectations for test environment
- Refactor: resolve all Credo complexity/nesting issues
  - Extract helper functions in NetBox sync, VISP sync, Sonar sync
  - Flatten nested conditionals in settings, integrations, alerts
  - Use with/validate_present pattern for connection testing
2026-02-15 11:55:49 -06:00
669dec5d89 proto: sync agent.proto with towerops-agent
- Add transport field to SnmpDevice and SnmpConfig
- Add arch field to AgentHeartbeat (already in .pb.ex)
- Add Check, CheckResult, CheckList messages for service checks (HTTP/TCP/DNS)
- Add checks field to AgentConfig
- Add CheckResult to Metric oneof
- Sync proto source with compiled .pb.ex
2026-02-15 08:56:48 -06:00
75aece28a7 i18n: complete Spanish translations for all gettext domains 2026-02-14 17:45:19 -06:00
8752dfec49 fix: netbox url field type, gaiia ipRange→block, remove unknown webhook log 2026-02-14 17:44:01 -06:00
a3730db579 i18n: add Spanish locale with full translations, language selector, and locale hook 2026-02-14 17:44:01 -06:00
29c2039832 Auto-resolve device_up recovery alerts on creation
device_up alerts are informational recovery events, not ongoing issues.
Previously they were created without resolved_at, inflating the
unresolved alert count. Now they're born resolved.

Includes a data migration to fix existing stale device_up alerts.
2026-02-14 14:53:17 -06:00
61f803afe5 Add language user setting (default: English) 2026-02-14 14:53:17 -06:00
db40a3e5b7 feat: show human-readable sync status messages for integrations
- Added last_sync_message field to integrations schema
- Gaiia sync: shows item counts on success, friendly error on failure
- Preseem sync: shows AP/device counts on success, friendly error on failure
- Error messages translated from technical errors to user-friendly text
- Message displayed on integration card next to status badge
- Exposed in REST API response
2026-02-14 14:11:02 -06:00
5df5c97c45 feat: add geographic location to sites with geocoding, address fields, and Leaflet.js network map 2026-02-14 13:07:14 -06:00
c37ae76ceb fix: device index uses device_role instead of nonexistent device_type, last_checked_at instead of last_seen_at; make TimescaleDB migrations conditional for dev/test 2026-02-14 10:13:03 -06:00
9e7ee5099d refactor: fix all credo strict issues, format all code, fix broken routes
- Fix broken route paths in dashboard (/discovery, /subscribers/trace, /config-changes)
- Fix insights empty state settings link (org-scoped route)
- Add underscores to all 86400 literals across 6 files
- Alphabetize aliases in search.ex and agent_channel.ex
- Extract changelog parser helper to reduce nesting
- Extract dashboard impact calculation to reduce nesting
- Refactor agent_channel config change detection (pattern match, extract function)
- Combine double Enum.reject into single call in trace.ex
- Fix line length in trace.ex search query
- Replace length/1 > 0 with != [] in trace_live
- Run mix format on all files
2026-02-13 19:34:40 -06:00
b07c322dc9 docs: update changelogs with navigation and visual polish 2026-02-13 19:18:36 -06:00
5a6643c6aa docs: update changelogs with UX polish and activity feed improvements 2026-02-13 19:13:32 -06:00
dec6f0f271 feat: rewrite marketing landing page for WISP operators
- Pain point section: disconnected tools, slow impact assessment
- WISP-specific features: impact dashboard, subscriber trace, alert triage, config tracking
- Pricing tiers: Starter (free/10 devices), Pro (/dev/mo), Business (/dev/mo)
- Use case sections: NOC, Help Desk, Owner perspectives
- Tech details: 786 device profiles, secure agent arch, SNMP v1/v2c/v3
- Updated both changelogs
2026-02-13 19:07:04 -06:00
5fb585d4fd docs: update changelogs with all new features 2026-02-13 18:59:25 -06:00
9c266e57ff feat: wire up Trace and Activity Feed routes + nav 2026-02-13 18:50:28 -06:00
c512f30e5a feat: add device list health indicators
- Health status dot (green/yellow/red/gray) from worst-case check state
- Relative 'last seen' with color-coded staleness thresholds
- Response time badge from latest monitoring check
- Subscriber count badge from Gaiia inventory items
- Batch queries to avoid N+1: Monitoring.get_device_health_summary/1,
  Monitoring.get_device_latest_response_times/1,
  Gaiia.get_device_subscriber_counts/1
2026-02-13 18:14:29 -06:00
5a3ede1f71 feat: config change → performance correlation system (Roadmap #2)
- New config_change_events table linking backup diffs to time windows
- ConfigChanges context with record/list functions and date filters
- Auto-detect changed sections from MikroTik config diffs
- Agent channel emits change events after backup creation
- Correlator engine: 2h before / 4h after QoE metric comparison
- Creates suspect_config_change insights on significant degradation
- Config Timeline LiveView with chart data, change list, detail view
- Device page: Config Timeline tab + Recent Config Changes card
- Dashboard: Recent Config Changes section with impact indicators
- Fix pre-existing integrations_live.html.heex scope bug
2026-02-13 17:56:55 -06:00
02f837ca5b
fix: replace em-dashes with hyphens in changelog to avoid encoding issues 2026-02-13 17:48:07 -06:00
314d93f3ab
docs: update changelogs for impact dashboard and integration nav improvements 2026-02-13 16:45:45 -06:00
872bef3512
feat: add 12/24-hour time format setting with consistent timezone display
wire time_format into scope, add settings UI selector, change defaults
to 24h, and replace all user-facing Calendar.strftime calls with
centralized TimeHelpers using the user's timezone and time format.
2026-02-13 15:16:29 -06:00
a6d5563b41
docs: update changelogs for command center dashboard feature 2026-02-13 14:56:01 -06:00
20f5f48372
feat: command center dashboard with unified insights and contextual enrichment
Transform the dashboard into a single-pane-of-glass command center that
blends operational metrics with business context from Gaiia subscriber data.

- Extend insight schema with multi-source support (snmp, gaiia, system)
- Add Oban workers for automated insight generation (device health, system, gaiia)
- New Dashboard context with health score computation and site summaries
- Rewrite dashboard LiveView with health overview, alert/insight feeds, site grid
- Add source filter pills for insights with URL state management
- Add metrics bar to site detail pages (device count, alerts, subscribers, MRR)
- Add subscriber/MRR context to alert list site links
- Add subscriber badges to device list site headers
- Real-time PubSub updates for alerts on dashboard
2026-02-13 14:52:49 -06:00
ee277a457e
update changelogs for gaiia integration 2026-02-13 13:29:54 -06:00
934ee5bb86
add gaiia outage impact analysis (stage 5)
Computes subscriber impact when device alerts fire. Three analysis
layers: device-level (inventory item -> direct subscriber) and
site-level (network site -> all subscribers at tower). Impact data
stored as JSONB on alert records and displayed as badges on alert list.
2026-02-13 11:21:59 -06:00
3cd2303b6f
add gaiia integration stages 1-2: sync engine and entity mapping UI
Stage 1 - Foundation:
- GraphQL client with Relay cursor pagination and rate limiting
- 4 Ecto schemas: accounts, network sites, inventory items, billing subscriptions
- Gaiia context with upsert operations preserving user mappings
- Sync module orchestrating full organization sync
- Oban cron worker running every 15 minutes
- Migration creating 4 tables with org-scoped unique constraints

Stage 2 - Entity Mapping:
- GaiiaMappingLive with two tabs (Network Sites / Inventory Items)
- Match suggestion engine (IP/name matching with confidence levels)
- Inline search and link/unlink UI following Preseem devices pattern
- Filter by all/mapped/unmapped with URL-driven state
- Amber suggestion badges for potential matches
2026-02-13 10:31:32 -06:00
860fb7c3b6
Merge remote-tracking branch 'origin/main' into feature/preseem-integration
# Conflicts:
#	lib/towerops_web/live/device_live/show.ex
#	lib/towerops_web/live/device_live/show.html.heex
2026-02-13 09:11:45 -06:00
9a19b9b0a5
add preseem insights schema, generation, and queries 2026-02-13 08:53:01 -06:00
df2ca42404
add preseem device baselines and fleet profiles schemas 2026-02-13 08:31:15 -06:00
4174987a88 feat: complete DevicePollerWorker to CheckExecutorWorker migration
Completes full migration from device-level polling to per-check polling.

Migration Changes:
- Removed DevicePollerWorker scheduling from device lifecycle
- Checks automatically scheduled when created
- Discovery creates and schedules checks atomically

New Functions:
- Monitoring.stop_device_checks/1: Cancel all check jobs for device
- Monitoring.disable_device_checks/1: Disable checks when SNMP off

Discovery Integration:
- All create_*_check functions now schedule checks after creation

JobHealthCheckWorker Updates:
- Removed DevicePollerWorker recovery logic
- Only recovers DeviceMonitorWorker jobs now

Test Updates:
- Marked DevicePollerWorkerTest as skipped (deprecated)
- Updated JobHealthCheckWorkerTest expectations

All 6953 tests passing, 0 failures

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-12 17:52:33 -06:00
5118b7eb56
add preseem subscriber metrics schema and migration 2026-02-12 17:12:53 -06:00
26e19b00f7
add preseem sync logs schema and migration 2026-02-12 17:12:19 -06:00
5f17d5aed1
add preseem access points schema and migration 2026-02-12 17:11:36 -06:00
75d64d8772
feat: add check creation from discovery and backfill tool (Phase 5)
Completes Phase 5 of the unified checks system by enabling automatic check
creation during SNMP discovery and providing a backfill tool for existing devices.

Discovery Integration:
- create_checks_from_discovery/2: Creates checks for all discovered entities
  (sensors, interfaces, processors, storage) after SNMP discovery completes
- Links each check to its source entity via source_id for data lookup
- Returns detailed results with counts per type and error tracking
- Logs creation results with total counts and any failures

Helper Functions:
- create_sensor_check/2: Creates snmp_sensor checks with config
- create_interface_check/2: Creates snmp_interface checks
- create_processor_check/2: Creates snmp_processor checks
- create_storage_check/2: Creates snmp_storage checks

All checks configured with:
- 60-second poll intervals (default)
- Auto-discovery source type
- Enabled by default
- Type-specific configuration in JSONB config field

Backfill Tool:
- mix backfill.checks: Creates checks for existing SNMP-enabled devices
- Supports --dry-run mode for preview
- Shows counts per device (sensors, interfaces, processors, storage)
- Reports success/failure counts and detailed errors
- Safely handles devices without discovery data
- Reduced nesting depth using cond and helper functions for credo compliance

This enables the unified monitoring system to work with both new discoveries
and existing devices, ensuring all SNMP data can be monitored through the
unified checks interface.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-12 17:04:06 -06:00
d7741cdc0e
feat: implement unified checks system (Phase 1-4)
This commit implements the unified checks architecture that consolidates
SNMP monitoring with HTTP/TCP/DNS checks under a single "check" abstraction,
enabling consistent graphing, alerting, and management across all check types.

Database Changes:
- Add source_type and source_id to checks table for tracking auto-discovered
  vs manually created checks
- Add value field to check_results for storing numeric sensor readings
- Maintain backward compatibility with existing check_results data

New SNMP Executors:
- SnmpSensorExecutor: Poll sensor OIDs and return formatted values with
  status determination (OK/WARNING/CRITICAL based on limits)
- SnmpInterfaceExecutor: Poll interface stats (bandwidth, packets, errors)
- SnmpProcessorExecutor: Poll CPU/processor usage
- SnmpStorageExecutor: Poll disk/memory usage with percentage calculations

Check Execution Worker:
- CheckExecutorWorker: Unified Oban worker that dispatches to appropriate
  executor based on check_type (snmp_sensor, snmp_interface, http, tcp, etc.)
- Self-schedules next execution with distributed polling offsets
- Records results in check_results TimescaleDB hypertable
- Updates check state (OK/WARNING/CRITICAL/UNKNOWN)

Discovery Integration:
- Auto-creates checks during SNMP discovery for sensors, interfaces,
  processors, and storage
- Links checks to source entities via source_id for data lookup
- Enables/disables checks based on discovery results

UI Enhancements:
- Checks tab on device detail page with grouped display
- FormComponent for adding manual HTTP/TCP/DNS checks
- Empty state with "Run Discovery" prompt
- Check status badges and last checked times

Graphing:
- Update GraphLive to accept check_id parameter
- Query check_results table for time-series data
- Support all check types (SNMP, HTTP response times, etc.)

Testing:
- Comprehensive test suite for SnmpSensorExecutor (5 tests)
- Test suite for CheckExecutorWorker (7 tests)
- Test coverage for discovery check creation (6 tests)
- Remove deprecated monitoring_test.exs testing old API

Bug Fixes:
- Fix SNMP executors reading credentials from correct Device schema fields
  (device.snmp_version instead of device.snmp_device.version)
- Update agent channel test to query MonitoringCheck table directly

Code Quality:
- Extract add_snmp_credentials helper to reduce cyclomatic complexity
- Use map-based lookups for sensor formatting and check type grouping
- Apply pattern matching in dispatcher to reduce complexity
- All credo checks passing with no issues

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-12 16:58:40 -06:00
39f0ffe253
add integrations schema with encrypted credentials 2026-02-12 16:54:14 -06:00
291ecd1054
feat: add full-featured monitoring platform (HTTP/TCP/DNS checks)
Implements Icinga2-style generic check system for comprehensive monitoring.

Phoenix Backend:
- Generic checks table with JSONB config for all check types
- TimescaleDB check_results hypertable for time-series data
- Check executors (HTTP, TCP, DNS)
- CheckWorker with self-scheduling and state transitions
- Extended alerts system for check-based alerting

Check Types: HTTP/HTTPS, TCP, DNS
Architecture: Icinga2 Checkable pattern, soft/hard states, flapping detection

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-12 14:05:31 -06:00
00c366ac9a
feat: add LLDP, CDP, MAC, and ARP evidence collectors
LLDP/CDP evidence (0.95 confidence) from snmp_neighbors.
MAC evidence (0.7) cross-references MAC tables with device
interface MACs. ARP evidence (0.6) matches by IP and MAC.
Self-links excluded. Unmatched entries tracked as discovered.
2026-02-12 12:55:39 -06:00
c6b271d204
docs: update changelogs for BruteForceProtection fix 2026-02-12 12:54:25 -06:00
cd6f2153ce
fix: reload device stream when switching back to existing devices tab 2026-02-12 12:47:30 -06:00
3abcbebdd9
feat: add topology inference database tables
Add device_role/device_role_source to devices table.
Create device_links and device_link_evidence tables for
persistent topology model.
2026-02-12 12:45:36 -06:00
4e58bf24dd
docs: update changelogs for graceful shutdown handling 2026-02-12 12:18:15 -06:00
19ceae0b4e
feat: Phase 1 Quick Wins - 11 vendor profiles implemented
Completed Phase 1 of comprehensive coverage analysis, adding support for 11
vendor profiles across 6 vendors in ~24 hours of implementation time.

**New Vendor Modules:**
- VyOS (vyos.ex) - VyOS and Vyatta router support
- Edgeswitch (edgeswitch.ex) - Ubiquiti EdgeSwitch/EdgePoint/USW series
- Omnitron (omnitron.ex) - Omnitron iConverter optical networking

**Enhanced Vendor Modules:**
- Adva (adva.ex) - Added adva-alm profile, now 7 total profiles
- Unifi (unifi.ex) - Added unifi-usp (SmartPower) profile
- APC (apc.ex) - Added aos-emu2 (Environmental Monitoring Unit) profile
- HP (hp.ex) - Added comware and procurve profiles, now 11 total
- Dell (dell.ex) - Added powerconnect profile
- Powervault (powervault.ex) - Added powervault profile variant

**New YAML Profiles:**
- adva-fsp150cp.yaml - ADVA FSP150CP optical transport
- Plus corresponding os_detection profiles for all new vendors

**Updated YAML Profiles:**
- comware.yaml, procurve.yaml - Comprehensive HP legacy switch support
- powerconnect.yaml, powervault.yaml - Dell legacy product support
- adva-alm.yaml - Enhanced Adva optical support

**Test Coverage:**
- 54 new tests across all new modules
- All 6857 tests passing
- Comprehensive coverage for each vendor module

**Profiles Added (11 total):**
1. vyos, vyatta (VyOS routers)
2. adva-alm (Adva optical)
3. edgeswitch, unifi-usp (Ubiquiti)
4. comware, procurve (HP legacy switches)
5. powerconnect, powervault (Dell legacy)
6. aos-emu2 (APC environmental)
7. omnitron-iconverter (Omnitron optical)

**Impact:**
- Closes 11-profile gap from coverage analysis
- Improves enterprise switch coverage (HP/Dell legacy)
- Enhances optical networking support (Adva, Omnitron)
- Strengthens Ubiquiti portfolio (EdgeSwitch, USP)
- Adds network router support (VyOS)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-12 10:52:51 -06:00
f44510bf47
docs: update user-facing changelog and document changelog maintenance
Updated priv/static/changelog.txt with all recent changes from CHANGELOG.txt
in concise, user-facing format. Enhanced CLAUDE.md to document the dual
changelog system (technical CHANGELOG.txt vs user-facing priv/static/changelog.txt)
with clear guidelines on what to include in each and when to update them.
2026-02-12 10:38:27 -06:00
aa1bac0d46
perf: batch inserts, selective LiveView reloading, and database indexes for scale
Replace individual Repo.insert() with Repo.insert_all() for sensor readings,
interface stats, processor readings, and storage readings in agent channel and
device poller worker. Add partial/composite database indexes for common query
patterns. Dashboard now uses GROUP BY aggregation instead of loading all devices.
DeviceLive.Show only reloads data for the active tab on PubSub events.
DeviceLive.Index debounces status changes and skips quota queries on updates.
Agent polling preloads filtered to monitored sensors/interfaces only.
2026-02-12 10:16:10 -06:00