Commit graph

363 commits

Author SHA1 Message Date
f326eb5dd4 feat: require email verification before first login
- Remove auto-confirmation on registration (register_user and register_user_with_organization)
- Add deliver_user_confirmation_instructions/2 and confirm_user/1 to Accounts context
- Add verify_email_token_query/2 and by_user_and_contexts_query/2 to UserToken
- Make deliver_confirmation_email public in UserNotifier
- Send confirmation email after registration in both flows
- Block unconfirmed users at password login with helpful message
- Add UserConfirmationController with confirm/resend routes
- Add resend confirmation link to login page
- Update test fixtures to confirm users after registration
2026-02-14 11:28:57 -06:00
6d8c3f932b test: add stream_data property tests for org settings and integrations
- Organization changeset: name length, SNMP port range, version enum, SNMPv3 fields
- Integration changeset: sync interval validation, provider enum
- Settings LiveView: tab routing with garbage params, arbitrary name/port/community input
- All capped at max_runs: 10-50 for fast execution
2026-02-14 11:28:57 -06:00
dbed25b366 test: add stream_data property-based tests for org settings 2026-02-14 11:28:57 -06:00
8d00a04187
feat: add FluxCD image automation, fix Preseem duplicate AP crash
Switch deployment pipeline from GitLab Agent kubectl to FluxCD image
automation. CI now only builds and pushes tagged images to Forgejo
registry; FluxCD detects new tags and updates the deployment manifest.

Fix Ecto.MultipleResultsError on /dashboard when a device has multiple
Preseem access point records by using limit(1) instead of Repo.get_by.
2026-02-14 10:30:12 -06:00
1dbb8926ed
fix: update remaining gaiia graphql status fields for object schema
Gaiia migrated all status fields from scalar enums to object types,
not just AccountStatus. Update BillingSubscription.status,
InventoryItem.status, and nested billingSubscriptions.status in the
accounts query to request { id name } subfields and extract the name
value in sync mappers.
2026-02-13 16:16:00 -06:00
d603a12322
feat: add per-organization Gaiia webhook setup UI
Auto-generate HMAC-SHA256 webhook secret when enabling the Gaiia
integration. Display webhook URL, secret with copy buttons, regenerate
option, and setup instructions on the integrations page.
2026-02-13 16:15:38 -06:00
20ec04379d
fix: update gaiia graphql queries for account status/type schema change
Gaiia API changed AccountStatus and AccountType from scalar enums to
objects with id/name/type subfields. Updated the accounts query to
request nested fields and the sync mapper to extract the name values.
2026-02-13 15:20:06 -06:00
872bef3512
feat: add 12/24-hour time format setting with consistent timezone display
wire time_format into scope, add settings UI selector, change defaults
to 24h, and replace all user-facing Calendar.strftime calls with
centralized TimeHelpers using the user's timezone and time format.
2026-02-13 15:16:29 -06:00
20f5f48372
feat: command center dashboard with unified insights and contextual enrichment
Transform the dashboard into a single-pane-of-glass command center that
blends operational metrics with business context from Gaiia subscriber data.

- Extend insight schema with multi-source support (snmp, gaiia, system)
- Add Oban workers for automated insight generation (device health, system, gaiia)
- New Dashboard context with health score computation and site summaries
- Rewrite dashboard LiveView with health overview, alert/insight feeds, site grid
- Add source filter pills for insights with URL state management
- Add metrics bar to site detail pages (device count, alerts, subscribers, MRR)
- Add subscriber/MRR context to alert list site links
- Add subscriber badges to device list site headers
- Real-time PubSub updates for alerts on dashboard
2026-02-13 14:52:49 -06:00
848efa8fc4
fix: align gaiia graphql queries with actual api schema
field mappings from docs:
- accountStatus/accountType -> status/type
- accountBillingSubscriptions -> billingSubscriptions
- billingSubscriptionStatus -> status
- address city/state/zip -> locality/region/postalCode
- networkSiteIpBlocks -> ipBlocks
- inventoryItemStatus -> status, ipAddress -> ipAddressV4
- inventoryModel -> model, inventoryModelCategory -> model.category
- account/networkSite -> assignation.assignee (union type)
- product/recurringAmount/currency -> productVersion (price in cents)
- account on subscription -> entity (union type)
2026-02-13 14:41:14 -06:00
c01f638d3a
fix: use valid graphql query for gaiia connection test
the viewer query doesn't exist in gaiia's schema, causing a 400
validation error. use a minimal accounts query instead. also handle
400 responses with graphql errors properly.
2026-02-13 13:44:58 -06:00
55382a7ec6
add gaiia inventory reconciliation (stage 7) 2026-02-13 13:28:52 -06:00
6da05d461b
add gaiia write-back actions (stage 6)
Three write-back capabilities: ticket creation from alerts, subscriber
notes during outages, and inventory item updates. Adds GraphQL mutation
methods to the client and an Actions module that handles credential
retrieval and API call orchestration.
2026-02-13 11:25:21 -06:00
934ee5bb86
add gaiia outage impact analysis (stage 5)
Computes subscriber impact when device alerts fire. Three analysis
layers: device-level (inventory item -> direct subscriber) and
site-level (network site -> all subscribers at tower). Impact data
stored as JSONB on alert records and displayed as badges on alert list.
2026-02-13 11:21:59 -06:00
2e9f1660d0
stagger preseem sync across organizations
Replace the thundering-herd cron job with a dispatcher that enqueues
individual per-integration Oban jobs staggered across the 10-minute
window using PollingOffset. Show last synced timestamp in user's
local timezone via the <.timestamp> component.
2026-02-13 10:52:32 -06:00
bad1590fed
add gaiia webhook listener with per-org signature verification
Stage 3 of Gaiia integration: webhook event processing for real-time
incremental updates from Gaiia. HMAC-SHA256 signature verification
using per-organization secrets stored in integration credentials.
Handles account, billing subscription, and inventory item events.
2026-02-13 10:40:49 -06:00
3cd2303b6f
add gaiia integration stages 1-2: sync engine and entity mapping UI
Stage 1 - Foundation:
- GraphQL client with Relay cursor pagination and rate limiting
- 4 Ecto schemas: accounts, network sites, inventory items, billing subscriptions
- Gaiia context with upsert operations preserving user mappings
- Sync module orchestrating full organization sync
- Oban cron worker running every 15 minutes
- Migration creating 4 tables with org-scoped unique constraints

Stage 2 - Entity Mapping:
- GaiiaMappingLive with two tabs (Network Sites / Inventory Items)
- Match suggestion engine (IP/name matching with confidence levels)
- Inline search and link/unlink UI following Preseem devices pattern
- Filter by all/mapped/unmapped with URL-driven state
- Amber suggestion badges for potential matches
2026-02-13 10:31:32 -06:00
21bbcccc30
fix preseem api base url and remove base_url override
the default base url was pointing at the docs site
(apidocs.preseem.com) instead of the actual api endpoint
(api.preseem.com), causing all api calls to fail with
unauthorized. removed the unnecessary base_url user override
since the api endpoint is fixed.
2026-02-13 10:12:43 -06:00
b3f8204047
test fix and preseem check fix 2026-02-13 09:42:39 -06:00
860fb7c3b6
Merge remote-tracking branch 'origin/main' into feature/preseem-integration
# Conflicts:
#	lib/towerops_web/live/device_live/show.ex
#	lib/towerops_web/live/device_live/show.html.heex
2026-02-13 09:11:45 -06:00
9ccea8daaf
add network insights feed page with filters and bulk dismiss 2026-02-13 09:00:34 -06:00
9a19b9b0a5
add preseem insights schema, generation, and queries 2026-02-13 08:53:01 -06:00
6a827a0ed6
add preseem baseline and fleet intelligence computation 2026-02-13 08:40:04 -06:00
df2ca42404
add preseem device baselines and fleet profiles schemas 2026-02-13 08:31:15 -06:00
b9db1d2224
add preseem devices management page with manual linking 2026-02-12 18:12:27 -06:00
59cf53144f
add preseem context and device detail preseem tab 2026-02-12 18:05:26 -06:00
b04d25e607
add preseem sync worker oban cron job 2026-02-12 17:56:52 -06:00
78040c0df7 add device matcher for preseem AP to towerops device linking 2026-02-12 17:52:54 -06:00
4174987a88 feat: complete DevicePollerWorker to CheckExecutorWorker migration
Completes full migration from device-level polling to per-check polling.

Migration Changes:
- Removed DevicePollerWorker scheduling from device lifecycle
- Checks automatically scheduled when created
- Discovery creates and schedules checks atomically

New Functions:
- Monitoring.stop_device_checks/1: Cancel all check jobs for device
- Monitoring.disable_device_checks/1: Disable checks when SNMP off

Discovery Integration:
- All create_*_check functions now schedule checks after creation

JobHealthCheckWorker Updates:
- Removed DevicePollerWorker recovery logic
- Only recovers DeviceMonitorWorker jobs now

Test Updates:
- Marked DevicePollerWorkerTest as skipped (deprecated)
- Updated JobHealthCheckWorkerTest expectations

All 6953 tests passing, 0 failures

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-12 17:52:33 -06:00
bf03a5a9a1 add preseem sync logic for AP upsert and metrics 2026-02-12 17:52:26 -06:00
5118b7eb56
add preseem subscriber metrics schema and migration 2026-02-12 17:12:53 -06:00
26e19b00f7
add preseem sync logs schema and migration 2026-02-12 17:12:19 -06:00
5f17d5aed1
add preseem access points schema and migration 2026-02-12 17:11:36 -06:00
9f3eb55476
add integrations test fixture 2026-02-12 17:03:00 -06:00
9ab0380c87
add integrations settings page with preseem configuration 2026-02-12 17:02:26 -06:00
d7741cdc0e
feat: implement unified checks system (Phase 1-4)
This commit implements the unified checks architecture that consolidates
SNMP monitoring with HTTP/TCP/DNS checks under a single "check" abstraction,
enabling consistent graphing, alerting, and management across all check types.

Database Changes:
- Add source_type and source_id to checks table for tracking auto-discovered
  vs manually created checks
- Add value field to check_results for storing numeric sensor readings
- Maintain backward compatibility with existing check_results data

New SNMP Executors:
- SnmpSensorExecutor: Poll sensor OIDs and return formatted values with
  status determination (OK/WARNING/CRITICAL based on limits)
- SnmpInterfaceExecutor: Poll interface stats (bandwidth, packets, errors)
- SnmpProcessorExecutor: Poll CPU/processor usage
- SnmpStorageExecutor: Poll disk/memory usage with percentage calculations

Check Execution Worker:
- CheckExecutorWorker: Unified Oban worker that dispatches to appropriate
  executor based on check_type (snmp_sensor, snmp_interface, http, tcp, etc.)
- Self-schedules next execution with distributed polling offsets
- Records results in check_results TimescaleDB hypertable
- Updates check state (OK/WARNING/CRITICAL/UNKNOWN)

Discovery Integration:
- Auto-creates checks during SNMP discovery for sensors, interfaces,
  processors, and storage
- Links checks to source entities via source_id for data lookup
- Enables/disables checks based on discovery results

UI Enhancements:
- Checks tab on device detail page with grouped display
- FormComponent for adding manual HTTP/TCP/DNS checks
- Empty state with "Run Discovery" prompt
- Check status badges and last checked times

Graphing:
- Update GraphLive to accept check_id parameter
- Query check_results table for time-series data
- Support all check types (SNMP, HTTP response times, etc.)

Testing:
- Comprehensive test suite for SnmpSensorExecutor (5 tests)
- Test suite for CheckExecutorWorker (7 tests)
- Test coverage for discovery check creation (6 tests)
- Remove deprecated monitoring_test.exs testing old API

Bug Fixes:
- Fix SNMP executors reading credentials from correct Device schema fields
  (device.snmp_version instead of device.snmp_device.version)
- Update agent channel test to query MonitoringCheck table directly

Code Quality:
- Extract add_snmp_credentials helper to reduce cyclomatic complexity
- Use map-based lookups for sensor formatting and check type grouping
- Apply pattern matching in dispatcher to reduce complexity
- All credo checks passing with no issues

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-12 16:58:40 -06:00
06f822dd50
add preseem API client with test connection support 2026-02-12 16:58:16 -06:00
e302b8e9c4
add integrations context with CRUD operations 2026-02-12 16:55:57 -06:00
39f0ffe253
add integrations schema with encrypted credentials 2026-02-12 16:54:14 -06:00
2fd195e6b3
add encrypted map ecto type for integration credentials 2026-02-12 16:52:05 -06:00
a61abd836c
fix: cloud poller automatic rebalancing
- create MonitoringCheck schema for the monitoring_checks table, fixing
  silent data loss where ping results were dropped by the wrong schema
- update agent channel and device monitor worker to use new schema
- fix Stats queries to use MonitoringCheck instead of Check schema
- add list_online_cloud_pollers and list_cloud_polled_devices queries
- add CloudLatencyProbeWorker to dispatch cross-agent latency probes
  every 8 hours via PubSub to all online cloud pollers
- scope AgentLatencyEvaluator to cloud pollers only with cloud_only
  filter, lower min_checks (5), wider time window (48h), and
  device-level assignments to avoid changing site/org defaults
- update cron schedule: probes at 0/8/16h, evaluation at 0:30/8:30/16:30
- refactor monitoring.ex and http_executor.ex to fix credo complexity
2026-02-12 15:32:33 -06:00
07a6128877
feat: add checks tab to device detail page (TDD cycle 1)
- Add 'Checks' tab link to device navigation
- Display empty state when no checks configured
- Add tests for tab visibility and empty state
- Fix unused variable warnings in monitoring context
- Remove unused CheckResult alias and module attributes
- Add stub for legacy get_latency_data for backward compatibility

Tests: 2 new tests added, all passing
Following TDD: RED-GREEN cycle verified for each feature
2026-02-12 14:50:14 -06:00
fa2ebdb973
fix: add diagnostic logging for agent job refresh on device deletion
Enhanced AgentChannel logging to track device IDs across job list updates.
Added tests verifying server correctly excludes deleted devices from job lists.

Root cause: Bug is in Go agent (appends jobs without clearing), not Phoenix server.
2026-02-12 14:36:25 -06:00
7371ceb942
feat: add automatic network topology inference
Build a rich network topology from SNMP polling data using evidence-based
confidence scoring. LLDP/CDP neighbors, MAC address tables, and ARP data
are combined to infer device links with weighted confidence merging.

- Add DeviceLink and DeviceLinkEvidence schemas for persistent topology
- Implement evidence collectors: LLDP (0.95), CDP (0.95), MAC (0.7), ARP (0.6)
- Add device role inference from sysObjectID/sysDescr patterns
- Hook topology inference into DevicePollerWorker pipeline
- Add stale link cleanup (24h mark stale, 72h delete) via NeighborCleanupWorker
- Update NetworkMapLive with "Added" vs "All Devices" tabs
- Add connected devices section to device detail page
- Add device role selector to device edit form
- Update Cytoscape.js with role-based node shapes/colors and confidence edges
2026-02-12 13:28:01 -06:00
be153d364e
feat: add device role inference
Infers device roles from LLDP/CDP capabilities, interface count,
vendor/manufacturer, and system description. Only overwrites
inferred roles, never manual overrides. Wired into process_device
pipeline.
2026-02-12 13:06:41 -06:00
8e355fce79
feat: add process_device/2 topology pipeline
Orchestrates evidence collection from LLDP, CDP, MAC, and ARP
data. Groups evidence by remote device, resolves to managed
devices, upserts links with merged confidence. Broadcasts
topology changes via PubSub. Skips self-links.
2026-02-12 13:02:14 -06:00
9327fd3f13
feat: add confidence merging and link upsert
merge_confidence/2 combines scores using 1-(1-a)(1-b).
upsert_link/1 creates or updates links keyed on source device +
interface + remote MAC. Merges confidence on update, appends
evidence records.
2026-02-12 12:58:06 -06:00
00c366ac9a
feat: add LLDP, CDP, MAC, and ARP evidence collectors
LLDP/CDP evidence (0.95 confidence) from snmp_neighbors.
MAC evidence (0.7) cross-references MAC tables with device
interface MACs. ARP evidence (0.6) matches by IP and MAC.
Self-links excluded. Unmatched entries tracked as discovered.
2026-02-12 12:55:39 -06:00
8d9d385683
fix: prevent AlreadySentError in BruteForceProtection plug
The plug was attempting to register a before_send callback after
blocking a banned IP and sending a 403 response. This caused
Plug.Conn.AlreadySentError in production.

Fixed by checking conn.halted before attempting to register the
404 tracking callback. If the request was already blocked and sent,
we skip the callback registration.

Also updated the test to verify proper behavior without the
workaround for AlreadySentError.
2026-02-12 12:53:32 -06:00
c74243d321
feat: add Topology context with device matching
Build lookup maps to resolve evidence (MAC, IP, hostname) to
managed device IDs. Foundation for evidence collectors.
2026-02-12 12:52:04 -06:00