towerops/CHANGELOG.txt

2026-03-22
feat: simplify device type to manual-only with 6 options
  - Reduced device role dropdown from 10 options to 6: server, switch, router, access_point, backhaul, other
  - Changed label from "Device Role" to "Device Type" for clarity
  - Disabled auto-detection of device type - all types must be manually selected (required field)
  - Migration maps old values to new set (core_router→router, firewall→router, distribution_switch→switch, etc.)
  - Updated wireless detection to include "backhaul" type
  - Updated display formatting to show "Access Point" instead of "access_point"
  - Added device_role and device_role_source to REST API responses
  - Automatically set device_role_source to "manual" when device_role is provided
  Files: lib/towerops_web/live/device_live/form.html.heex,
         lib/towerops/topology.ex,
         lib/towerops/snmp/discovery.ex,
         lib/towerops/devices/device.ex,
         lib/towerops_web/live/device_live/index.ex,
         lib/towerops_web/controllers/api/v1/devices_controller.ex,
         priv/repo/migrations/20260322152809_migrate_device_roles_to_simplified_set.exs,
         test/towerops/topology_test.exs,
         test/towerops_web/live/device_live/form_test.exs

2026-03-19
ui: improve agent status display on device detail page
  - Wrap agent name and icon inside the offline amber badge so it's clear what is offline
  - Add "Agent:" label before the agent indicator in all states (cloud, online, offline)
  Files: lib/towerops_web/live/device_live/show.html.heex

2026-03-19
fix: subscriber count pluralization ("1 sub" instead of "1 subs")
  - Use ngettext in format_subscriber_count in device list and dashboard
  - Dashboard impact column was showing "73s" (appended "s" to number) — now shows "73 subs"
  Files: lib/towerops_web/live/device_live/index.ex,
         lib/towerops_web/live/dashboard_live.ex,
         lib/towerops_web/live/dashboard_live.html.heex

2026-03-19
fix: show ms unit for ping/http/tcp/dns check result values
  - format_check_value now appends " ms" for latency-based check types
  Files: lib/towerops_web/live/device_live/show.ex

2026-03-19
refactor: activity feed from vertical timeline to structured log table
  - Replace left-anchored timeline with full-width table: TIME | TYPE | EVENT | DEVICE/SITE
  - Colored left bar per row replaces timeline dot for severity
  - Period group headers become table section rows
  Files: lib/towerops_web/live/activity_feed_live.html.heex

2026-03-19
fix: pre-existing test failures
  - TimeSeriesTest: switch to async: false to avoid TimescaleDB chunk creation deadlocks
  - settings_live_test: property test now skips assertion when grapheme count < 2 (combining chars)
  Files: test/towerops_web/graphql/resolvers/time_series_test.exs,
         test/towerops_web/live/org/settings_live_test.exs

2026-03-17
fix: DNS and ping checks not executing in production
  - ensure_default_ping_check now calls schedule_check after creating the check
  - CheckExecutorWorker now reschedules skipped checks (agent-assigned, SNMP disabled)
    so the scheduling chain is not permanently broken
  - JobHealthCheckWorker now recovers orphaned CheckExecutorWorker jobs (enabled checks
    with no active Oban job) in addition to DeviceMonitorWorker jobs
  - Wrapped ensure_default_ping_check in try/rescue during device creation to prevent
    transient DB deadlocks from failing the entire device creation
  Files: lib/towerops/monitoring.ex,
         lib/towerops/devices.ex,
         lib/towerops/workers/check_executor_worker.ex,
         lib/towerops/workers/job_health_check_worker.ex,
         test/towerops/monitoring_test.exs,
         test/towerops/workers/check_executor_worker_test.exs,
         test/towerops/workers/job_health_check_worker_test.exs,
         test/towerops/snmp_test.exs

2026-03-16
feat: add service checks REST API, GraphQL API, and ping executor
  - New REST API: full CRUD at /api/v1/checks for HTTP, TCP, DNS, and ping checks
  - New GraphQL API: checks/check queries and createCheck/updateCheck/deleteCheck mutations
  - New ping executor: replaces stub with real ICMP ping via system ping command
  - Ping executor parses both macOS and Linux output for packet loss and RTT
  - Check config validation added for ping type (requires "host")
  - API docs updated with Checks resource section
  Files: lib/towerops/monitoring/executors/ping_executor.ex (new),
         lib/towerops/workers/check_executor_worker.ex,
         lib/towerops/monitoring/check.ex,
         lib/towerops_web/controllers/api/v1/checks_controller.ex (new),
         lib/towerops_web/router.ex,
         lib/towerops_web/graphql/types/check.ex (new),
         lib/towerops_web/graphql/resolvers/check.ex (new),
         lib/towerops_web/graphql/schema.ex,
         lib/towerops_web/controllers/api_docs_html/index.html.heex

2026-03-14
fix: network map label size, site layout, and missing links
  - Increased node label font from 11px to 13px and compound node label to 14px
  - Replaced physics-based cose layout with manual grid preset layout to prevent site group overlap
  - Fixed nil source_interface_id crash in topology.ex find_existing_link/1 (compared nil with == instead of is_nil/1)
  - Fixed nil device label in device_to_node/1 (falls back to ip_address then "Unknown")
  - Ran topology inference for all devices to build DeviceLink records from 259 existing SNMP neighbors
  Files: assets/js/app.ts, lib/towerops/topology.ex

2026-03-13
ux: overhaul Preseem devices page to match Gaiia mapping UX
  - Replace filter tabs with pill buttons (All/Unmatched/Matched)
  - Add IP-based match suggestions: detect when a Preseem AP IP matches a TowerOps device IP
  - Show "Match found" badge on rows with suggestions; smart sort (suggestions first)
  - Show suggestion chips in inline linking row for one-click linking
  - Make unlinked AP rows clickable to open linking panel
  - Make IP addresses clickable http links
  Files: preseem_devices_live.ex, preseem_devices_live.html.heex, preseem_devices_live_test.exs

2026-03-12
fix: comprehensive mobile responsive audit and alignment improvements
  - Devices index: restructured header for 390px (tabs on second row, icon-only buttons)
  - Devices index: hide IP/Type/Last Seen/Response/Subs columns at sm/md/lg breakpoints
  - Sites index: overflow-x-auto wrapper, hide QoE/Subs/Location/Coordinates on mobile
  - Dashboard: overflow-x-auto on alerts + site health tables, hide secondary columns mobile
  - Agents index: overflow-x-auto wrappers around both agent tables
  - Device show: restructured sticky header for mobile (name+status / IP+agent row)
  - Site show: min-w-0 on grid columns, hide IP/Subs columns in device health table
  - Agent show: icon-only action buttons on mobile, fix header component flex-wrap
  - Schedule show: flex-wrap header, icon-only buttons, overflow-x-auto on timelines
  - core_components.ex header component: flex-wrap + flex gap-2 justify-end for actions
  - Device show: all dl/dt/dd value pairs now flex-1 text-right ml-4 for consistent column
  - Device show ports table: Speed column now text-right to match In/Out
  Files: device_live/index.html.heex, site_live/index.html.heex, dashboard_live.html.heex,
         agent_live/index.html.heex, device_live/show.html.heex, site_live/show.html.heex,
         agent_live/show.html.heex, schedule_live/show.html.heex,
         components/core_components.ex

2026-03-12
feat: add real-time GraphQL API with time-series queries and subscriptions
  - Added absinthe_phoenix dependency for WebSocket subscription support
  - Created GraphQLSocket with API token auth at /socket/graphql
  - Added Absinthe.Phoenix.Endpoint to endpoint, Absinthe.Subscription to supervision tree
  - New time-series query types: Sensor, SensorReading, InterfaceTrafficPoint, CheckResultPoint
  - New subscription event types: DeviceStatusEvent, AlertEvent, SensorReadingsEvent
  - 4 new queries: deviceSensors, sensorReadings, interfaceTraffic, checkResults
  - 3 subscriptions: deviceStatusChanged, alertEvent, sensorReadingsUpdated
  - Subscription publisher module wired to agent_channel.ex and alerts.ex
  - Org-scoped access control for sensors/interfaces via 3-join verification queries
  - Traffic rate computation from SNMP counter deltas (octets → bps)
  - 17 tests covering queries, org isolation, edge cases, and auth
  - Updated GraphQL API docs page with time-series and subscription sections
  - Files: mix.exs, lib/towerops/application.ex, lib/towerops_web/endpoint.ex,
    lib/towerops_web/graphql_socket.ex, lib/towerops_web/graphql/types/time_series.ex,
    lib/towerops_web/graphql/schema.ex, lib/towerops_web/graphql/resolvers/time_series.ex,
    lib/towerops_web/graphql/subscriptions.ex, lib/towerops_web/channels/agent_channel.ex,
    lib/towerops/alerts.ex, lib/towerops_web/controllers/graphql_docs_html/index.html.heex

2026-03-09
fix: add navigation and footer to changelog page
  - Wrapped ChangelogLive content in Layouts.authenticated component
  - /changelog now includes standard navigation header and footer
  - Matches layout of other authenticated pages
  - Files: lib/towerops_web/live/changelog_live.ex

2026-03-09
feat: add ErrorTrackerNotifier for email alerts on exceptions
  - Added error_tracker_notifier dependency (~> 0.2)
  - Configured email notifications in config/runtime.exs (production only)
  - Sends exception alerts from alerts@towerops.net to graham@towerops.net
  - Added to supervision tree in lib/towerops/application.ex
  - Includes throttling to prevent duplicate error spam
  - Gracefully shuts down in dev/test (no emails sent)
  - Files: mix.exs, config/runtime.exs, lib/towerops/application.ex

2026-03-09
fix: AlertNotificationWorker failing when PagerDuty not configured
  - Worker was treating {:error, :not_configured} as failure, causing retries
  - Added graceful handling for :not_configured in all three actions (trigger, acknowledge, resolve)
  - Now logs as debug and succeeds when PagerDuty integration not set up
  - Prevents unnecessary error noise in ErrorTracker for organizations without PagerDuty
  - Files: lib/towerops/workers/alert_notification_worker.ex

2026-03-06
feat: improve Gaiia webhook setup instructions and descriptions
  - Updated Gaiia provider description to explain what it enables
  - Added "What webhooks enable" section explaining account, billing, and inventory events
  - Enhanced webhook description to explain real-time vs scheduled sync difference
  - Added webhook secret explanation text
  - Improved setup instructions with bold formatting and verification tip
  - Files: lib/towerops_web/live/org/integrations_live.ex, integrations_live.html.heex

2026-03-06
fix: global search (Cmd+K) not returning results
  - handle_event("search") was reading from phx-value-query which contained stale assign
  - Changed to read "value" key from native keyup event params
  - Removed unused phx-value-query attribute from search input
  - Files: lib/towerops_web/live/components/global_search_component.ex

2026-03-06
feat: dynamic global pricing configuration with Stripe integration
  - Add default_free_devices and default_price_per_device to ApplicationSettings
  - Update default price from $1.00 to $2.00/device/month
  - Add Billing.default_free_devices/0 and default_price_per_device/0 with fallbacks
  - Add StripeClient.create_price/1 for metered billing Price creation
  - Add StripeClient.update_subscription_price/2 for price migrations
  - Add Billing.migrate_all_subscriptions_to_price/1 with best-effort migration
  - Add Admin.update_global_pricing/3 with validation and audit logging
  - Add global defaults UI on /admin/organizations with confirmation dialog
  - Update marketing pages to reflect $2/device/month pricing
Files: priv/repo/migrations/*seed_billing_settings.exs,
       lib/towerops/settings/application_setting.ex,
       lib/towerops/billing.ex,
       lib/towerops/billing/stripe_client.ex,
       lib/towerops/admin.ex,
       lib/towerops/admin/audit_log.ex,
       lib/towerops_web/live/admin/org_live/index.ex,
       lib/towerops_web/live/admin/org_live/index.html.heex,
       lib/towerops_web/controllers/page_html/home.html.heex

2026-03-06
feat: per-organization billing override admin UI
  - Add custom_free_device_limit and custom_price_per_device nullable fields to organizations
  - Add billing_override_changeset/2 with validation (separate from main changeset)
  - Add effective_device_limit/1 to SubscriptionLimits respecting custom overrides
  - Add effective_free_device_count/1 and effective_price_per_device/1 to Billing
  - Update billable_device_count and estimated_monthly_cost to use effective values
  - Add Admin.update_billing_overrides/4 with audit logging
  - Add override editing UI to /admin/organizations with edit panel
  - Update org settings page to display effective limits/pricing instead of hardcoded values
Files: priv/repo/migrations/20260306184112_add_billing_overrides_to_organizations.exs,
       lib/towerops/organizations/organization.ex,
       lib/towerops/organizations/subscription_limits.ex,
       lib/towerops/billing.ex,
       lib/towerops/admin.ex,
       lib/towerops/admin/audit_log.ex,
       lib/towerops_web/live/admin/org_live/index.ex,
       lib/towerops_web/live/admin/org_live/index.html.heex,
       lib/towerops_web/live/org/settings_live.html.heex

fix: access interface stats through latest_stat association
  - Template accessed if_in_octets/if_out_octets directly on Interface struct
  - These fields live on InterfaceStat (interface.latest_stat)
  - Added nil guard to prevent KeyError when association not loaded
  - Fixes production 500 error on device detail page
Files: lib/towerops_web/live/device_live/show.html.heex

fix: allow data URLs in CSP for dynamic favicon
  - Added data: to default-src CSP directive to support canvas-generated favicons
  - Fixes dashboard status indicator favicon not working in staging/production
  - DynamicFavicon hook uses canvas.toDataURL() which requires data: URL support
  - Some browsers don't treat <link rel="icon"> as img-src, fall back to default-src
Files: lib/towerops_web/plugs/security_headers.ex

2026-03-05
feat: add rate limiting to admin endpoints (100 req/min)
  - Add :admin type to RateLimit plug with 100 requests per minute per IP
  - Apply rate limiting to all /admin routes (LiveView and controller actions)
  - Protects superuser endpoints from abuse and brute force attempts
  - Aligns with existing auth (10 req/min) and API (1000 req/min) rate limits
Files: lib/towerops_web/plugs/rate_limit.ex,
       lib/towerops_web/router.ex

feat: add TowerOps suffix to all page titles for better tab identification
  - Page titles now show "Page Title | TowerOps" format
  - Helps users identify TowerOps tabs when multiple tabs are open
  - Uses live_title suffix parameter for automatic appending
  - Falls back to just "TowerOps" when no page_title is set
Files: lib/towerops_web/components/layouts/root.html.heex

fix: add --skip-if-loaded flag to test alias to prevent prompts
  - Test alias now uses 'ecto.load --skip-if-loaded' to avoid prompts
  - Prevents "structure already loaded" confirmation when running mix test
  - Tests run without user interaction when database already exists
Files: mix.exs

fix: correct mail adapter message interpolation in login page
  - Fix gettext translation to use proper %{link} interpolation
  - Consolidate fragmented translation strings into single interpolated message
  - Update Spanish translation to include placeholder
  - Prevents display of raw "%{link}" text in development mode
Files: lib/towerops_web/controllers/user_session_html/new.html.heex,
       priv/gettext/auth.pot,
       priv/gettext/en/LC_MESSAGES/auth.po,
       priv/gettext/es/LC_MESSAGES/auth.po

perf: optimize CI database setup with structure.sql dumps (99.7% faster)
  - Replace sequential migration runs (4m8s) with structure.sql loading (416ms)
  - Configure Ecto to dump schema to priv/repo/structure.sql
  - Update test alias to use ecto.load instead of ecto.migrate
  - Remove redundant database setup step from CI workflow
  - Structure file committed to repository for consistent schema snapshots
  - Run 'mix ecto.dump' after migrations to update structure.sql
Files: config/config.exs,
       mix.exs,
       .forgejo/workflows/build.yaml,
       priv/repo/structure.sql (new)

perf: optimize dashboard device count queries (partial N+1 fix)
  - Add Devices.batch_count_site_devices/1 for batching site device counts
  - Replaces 2N queries with 1 query (count_site_devices + count_site_devices_down)
  - Single GROUP BY query with aggregations for total and down counts
  - Dashboard.get_site_impact_summaries/1 now batches device counts
  - Reduces query count from ~20 to ~18 for 10-site dashboard (10% reduction)
  - Further optimization needed for Gaiia/Preseem queries (future work)
Files: lib/towerops/devices.ex,
       lib/towerops/dashboard.ex

fix: suppress noisy health check logs in production
  - Add logger filter to drop K8s health probe logs
  - Prevents log flooding from /health endpoint calls every few seconds
  - Configured in prod.exs logger :default_handler filters
  - TelemetryFilter.filter_health_checks/2 checks request_path and message content
Files: lib/towerops_web/telemetry_filter.ex,
       lib/towerops/application.ex,
       config/prod.exs

security: comprehensive security audit fixes (9 critical/high priority issues)
  - Remove /health/time endpoint exposing system time information (CRITICAL)
  - Add email confirmation check to account data export endpoint (CRITICAL)
  - Add path traversal validation for MIB archive uploads (HIGH)
  - Add input validation for mobile auth device parameters (HIGH)
  - Add heartbeat rate limiting to agent channel (30s min between DB updates) (HIGH)
  - Sanitize 500 error responses to prevent information disclosure (HIGH)
  - Add GraphQL query depth limits (max depth: 10) (MEDIUM)
  - Add message size limits to agent channel (10MB max) (MEDIUM)
Files: lib/towerops_web/controllers/health_controller.ex,
       lib/towerops_web/controllers/api/account_data_controller.ex,
       lib/towerops_web/controllers/api/v1/mib_controller.ex,
       lib/towerops/mobile_sessions/mobile_session.ex,
       lib/towerops_web/channels/agent_channel.ex,
       lib/towerops_web/controllers/error_json.ex,
       lib/towerops_web/router.ex,
       test/towerops_web/controllers/health_controller_test.exs,
       test/towerops_web/controllers/error_json_test.exs

feat: implement LLDP topology discovery via SNMP
  - Add LLDP-MIB walker for discovering network neighbors via SNMP
  - Create device_neighbors table with neighbor relationships
  - Implement Towerops.Topology.Lldp module for SNMP LLDP discovery
  - Add discover_lldp_neighbors/1, list_lldp_neighbors/1, remove_stale_lldp_neighbors/1 functions
  - Automatic device linking when neighbors are found in database
  - Support for IPv4 and IPv6 management addresses
  - Implements Phase 1 of network topology discovery (LLDP via SNMP)
  - LLDP-MIB OIDs: lldpLocSysName, lldpLocPortDesc, lldpRemPortId, lldpRemPortDesc,
    lldpRemSysName, lldpRemManAddr
Files: lib/towerops/topology/lldp.ex, lib/towerops/topology/device_neighbor.ex,
       lib/towerops/topology.ex, priv/repo/migrations/20260305164021_create_device_neighbors.exs

fix: use DATABASE_URL from environment in test config
  - Check for DATABASE_URL environment variable in config/test.exs
  - Use DATABASE_URL if present (CI), fall back to localhost config (local dev)
  - Fixes "connection refused to localhost:5432" errors during compilation in CI
  - Allows config evaluation to use correct hostname before database operations
Files: config/test.exs

ci: set DATABASE_URL globally for test job
  - Move MIX_ENV and DATABASE_URL to job-level environment variables
  - Ensures all database operations (ecto.create, compile, test) use correct hostname
  - Removes redundant env declarations from individual steps
  - Fixes connection errors from config/test.exs hardcoded localhost
Files: .forgejo/workflows/build.yaml

ci: install postgresql-client for pg_isready command
  - Add postgresql-client to system dependencies installation
  - Resolves "pg_isready: command not found" errors during PostgreSQL health checks
  - Essential for database readiness verification before tests run
Files: .forgejo/workflows/build.yaml

2026-03-04
feat: add admin user impersonation with comprehensive audit logging
  - Superusers can impersonate other users (including other superusers) via /admin/users
  - All impersonation events logged to audit_logs table with full context
  - Stop impersonation link appears in user menu when session is impersonated
  - Session assigns track impersonation state (:impersonating_user_id)
  - Audit log captures: impersonator email, target user email, IP address, user agent
  - Access control: only superusers can impersonate, enforced at controller and route level
Files: lib/towerops_web/controllers/admin/user_controller.ex,
       lib/towerops_web/live/nav_live.ex,
       lib/towerops_web/router.ex,
       lib/towerops/admin/audit_logger.ex,
       lib/towerops_web/components/layouts/root.html.heex,
       test/towerops_web/controllers/admin/user_controller_test.exs

feat: add Redis health check with timeout handling
  - Create Towerops.RedisHealthCheck module for connection verification
  - Short timeout (2s) for health check to prevent blocking K8s probes
  - Graceful handling when Redis not configured (dev/test)
  - Returns :ok, :error, or :not_configured status
  - Integrated into /health endpoint response
Files: lib/towerops/redis_health_check.ex,
       lib/towerops_web/controllers/health_controller.ex,
       test/towerops_web/controllers/health_controller_test.exs

fix: add FilterNoisyLogs plug to silence health check spam
  - Create plug to disable logging for /health endpoint
  - Sets phoenix_log: false and plug_skip_telemetry: true
  - Prevents Kubernetes liveness/readiness probes from flooding logs
  - Applied before router in endpoint pipeline
Files: lib/towerops_web/plugs/filter_noisy_logs.ex,
       lib/towerops_web/endpoint.ex

feat: add comprehensive audit logging system
  - Create audit_logs table with action, metadata, IP, user agent tracking
  - Implement Towerops.Admin.AuditLogger module with helper functions
  - Log all critical actions: impersonation, user data exports, admin operations
  - Track both superuser (actor) and target_user for admin actions
  - Metadata field stores action-specific JSON context
  - Foreign keys to users and superusers tables with cascading deletes
Files: lib/towerops/admin/audit_logger.ex,
       lib/towerops/admin/audit_log.ex,
       priv/repo/migrations/20260304XXXXXX_create_audit_logs.exs

2026-03-03
perf: optimize dashboard N+1 queries with batch loading
  - Replace per-site device queries with single batch query using site_id IN (...)
  - Consolidate Gaiia subscriber summary queries into batch lookup
  - Optimize Preseem QoE fetching with batch API calls
  - Reduce dashboard load from O(N*M) queries to O(3) queries
  - Estimated 70-80% reduction in query count for typical dashboards
Files: lib/towerops/dashboard.ex

perf: add database indexes for frequently queried fields
  - Add composite index on alerts (organization_id, alert_type, resolved_at)
  - Add index on devices.organization_id for faster org-level queries
  - Add index on sites.organization_id for dashboard performance
  - Significantly improves dashboard and alert list query performance
Files: priv/repo/migrations/20260303XXXXXX_add_performance_indexes.exs

2026-03-02
fix: prevent TOTP replay attacks with nonce tracking
  - Store used TOTP codes in totp_nonces table with 90-second expiration
  - Validate code hasn't been used before accepting
  - Automatic cleanup of expired nonces via Oban cron job
  - Prevents code reuse within TOTP validity window
Files: lib/towerops/accounts/totp_nonce.ex,
       lib/towerops/accounts.ex,
       priv/repo/migrations/20260302XXXXXX_create_totp_nonces.exs

2026-03-01
feat: add equipment manufacturer detection and tracking
  - Auto-detect manufacturer from SNMP sysDescr during discovery
  - Store in devices.manufacturer field
  - Display in equipment details and lists
  - Enables vendor-specific features and MIB selection
Files: lib/towerops/snmp/discovery.ex,
       lib/towerops/devices/device.ex,
       priv/repo/migrations/20260301XXXXXX_add_manufacturer_to_devices.exs