Promoted pure presentation and utility helpers from `defp` to `def @doc false`
across ~20 LiveViews, Oban workers, and sync modules so they're reachable from
unit tests. Refactored several `cond` blocks into idiomatic function heads with
guards. Added ~250 new test cases in new files under test/towerops and
test/towerops_web, including DB-backed tests for CnMaestro.Sync and
AlertNotificationWorker, and removed dead LiveView tab components and
CapacityLive (no callers anywhere in lib/test).
Configured mix.exs test_coverage.ignore_modules to exclude vendored third-party
code (SnmpKit, protobuf-generated Towerops.Agent.*, Absinthe GraphQL types,
Phoenix HTML modules, Inspect protocol impls) from coverage calculations —
these are not our project code.
Coverage: 66.93% → 70.09%. Full suite: 10,127 tests, 0 failures.
Prefix fire-and-forget calls (PubSub.broadcast, Oban enqueue, Logger,
update_*, etc.) with `_ =` so dialyzer knows the return value is
intentionally discarded. Wrap a few if/case expressions whose
branches produce mixed types the same way.
No behavior changes — only explicit acknowledgement of discarded
returns.
Warnings: 242 → 88.
- Add @type t :: %__MODULE__{} to schemas referenced by callers:
ApiToken, Monitoring.Check, OnCall.Schedule, SiteOutage,
NotificationDigest, Snmp.WirelessClient, Topology.DeviceLink,
Agent.AgentJob, Agent.MikrotikResult.
- Proto.Types: qualify forward references to nested modules with
Types. prefix, and fix Metric typo (should be union type metric()).
- data_loaders.ex / chart_builders.ex: Snmp.SNMPDevice.t was a ghost
spec — module is Snmp.Device.
Warnings: 328 → 242.
Phase 1: Foundation Types (100% complete)
- query_helpers: SQL LIKE sanitization with pipe operators
- numeric: Integer parsing with pattern matching guards
- result: Pure Elixir Result monad (map, and_then, unwrap_or)
Phase 2: Ecto Domain Types (100% complete)
- ip_address: IPv4/IPv6 validation using :inet directly
- mac_address: Multi-format MAC parsing (colon/hyphen/dot/compact)
- snmp_oid: OID parsing/manipulation with recursive pattern matching
All 198 tests passing across converted modules.
API changed from Gleam-style {:some/:none to idiomatic {:ok/:error.
Refactored parse_numeric_oid to use with statement, reducing nesting depth.
Reviewed-on: graham/towerops-web#196
The wrap detection only applied the 2^32 correction when the previous
counter was in the upper half of the 32-bit range (> 2^31). On fast
links, counters wrap regardless of where in the range they sit. Three
graph rendering paths had no wrap detection at all, just max(0).
Consolidated all bps calculations into a single public
Capacity.calculate_bps/3 that always tries the 2^32 correction for
negative deltas. 64-bit HC counter resets remain clamped to zero since
the corrected delta stays negative.
Reviewed-on: graham/towerops-web#163
Standard ifInOctets/ifOutOctets are 32-bit counters that wrap around
4,294,967,296 bytes (~4.3 GB). At 1 Gbps this wraps every ~34 seconds,
causing brief drops to 0 bps in traffic charts and capacity reports.
When prev > 2^31 and delta is negative, add 2^32 to recover the correct
rate. If the result is still negative (HC 64-bit counter after reboot),
clamp to 0. Small-value decreases (prev < 2^31) continue to clamp to 0
since they cannot be real 32-bit wraps.
Fixes both capacity.ex (utilization reports) and device_live/show.ex
(per-device traffic chart).
Reviewed-on: graham/towerops-web#161
CRITICAL FIXES:
- Fix unsafe pattern matching in SNMP discovery that would crash on timeout
- Changed {:ok, _} = ... to proper case statements with error handling
- Extracted safe_discover/3 helper to reduce cyclomatic complexity
- Added fallback to empty lists and warning logging for timeouts
- Affects: discover_vlans, discover_ip_addresses, discover_processors, discover_storage
MEMORY LEAK FIXES (JS Hooks):
- GlobalSearchTrigger: Added destroyed() cleanup for click listener
- SidebarCollapse: Added destroyed() cleanup for click listener
- ThemeSelector: Added destroyed() cleanup for phx:set-theme window listener
- NetworkMap: Added destroyed() cleanup for 3 button listeners (zoom in/out/fit)
- All hooks now properly store handler references and remove listeners on unmount
LIVEVIEW FIXES:
- Added phx-update="ignore" to 5 SensorChart hooks to prevent chart re-initialization
- Added catch-all handle_info(_msg, socket) to 5 LiveViews to prevent crashes on unexpected messages
- device_live/show.ex, device_live/index.ex, alert_live/index.ex, activity_feed_live.ex, agent_live/index.ex
N+1 QUERY FIXES:
- Created Gaiia.get_site_subscriber_summaries/1 batch function
- Refactored alert_live and device_live to use batch query instead of looping
- Reduces queries from N to 1 when loading site subscriber data
Impact:
- Prevents discovery worker crashes during SNMP timeouts
- Eliminates memory accumulation from leaked event handlers
- Prevents chart state loss on LiveView updates
- Prevents LiveView crashes from unexpected PubSub messages
- Improves database performance by eliminating N+1 queries in alert/device views
- Passes credo --strict with 0 issues
Reviewed-on: graham/towerops-web#132
Seconds-level precision adds noise without value for these fields.
Shows "less than a minute" instead of "0 seconds" for very short durations.
Reviewed-on: graham/towerops-web#120
- Add missing flex-1/text-right classes to Resolved IP row
- Add justify-end to Operating System flex container
- Fix duplicate time labels on 24h traffic chart by excluding boundary ticks
Reviewed-on: graham/towerops-web#117
- Capacity: Sensor values (Rx/Tx Capacity) now represent total device
capacity, not per-interface. Previous bug applied the same sensor value
to all interfaces and summed them (e.g., 773.6 Mbps × 5 = 3.87 Gbps).
- Wireless sensors: Changed Float.round/2 to :erlang.float_to_binary/2
to prevent scientific notation in frequency display (was showing
"2.41e4 MHz" instead of "24100.0 MHz").
Files:
- lib/towerops_web/live/device_live/show.ex
- lib/towerops_web/live/device_live/show.html.heex
- CHANGELOG.txt
- priv/static/changelog.txt
Reviewed-on: graham/towerops-web#116
Radio frequencies (24 GHz) were displaying as "2.42e4 MHz" instead of
"24200.0 MHz" because Float.round/2 returns a float that Phoenix
converts to scientific notation for large numbers.
Changed format_sensor_value/2 to use :erlang.float_to_binary/2 with
decimals: 1 option, which formats as a string and avoids scientific
notation.
Verified: 24200.0 now displays as "24200.0" instead of "2.42e4"
Reviewed-on: graham/towerops-web#114
For devices with device_role="backhaul", traffic graphs now show capacity
reference lines that update over time based on which interfaces are active.
Features:
- Calculates total capacity per timestamp from active monitored interfaces
- Uses configured_capacity_bps if set, falls back to if_speed
- Capacity adjusts as interfaces go up/down throughout the time period
- Shows as dashed lines at +capacity (inbound) and -capacity (outbound)
- Matches the existing 0-axis graph style
Implementation:
- Small graph (device detail page): Updated build_traffic_json_datasets/2
and add_capacity_datasets/3 in device_live/show.ex
- Full-page graph: Updated build_traffic_datasets/2 and
add_capacity_to_full_graph/4 in graph_live/show.ex
The capacity lines provide visual reference for backhaul link saturation,
showing how close actual throughput is to maximum capacity at each point
in time.
Reviewed-on: graham/towerops-web#113
- Removed device_role_source assignment in sanitize_device_role/1
- Updated structure.sql to reflect database schema without device_role_source column
- Fixes ERROR 42703 (undefined_column) in production workers
The form was still trying to set device_role_source in params even though
the database column was dropped, causing runtime errors in:
- AgentLatencyEvaluator
- CapacityInsightWorker
- WirelessInsightWorker
All device roles are now manually set with default value 'other'.
Reviewed-on: graham/towerops-web#110
- Reduce device type options from 10 to 6 (server, switch, router, access_point, backhaul, other)
- Change label from "Device Role" to "Device Type"
- Disable auto-detection - device type is now required and manual-only
- Add migration to map existing device roles to new simplified set
- Update REST API to include device_role and device_role_source fields
- Automatically set device_role_source to "manual" when device_role is provided
- Update wireless detection to include "backhaul" type
- Enhance display formatting for "Access Point"
Breaking Changes:
- Auto-inference of device type has been disabled
- device_role is now a required field in forms
- GraphQL/REST API consumers will see changed device_role values after migration
Files Changed:
- lib/towerops_web/live/device_live/form.html.heex
- lib/towerops/topology.ex
- lib/towerops/snmp/discovery.ex
- lib/towerops/devices/device.ex
- lib/towerops_web/live/device_live/index.ex
- lib/towerops_web/controllers/api/v1/devices_controller.ex
- priv/repo/migrations/20260322152809_migrate_device_roles_to_simplified_set.exs
- test/towerops/topology_test.exs
- test/towerops_web/live/device_live/form_test.exs
Reviewed-on: graham/towerops-web#109
This commit addresses multiple CRITICAL, HIGH, and MEDIUM severity security vulnerabilities identified in the security audit:
CRITICAL FIXES:
- Fix weak RNG for recovery codes - replaced Enum.random() with :crypto.strong_rand_bytes/1 for cryptographically secure token generation
- Fix subscription limit race conditions - moved free org and device quota checks inside transactions with FOR UPDATE locks to prevent concurrent bypass
- Fix default organization race condition - moved is_default check inside transaction to prevent multiple defaults per user
HIGH SEVERITY FIXES:
- Fix agent token deletion race condition - moved PubSub broadcast inside transaction to ensure agents only receive notification after successful deletion
MEDIUM SEVERITY FIXES:
- Fix LIKE wildcard injection in search - applied sanitize_like() to all user-facing search queries in devices.ex, sites.ex, and gaiia.ex to prevent enumeration attacks
- Fix Jason.decode! DoS - replaced with safe Jason.decode/1 with error handling in device_live/index.ex
- Fix SSRF vulnerability - added URL validation in HTTP executor to block requests to private/internal IP ranges (127.0.0.0/8, 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 169.254.0.0/16)
- Fix error information leakage - replaced inspect() in API responses with generic error messages, logging details server-side only
- Fix atom table pollution - HTTP method normalization now uses whitelist mapping instead of String.to_atom()
SECURITY IMPROVEMENTS:
- All quota checks now use pessimistic locking (SELECT FOR UPDATE) to prevent TOCTOU race conditions
- Private IP validation prevents cloud metadata service access (169.254.169.254)
- DNS resolution performed before HTTP requests to detect IP spoofing
- Error details logged server-side but not exposed to clients
Files changed:
- lib/towerops/accounts/user_recovery_code.ex
- lib/towerops/organizations.ex
- lib/towerops/devices.ex
- lib/towerops/sites.ex
- lib/towerops/gaiia.ex
- lib/towerops/agents.ex
- lib/towerops/monitoring/executors/http_executor.ex
- lib/towerops_web/live/device_live/index.ex
- lib/towerops_web/controllers/api/v1/mib_controller.ex
- lib/towerops_web/controllers/api/v1/agent_release_webhook_controller.ex
- lib/towerops_web/controllers/api/v1/geoip_controller.ex
Reviewed-on: graham/towerops-web#108
Migrate three pure-function modules from Elixir to Gleam:
- polling_offset: deterministic hash-based polling offset (pure Gleam, no FFI)
- changelog_parser: text parsing logic in Gleam, thin Elixir wrapper for File I/O
- user_agent_parser: regex-based UA parsing in Gleam with Erlang FFI for
atom-keyed map conversion
Add gleam_regexp dependency for regex support in Gleam modules.
Update all call sites and tests to use Gleam module atoms.
Reviewed-on: graham/towerops-web#103
- Wrap agent name + icon inside the offline amber badge so it's clear
what is offline on the device detail page
- Add "Agent:" label before the agent indicator for all states
- Fix TimeSeriesTest deadlock: switch to async: false since check_results
is a TimescaleDB hypertable and concurrent chunk creation deadlocks
- Fix unicode property test: String.length/1 counts grapheme clusters,
not code points; combining chars can make a 2-codepoint string have
only 1 grapheme, legitimately failing the min: 2 validation
Reviewed-on: graham/towerops-web#83
## Summary
- **DNS checks**: `parse_server` returned a flat 5-tuple `{a,b,c,d,53}` instead of `{{a,b,c,d}, 53}` — the format `:inet_res` requires for nameservers. All DNS checks using a custom server failed with argument error.
- **Ping checks**: `System.cmd` `:timeout` option not supported on Elixir 1.19.4 (staging). Replaced with `Task.async`/`Task.yield` wrapper for cross-version compatibility.
- **Checks table UI**: Removed unnecessary flex div wrapper on status badge column that caused vertical misalignment with other table cells.
## Test plan
- [x] DNS executor tests pass (custom server resolution now succeeds)
- [x] Ping executor tests pass (no more ArgumentError)
- [x] Full test suite passes (8593 tests, 0 new failures)
Reviewed-on: graham/towerops-web#72
add ssl check executor that connects via TLS, reads peer certificate
expiry, and returns OK/WARNING/CRITICAL based on configurable
warning_days threshold. includes protobuf config, liveview form fields,
agent channel encoding, oban worker dispatch, and unit tests.
Reviewed-on: graham/towerops-web#64
- Skip HTTP/TCP/DNS checks in Phoenix when device is assigned to agent
- Agents now execute service checks from the device's network location
- Fix cancel button in service check modal (text link instead of button)
- Fix status badge alignment in checks table
Reviewed-on: graham/towerops-web#63
send check_jobs protobuf to agents during job dispatch cycle, handle
check_result messages back with state transitions and alert
creation/resolution. auto-assign checks to device's effective agent
token on creation, broadcast check changes via PubSub so connected
agents get immediate updates.
add edit/delete support for service checks on device show page with
form pre-population from existing check config. auto-fill TCP host
from device IP address.
includes validate_check_result/1 for protobuf validation and
list_checks_for_agent/2 for querying agent-assigned service checks.
also fixes pre-existing broken device_type_icon reference in device
index template.
Reviewed-on: graham/towerops-web#50
- LIKE wildcard injection: sanitize %, _ in search queries (devices, sites, gaiia)
- Jason.decode! → Jason.decode with error handling for untrusted input
- inspect() leak: replace with generic error messages, log details server-side
- SSRF protection: URL validator blocks private IPs, localhost, non-HTTP schemes
- SSRF validation added to HTTP monitoring executor and integration credentials
- GraphQL complexity limits: always applied, not just in prod
- GraphQL introspection: also check GET query params, not just body
- Stripe webhook: explicit nil/empty checks for signature and body
- Cookie security: secure flag for session (prod), http_only+secure for remember_me
- Honeybadger API key: read from env var with fallback
- String.to_integer → Integer.parse with fallback for URL params
- String.to_atom → whitelist map for HTTP methods
- Gaiia webhook: remove secret_len and expected signature from log
- Admin API: add rate limiting pipeline
- to_atom_keys: per-key fallback instead of all-or-nothing rescue
- Hide less-critical table columns on small screens (IP, subs, MRR, etc.)
- Add overflow-x-auto wrappers on tables to prevent horizontal scroll
- Fix <.header> component to flex-wrap action buttons on mobile
- Make page action button labels icon-only on small screens
- Add min-w-0 on grid children to prevent viewport overflow
- Right-align device detail dt/dd values consistently
- Fix schedule timeline cards with overflow-x-auto wrappers
Built-in PagerDuty-equivalent system for on-call scheduling and alert
escalation. Users can now manage schedules, rotation layers, overrides,
and escalation policies directly in the app alongside PagerDuty.
- On-call schedules with rotation layers (daily/weekly/custom), member
management, and temporary overrides
- Escalation policies with ordered rules, timeout-based escalation,
and user/schedule targets
- Automatic escalation via Oban worker with configurable repeat count
- Email notifications via Swoosh for on-call alerts
- Resolver computes who's on-call from layer stacking and overrides
- AlertNotificationWorker integration: starts escalation alongside
PagerDuty, acknowledges/resolves incidents on alert state changes
- Device and organization schemas support escalation_policy_id
- Escalation policy picker on device form
- Schedules nav item with tabbed index (schedules + escalation policies)
- Full CRUD UI for schedules, layers, members, overrides, rules, targets
- 62 LiveView tests, 56 context/schema/resolver/escalation tests
- 26 E2E Playwright tests for smoke and critical path coverage
Proactively add catch-all handlers to device index/show pages to
prevent FunctionClauseError when StatusTitleComponent broadcasts
alert changes. These are the most frequently open pages where users
would encounter the crash.
Removed:
- Compact view toggle button from the header
- toggle_compact_mode event handler
- @compact_mode assign
The button was not connected to any actual functionality.
The discovered tab badge was showing length(@discovered_devices) which is
the paginated list (20 items per page), not the total count. Now shows
@pagination.total_count to display the actual total (e.g., 876 devices).
- Fixed debounce_ms/0 function still using Mix.env()
- Changed to Application.get_env(:towerops, :env)
- This was causing crashes in staging/production
Add CapacityInsightWorker (every 15 min) that generates critical/warning
insights when backhaul utilization exceeds 90%/75% and auto-resolves
when it drops below 70%.
Add capacity and utilization columns to the device ports tab with
set/clear capacity controls. Add organization-level /capacity page
with summary cards and per-site capacity table. Add capacity summary
card to site show page. Add Capacity link to nav menu.
