graham/towerops
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
towerops/docs/TEST_COVERAGE.md
Graham McIntie 1590f78bdc fix: input validation, SSRF, API hardening, and cookie security 
- LIKE wildcard injection: sanitize %, _ in search queries (devices, sites, gaiia)
- Jason.decode! → Jason.decode with error handling for untrusted input
- inspect() leak: replace with generic error messages, log details server-side
- SSRF protection: URL validator blocks private IPs, localhost, non-HTTP schemes
- SSRF validation added to HTTP monitoring executor and integration credentials
- GraphQL complexity limits: always applied, not just in prod
- GraphQL introspection: also check GET query params, not just body
- Stripe webhook: explicit nil/empty checks for signature and body
- Cookie security: secure flag for session (prod), http_only+secure for remember_me
- Honeybadger API key: read from env var with fallback
- String.to_integer → Integer.parse with fallback for URL params
- String.to_atom → whitelist map for HTTP methods
- Gaiia webhook: remove secret_len and expected signature from log
- Admin API: add rate limiting pipeline
- to_atom_keys: per-key fallback instead of all-or-nothing rescue
2026-03-14 14:48:59 -05:00

234 lines
12 KiB
Markdown
Raw Blame History

# TowerOps Test Coverage Report
**Generated:** 2026-03-13
**Test Run:** 90 doctests, 55 properties, 8,134 tests | 4 failures, 38 skipped, 231 excluded
**Duration:** 158.2s (40.5s async, 117.6s sync)
## Summary
| Metric | Count |
|---|---|
| Total test files | 547 |
| Total tests (incl. doctests/properties) | 8,279 |
| Passing | 8,237 |
| Failures | 4 |
| Skipped | 38 |
| Excluded | 231 |
| Source modules (excl. vendor profiles, migrations, proto) | ~230 |
| Web modules (controllers, LiveViews, plugs, etc.) | ~140 |
| Modules with NO test file | ~100 |
## Current Failures (4)
1. **`ToweropsWeb.AgentLive.IndexTest`** — Unique constraint violation on `agents.slug` during setup (test data collision)
2. **`ToweropsWeb.AgentLive.IndexTest`** — Same slug collision in cloud poller creation setup
3. **`ToweropsWeb.AgentLive.IndexTest`** — Same root cause, different describe block
4. **`ToweropsWeb.GraphQL.Resolvers.TimeSeriesTest`** — Deadlock on `create_check_result` (async test hitting `ShareRowExclusiveLock`)
> Both are infrastructure/setup issues, not logic bugs. The agent slug collision likely needs a unique factory counter; the deadlock needs `async: false` or transaction isolation.
---
## Test Infrastructure & Patterns
### Case Templates
- **`ToweropsWeb.ConnCase`** — HTTP/LiveView tests. Imports `Phoenix.ConnTest`, `Plug.Conn`, verified routes. Has `register_and_log_in_user/1` and sudo variant.
- **`Towerops.DataCase`** — Context/schema tests. Imports `Ecto`, `Ecto.Changeset`, `Ecto.Query`, `Towerops.Repo`. SQL Sandbox isolation.
- **`ToweropsWeb.LiveViewTestHelpers`** — Shared helpers for LiveView tests.
### Fixtures (Factory Pattern)
All in `test/support/fixtures/`:
- `accounts_fixtures.ex` — Users, tokens, TOTP
- `agents_fixtures.ex` — Agents, assignments, tokens
- `billing_fixtures.ex` — Stripe data
- `devices_fixtures.ex` — Devices, firmware
- `gaiia_fixtures.ex` — GAIIA accounts, subscribers
- `integrations_fixtures.ex` — Integration configs
- `jobs_fixtures.ex` — Oban jobs
- `maintenance_fixtures.ex` — Maintenance windows
- `on_call_fixtures.ex` — Schedules, escalation policies
- `organizations_fixtures.ex` — Orgs, memberships, invitations
- `snmp_fixtures.ex` — SNMP devices, interfaces
### Test Types
- **Async tests:** ~442 files use `async: true`
- **Property tests:** 55 (using `StreamData`)
- **Doctests:** 90
- **Integration tests:** `test/integration/` (SNMP, discovery parity)
- **SnmpKit tests:** Dedicated `test/snmpkit/` directory with ~45 test files
### Mocking
- **`Mox`** — Used for behaviour-based mocks (SNMP, ping, HTTP)
- **`ping_stub.ex`** — Stub for ping behaviour
- **`snmp_kit_mock.ex`** — SNMP client mock
---
## Coverage by Domain
### ✅ Well-Covered Domains
| Domain | Source Files | Test Files | Notes |
|---|---|---|---|
| **Accounts** | 13 | 14+ | Comprehensive auth, TOTP, sessions, HIBP, browser sessions |
| **Agents** | 5 | 8+ | Assignments, tokens, stats, release checker, cloud poller |
| **Alerts** | 2 | 2 | Alert schema + context |
| **API Tokens** | 2 | 2 | Token CRUD |
| **Billing** | 5 | 4 | Stripe client, webhook processor, notifier (missing: `stripe_webhook_event` schema) |
| **Devices** | 14 | 10+ | Core CRUD, firmware, backups, differ, version comparator |
| **GAIIA** | 14 | 13 | Excellent: sync, reconciliation, subscriber matching, impact analysis |
| **Organizations** | 6 | 7+ | Policies, invitations, subscriptions, property tests |
| **Preseem** | 14 | 14 | Full coverage: sync, baselines, insights, fleet intelligence |
| **Profiles** | 7 | 5 | YAML profiles, device/sensor OIDs |
| **SNMP Core** | 30+ | 25+ | Discovery, polling, sanitizer, sensor change, MIB parsing |
| **SNMP Vendors** | 180+ | 180+ | Auto-generated, 1:1 coverage |
| **Workers** | 30 | 22 | Most critical workers covered |
| **Web: Plugs** | 13 | 11 | Auth, rate limiting, security headers, brute force |
| **Web: Controllers** | 25+ | 25+ | API v1, admin, auth flow controllers |
| **Web: LiveViews** | 40+ | 35+ | Most major pages tested |
### ⚠️ Gaps — Modules Without Test Files
#### 🔴 Critical (Needs Tests NOW)
These modules handle auth, billing, alert routing, or data mutations with no dedicated tests:
| Module | Why Critical |
|---|---|
| `Towerops.Monitoring` (context) | Core monitoring orchestration — check creation, status evaluation |
| `Towerops.Monitoring.Executor` | Dispatches monitoring checks to executors |
| `Towerops.Monitoring.Executors.DnsExecutor` | DNS monitoring — untested execution path |
| `Towerops.Monitoring.Executors.HttpExecutor` | HTTP monitoring — untested execution path |
| `Towerops.Monitoring.Executors.TcpExecutor` | TCP monitoring — untested execution path |
| `Towerops.Monitoring.Executors.SnmpInterfaceExecutor` | SNMP interface monitoring |
| `Towerops.Monitoring.Executors.SnmpProcessorExecutor` | SNMP CPU monitoring |
| `Towerops.Monitoring.Executors.SnmpStorageExecutor` | SNMP storage monitoring |
| `Towerops.OnCall.Notifier` | Sends on-call notifications — critical alert path |
| `Towerops.OnCall.Incident` | Incident lifecycle — create, ack, resolve |
| `Towerops.OnCall.EscalationPolicy` | Escalation rule evaluation |
| `Towerops.OnCall.Notification` | Notification delivery schema |
| `Towerops.PagerDuty.Notifier` | External PagerDuty alert dispatch |
| `Towerops.Security.CloudflareClient` | Cloudflare IP banning — security mutation |
| `Towerops.Security.IpBlock` / `IpWhitelist` | IP access control schemas |
| `Towerops.Billing.StripeWebhookEvent` | Stripe event persistence schema |
| `Towerops.Accounts.UserRecoveryCode` | Account recovery — security-critical |
| `Towerops.Accounts.UserToken` | Token schema (tested indirectly via Accounts context but no direct schema test) |
#### 🟠 High Priority
| Module | Why |
|---|---|
| `Towerops.Maintenance` (context) | Maintenance window CRUD and scheduling logic |
| `Towerops.ConfigChanges` (context) | Config change tracking and correlation |
| `Towerops.ConfigChanges.Correlator` | Correlates config changes to alerts |
| `Towerops.Topology.Lldp` | LLDP neighbor discovery logic |
| `Towerops.Topology.Identifier` | Device identification heuristics |
| `Towerops.Netbox.Sync` | External NetBox sync — data import |
| `Towerops.Sonar.Client` / `Sync` | Sonar billing integration |
| `Towerops.Weather.Client` | Weather API integration |
| `Towerops.Workers.EscalationCheckWorker` | Periodic escalation evaluation |
| `Towerops.Workers.CheckWorker` | Monitoring check scheduling |
| `Towerops.Workers.MikrotikBackupWorker` | MikroTik backup orchestration |
| `Towerops.Workers.CloudflareBanWorker` | Security: auto-ban via Cloudflare |
| `ToweropsWeb.Controllers.Api.V1.PagerdutyWebhookController` | Inbound PagerDuty webhooks |
| `ToweropsWeb.Controllers.Api.V1.ActivityController` | Activity feed API |
| `ToweropsWeb.Controllers.Api.V1.CheckResultsController` | Check results API |
| `ToweropsWeb.Controllers.InvitationController` | Org invitation acceptance |
| `ToweropsWeb.Controllers.UserConfirmationController` | Email confirmation flow |
| `ToweropsWeb.Controllers.UserRegistrationController` | User signup flow |
| `ToweropsWeb.Live.OnboardingLive` | New user onboarding |
| `ToweropsWeb.Live.Org.GaiiaMappingLive` | GAIIA device mapping UI |
| `ToweropsWeb.Live.Org.GaiiaReconciliationLive` | GAIIA reconciliation UI |
#### 🟡 Medium Priority
| Module | Why |
|---|---|
| `Towerops.Geocoding` | Address-to-coordinates (read-only helper) |
| `Towerops.GeoIp.Block` / `Location` | GeoIP lookup schemas |
| `Towerops.Vault` | Encryption key management |
| `Towerops.RateLimit` | Rate limiting logic |
| `Towerops.Devices.Event` / `BackupRequest` | Event/request schemas |
| `Towerops.Devices.MikrotikBackup` | Backup schema |
| `Towerops.OnCall.Schedule` / `Layer` / `Override` | On-call schedule schemas |
| `Towerops.Organizations.Membership` | Membership schema |
| `Towerops.SNMP.Interface` / `Sensor` / `Device` schemas | Data schemas |
| `Towerops.Workers.*SyncWorker` (sonar, splynx, visp, weather, netbox) | Sync orchestration |
| `ToweropsWeb.Live.CapacityLive` | Capacity planning page |
| `ToweropsWeb.Live.ConfigTimelineLive` | Config change timeline |
| `ToweropsWeb.Live.ActivityFeedLive` | Activity feed page |
| `ToweropsWeb.Live.MikrotikBackupLive.Compare` | Backup diff viewer |
| `ToweropsWeb.GraphQL.Context` | GraphQL context setup |
#### 🟢 Low Priority
| Module | Notes |
|---|---|
| `Towerops.EctoTypes.EncryptedBinary` | Simple Ecto type (encrypted_map is tested) |
| `Towerops.Settings.ApplicationSetting` | Simple schema |
| `Towerops.Mailer` | Swoosh wrapper |
| `Towerops.Release` | Mix release tasks |
| `Towerops.Repo` | Ecto repo module |
| `Towerops.SNMP.SnmpBehaviour` | Behaviour definition |
| `Towerops.SNMP.PollerBehaviour` | Behaviour definition |
| `Towerops.Monitoring.PingBehaviour` | Behaviour definition |
| `Towerops.Profiles.MibCache` / `ProfileWatcher` | GenServer infrastructure |
| Various SNMP data schemas (readings, stats) | Simple structs |
---
## Top 10 Recommended Tests to Write
### 1. `test/towerops/monitoring_test.exs` — Monitoring Context
**Impact: 🔴 Critical**
Cover: `create_check/2`, `update_check/2`, `delete_check/2`, `list_checks_for_device/1`, check result creation and status evaluation. This is the core monitoring orchestration layer with zero test coverage.
### 2. `test/towerops/monitoring/executor_test.exs` — Check Executor Dispatch
**Impact: 🔴 Critical**
Cover: Executor dispatch logic, timeout handling, error wrapping. Test with mocked executors that the right executor is called for each check type (HTTP, DNS, TCP, SNMP).
### 3. `test/towerops/on_call/notifier_test.exs` — On-Call Notification Delivery
**Impact: 🔴 Critical**
Cover: Notification channel selection (email, SMS, push), retry logic, escalation triggering, de-duplication. Mock external delivery and verify correct notification flow.
### 4. `test/towerops/on_call/incident_test.exs` — Incident Lifecycle
**Impact: 🔴 Critical**
Cover: Incident creation from alert, acknowledgment, resolution, re-escalation on timeout. Verify state machine transitions and escalation policy evaluation.
### 5. `test/towerops/monitoring/executors/http_executor_test.exs` — HTTP Check Execution
**Impact: 🔴 Critical**
Cover: HTTP status checks, response time measurement, SSL verification, timeout handling, redirect following, body content matching. Mock HTTP client.
### 6. `test/towerops/config_changes_test.exs` — Config Change Tracking
**Impact: 🟠 High**
Cover: Change event creation, correlation with alerts/devices, timeline querying, diff computation. Verify correlator links config changes to relevant alerts.
### 7. `test/towerops/maintenance_test.exs` — Maintenance Window Context
**Impact: 🟠 High**
Cover: Window creation/scheduling, overlap detection, active window queries, alert suppression during maintenance. Verify `is_in_maintenance?/2` logic.
### 8. `test/towerops_web/controllers/api/v1/pagerduty_webhook_controller_test.exs` — PagerDuty Webhooks
**Impact: 🟠 High**
Cover: Webhook signature verification, event type handling (trigger, acknowledge, resolve), idempotency, error responses for invalid payloads.
### 9. `test/towerops/security/cloudflare_client_test.exs` — Cloudflare IP Management
**Impact: 🟠 High**
Cover: IP block creation/deletion via Cloudflare API, error handling, rate limiting. Mock Cloudflare API. Verify the ban worker integrates correctly.
### 10. `test/towerops_web/controllers/user_registration_controller_test.exs` — User Registration
**Impact: 🟠 High**
Cover: Successful registration, duplicate email handling, password validation, email confirmation trigger, registration with invitation token. Critical auth flow.
---
## Test Health Notes
- **Async ratio is excellent** — ~81% of test files use `async: true`
- **Property testing is in use** — 55 property tests via StreamData (found in schema validation, Ecto type tests)
- **Fixture coverage is comprehensive** — 14 fixture modules covering all major domains
- **Vendor profile tests are auto-generated** — 180+ vendor tests likely from `mix gen_vendor_tests`
- **The 4 current failures are infrastructure issues**, not logic bugs:
- Agent slug uniqueness collision in test factories → fix: add `System.unique_integer()` to slug generation
- Deadlock in async time series test → fix: set `async: false` or restructure DB access
- **231 excluded tests** — likely tagged `:integration` or `:external` for CI isolation
- **38 skipped tests** — worth auditing to see if any are permanently broken
Reference in a new issue View git blame Copy permalink