Commit graph

74 commits

Author SHA1 Message Date
efaf5558ff refactor: convert 6 Gleam modules to idiomatic Elixir with TDD (#196)
Phase 1: Foundation Types (100% complete)
- query_helpers: SQL LIKE sanitization with pipe operators
- numeric: Integer parsing with pattern matching guards
- result: Pure Elixir Result monad (map, and_then, unwrap_or)

Phase 2: Ecto Domain Types (100% complete)
- ip_address: IPv4/IPv6 validation using :inet directly
- mac_address: Multi-format MAC parsing (colon/hyphen/dot/compact)
- snmp_oid: OID parsing/manipulation with recursive pattern matching

All 198 tests passing across converted modules.
API changed from Gleam-style {:some/:none to idiomatic {:ok/:error.
Refactored parse_numeric_oid to use with statement, reducing nesting depth.

Reviewed-on: graham/towerops-web#196
2026-03-28 09:52:07 -05:00
39039a4455 fix: remove stale oban hex repo references from mix.lock (#195)
The previous dependency update left mix.lock with references to the
"oban" hex repository for vendored packages (oban_pro, oban_web, oban_met).
This caused production deployment failures with error:
"Unknown repository 'oban', add new repositories with the mix hex.repo add task"

Since these packages are vendored from vendor/ directory, they should not
have hex repo references in the lock file. Removed the hex entries entirely
and regenerated the lock file with only the vendored path dependencies.

Reviewed-on: graham/towerops-web#195
2026-03-27 17:18:48 -05:00
0399b30d21 Update Elixir dependencies (#194)
Updated packages:
- bandit: 1.10.3 → 1.10.4
- phoenix_live_view: 1.1.27 → 1.1.28
- phoenix_pubsub_redis: 3.0.1 → 3.1.0
- oban_met: 1.0.6 → 1.1.0 (vendored)
- oban_web: 2.11.8 → 2.12.1 (vendored)

All tests passing (8923 tests, 0 failures).

Reviewed-on: graham/towerops-web#194
2026-03-27 17:13:12 -05:00
11f9c4450c done i think (#177)
Reviewed-on: graham/towerops-web#177
2026-03-26 10:23:26 -05:00
7656052395 update deps (#173)
Reviewed-on: graham/towerops-web#173
2026-03-25 16:35:12 -05:00
f703e61b12 fix: code quality improvements and test reliability (#139)
Various code quality improvements and fixes:

- Settings: return error instead of auto-creating unknown settings
- PagerDuty: proper backoff timing with to_timeout()
- Gaiia sync: remove unnecessary transaction wrapper
- HTTP/UISP: simplify test plug setup
- Agent channel: tagged tuple config for check types
- SNMP: fix Task.yield/shutdown pattern, remove stale preload
- Discovery: simplify error handling
- Tests: bump executor timeouts for CI reliability
- Dockerfile: remove unnecessary --force compile
- Renovate/CI: config cleanup

Split from #135.

Reviewed-on: graham/towerops-web#139
2026-03-24 09:12:59 -05:00
e54ee75513 Rewrite protobuf encoding/decoding in pure Gleam with integrated validation (#104)
Replace the protobuf hex package (agent.pb.ex) and hand-written validator
(validator.ex) with a pure Gleam implementation: wire format primitives,
all 23 message types, encode/decode functions, and validation merged into
decode. Thin Elixir wrappers in agent.ex preserve the existing struct API.

- Wire format: varint, length-delimited, double (IEEE 754 via FFI), tag encode/decode
- Types: all proto3 message types as Gleam custom types with enums and oneof
- Decode: field-level parsing with integrated validation (UUID, IP, hostname, limits)
- Encode: proto3 default omission, bytes_tree concatenation, map/repeated fields
- Wrappers: Elixir structs with encode/decode delegating to Gleam, enum atom conversion
- Remove {:protobuf, "~> 0.12"} dependency

Reviewed-on: graham/towerops-web#104
2026-03-21 16:08:40 -05:00
348975dcc9 Rewrite polling_offset, changelog_parser, and user_agent_parser in Gleam (#103)
Migrate three pure-function modules from Elixir to Gleam:

- polling_offset: deterministic hash-based polling offset (pure Gleam, no FFI)
- changelog_parser: text parsing logic in Gleam, thin Elixir wrapper for File I/O
- user_agent_parser: regex-based UA parsing in Gleam with Erlang FFI for
  atom-keyed map conversion

Add gleam_regexp dependency for regex support in Gleam modules.
Update all call sites and tests to use Gleam module atoms.

Reviewed-on: graham/towerops-web#103
2026-03-21 10:11:06 -05:00
6859525a1b Integrate Gleam into the build pipeline and rewrite three pure-function modules (#98)
Set up mix_gleam archive, gleam_stdlib, and gleeunit deps. Add Gleam
compiler to all Dockerfiles, CI workflows, nix shell, and .tool-versions.

Rewrite Numeric, QueryHelpers, and URLValidator from Elixir to Gleam with
full ExUnit test coverage. Update all callers to use the Gleam modules
directly via :towerops@module syntax. Remove duplicate sanitize_like/1
private functions in snmp.ex and trace.ex.

Reviewed-on: graham/towerops-web#98
2026-03-21 07:30:36 -05:00
36cb080823 update deps (#82)
Reviewed-on: graham/towerops-web#82
2026-03-19 11:26:22 -05:00
b0d645148a
chore: update dependencies
- phoenix_live_view 1.1.26 -> 1.1.27
- stream_data 1.2.0 -> 1.3.0
- tidewave 0.5.5 -> 0.5.6
- pre-commit hooks configuration
2026-03-13 16:11:27 -05:00
05d79287bc
chore: remove sobelow dependency 2026-03-12 16:03:08 -05:00
72f81c572d
add real-time GraphQL API with time-series queries and subscriptions
- add absinthe_phoenix for WebSocket subscription support
- create GraphQL WebSocket socket with API token auth at /socket/graphql
- add time-series queries: deviceSensors, sensorReadings, interfaceTraffic, checkResults
- add subscriptions: deviceStatusChanged, alertEvent, sensorReadingsUpdated
- wire subscription triggers into agent_channel and alerts contexts
- add org-scoped access control for sensors/interfaces via join queries
- add 17 tests covering queries, org isolation, empty data edge cases, and auth
- update GraphQL API docs page with time-series and subscription sections
2026-03-12 10:24:39 -05:00
3b97e437f2
fix: add castore dep and sanitize sync error messages
Add castore as a direct dependency to provide CA certificates in
environments where the OS trust store is not available (Dokku staging).

Stop leaking raw system errors (like CA trust store details) to the
UI by logging the full error server-side and showing a generic
user-friendly message instead.
2026-03-11 10:41:09 -05:00
d811bcb503
feat: add ErrorTrackerNotifier and fix PagerDuty notification handling
- Add error_tracker_notifier for email alerts on exceptions
  * Sends alerts to graham@towerops.net
  * Configured in production only (gracefully shuts down in dev/test)
  * Includes throttling to prevent duplicate error spam

- Fix AlertNotificationWorker when PagerDuty not configured
  * Worker now handles :not_configured gracefully instead of failing
  * Prevents unnecessary retries and error noise
  * Logs as debug when PagerDuty integration isn't set up

Files changed:
- mix.exs, mix.lock
- config/runtime.exs
- lib/towerops/application.ex
- lib/towerops/workers/alert_notification_worker.ex
- CHANGELOG.txt, priv/static/changelog.txt
2026-03-09 13:14:30 -05:00
f473d1e920
update deps 2026-03-08 12:23:59 -05:00
01ebb8968d
chore: filter out external uptime monitor HEAD requests from logs 2026-03-04 16:48:59 -06:00
61586d14e2
chore: update all hex dependencies
Upgrades: phoenix 1.8.4, phoenix_live_view 1.1.26, ecto_sql 3.13.5,
bandit 1.10.3, credo 1.7.17, swoosh 1.22.1, yaml_elixir 2.12.1,
styler 1.11.0, error_tracker 0.8.0, finch 0.21.0, honeybadger 0.25.0.

Fix site_aggregation_test assertion to match actual DB default (NULL,
not 0) for account_count on skipped sites.
2026-03-04 13:30:32 -06:00
998bdf39b8
update 2026-02-18 10:19:56 -06:00
c92dfff6c9
Add ErrorTracker, remove Honeybadger email notice filter
Replace the HoneybadgerNoticeFilter that emailed raw stacktraces on
every exception with ErrorTracker for self-hosted error tracking.
Honeybadger itself is retained. ErrorTracker dashboard is mounted at
/admin/errors behind the superuser auth wall.
2026-02-17 08:13:34 -06:00
146f5745cf
fix: resolve compilation errors, test failures, and credo issues
- Escape HEEx template braces in GraphQL/API docs with raw(~S[...])
- Fix test assertions for updated marketing copy and UI text
- Extract helper functions in GraphQL resolvers to reduce nesting depth
- Create shared ErrorHelpers module for API controllers
- Fix ETS race condition in brute force whitelist cache for async tests
- Fix property test generators to use ASCII instead of printable unicode
- Add alert_severity helper to site_live/show
- Update accounts fixtures for explicit user confirmation
2026-02-14 12:23:10 -06:00
07a6128877
feat: add checks tab to device detail page (TDD cycle 1)
- Add 'Checks' tab link to device navigation
- Display empty state when no checks configured
- Add tests for tab visibility and empty state
- Fix unused variable warnings in monitoring context
- Remove unused CheckResult alias and module attributes
- Add stub for legacy get_latency_data for backward compatibility

Tests: 2 new tests added, all passing
Following TDD: RED-GREEN cycle verified for each feature
2026-02-12 14:50:14 -06:00
4d73e77f3a
Add 404-based brute force protection system
Implements automated detection and blocking of IPs exhibiting scanning behavior (5+ unique 404s within 1 minute).

Key features:
- Progressive ban escalation (5 min → 1 hour → permanent)
- CIDR range and exact IP whitelisting
- Redis-backed 404 path tracking with 60s TTL
- Cloudflare WAF integration for permanent bans
- Admin UI for whitelist and blocked IP management
- Oban cron jobs for cleanup (expired bans, stale violations)
- ETS caching for whitelist performance

Components:
- Plug: ToweropsWeb.Plugs.BruteForceProtection
- Context: Towerops.Security.BruteForce
- Schemas: IpBlock, IpWhitelist
- Workers: CloudflareBanWorker, ExpiredBanCleanupWorker, StaleViolationCleanupWorker
- Admin UI: /admin/security (superuser only)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-10 16:31:48 -06:00
6b4c098de8
fix warnings 2026-02-08 10:50:27 -06:00
24236dbe30
update oban_web 2026-02-08 10:11:53 -06:00
d431931cfe
update hammer 2026-02-08 10:08:31 -06:00
38eeb1c7b8
fix: improve agent polling and SNMP testing
Changes:
- Fix traffic graph to handle nil interface octets gracefully
- Add comprehensive tests for nil octets scenarios
- Fix agent polling query to support devices assigned directly to org (left join on sites)
- Fix test SNMP connection to inherit credentials from site/org
- Refactor agent_channel process_polling_result to reduce nesting (Credo)
- Add alias for JobCleanupTask in application.ex (Credo)

This fixes crashes when interface stats have nil octets and ensures
devices without a site assignment can be polled by agents.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-05 12:34:28 -06:00
23af86ba73
add mikrotik device handling 2026-02-01 16:42:39 -06:00
02833d0b44 Add rate limiting to auth and API endpoints using Hammer 2026-02-01 14:59:54 -06:00
c856b2142c
Add timezone validation to reject invalid IANA timezone strings
Prevents invalid timezone values from being stored in the database
by validating against Elixir's time zone database during registration.
2026-02-01 11:56:04 -06:00
61a06fc11c
Add firmware version tracking system
- Add firmware context module with upsert, query, and logging functions
- Add FirmwareVersionFetcherWorker to fetch MikroTik RSS daily
- Add Oban cron schedules (2 AM dev, 4 AM prod)
- Add version change detection to Discovery module
- Track firmware history with PubSub broadcasts
- All tests passing

Phase 3-5 of firmware tracking implementation complete.
Next: LiveView UI indicators.
2026-02-01 10:46:27 -06:00
feed25080c
add HIBP password check 2026-02-01 08:57:01 -06:00
2fab08f5f8
forgot password flow 2026-01-31 17:03:22 -06:00
5286d8acba nif ci fix complie in ci 2026-01-30 13:06:39 -06:00
836749ba6b complete overhaul of snmp engine 2026-01-30 10:41:07 -06:00
d29fb8cfd7 totp fixes 2026-01-28 16:16:59 -06:00
0995abbccb Add mandatory TOTP MFA 2026-01-28 15:09:38 -06:00
220f1edce3 cookie improvements 2026-01-28 12:30:28 -06:00
f350a9ade4 test cleanup 2026-01-28 10:31:22 -06:00
124d68d28d
credo fixes 2026-01-27 09:12:22 -06:00
bb86dec559
add oban pro 2026-01-26 17:07:42 -06:00
d0946c3cd0
refactor: simplify job architecture from Oban coordinators to direct workers
Removes unnecessary two-layer architecture (Oban Coordinator → GenServer)
in favor of direct Oban workers that perform the actual work.

Changes:
- Replace DevicePollerCoordinator + PollerWorker with DevicePollerWorker
- Replace DeviceMonitorCoordinator + DeviceMonitor with DeviceMonitorWorker
- Simplify Monitoring.Supervisor (removed Registries, DynamicSupervisors)
- Remove all Exq dependencies (workers, supervisor, mix deps)
- Convert DiscoveryWorker from Exq to Oban pattern
- Update Devices context to auto-start/stop monitoring and polling
- Update all LiveView callers to use new Oban enqueue pattern
- Fix all tests to use Oban.Job struct instead of raw IDs

Benefits:
- Simpler codebase (~850 lines removed)
- Better reliability (Oban handles retries, failures)
- Lower memory (no persistent GenServers)
- Better observability (work visible in Oban dashboard)
- Cluster-wide coordination via PostgreSQL
- Extensive PubSub usage for real-time events

Files deleted:
- lib/towerops/workers/device_monitor_coordinator.ex
- lib/towerops/workers/device_poller_coordinator.ex
- lib/towerops/snmp/poller_worker.ex
- lib/towerops/monitoring/device_monitor.ex
- lib/towerops/workers/monitor_worker.ex
- lib/towerops/workers/poll_worker.ex
- lib/towerops/exq_supervisor.ex
- All related test files

Files created:
- lib/towerops/workers/device_poller_worker.ex
- lib/towerops/workers/device_monitor_worker.ex

All 3,686 tests passing.
2026-01-24 16:36:57 -06:00
29593ac734
refactor: migrate from etcd to Oban for distributed job coordination
Replaces etcd-based distributed locking with Oban's PostgreSQL-backed job queue.
This simplifies the architecture by eliminating the need for a separate etcd cluster
while providing better reliability and observability.

Changes:
- Add Oban dependency and migration (oban_jobs table)
- Create DevicePollerCoordinator and DeviceMonitorCoordinator Oban workers
- Remove EtcdCoordinator and EtcdLock modules
- Update application supervisor to start Oban
- Configure Oban with pollers (50 workers) and monitors (50 workers) queues
- Remove etcd StatefulSet from Kubernetes manifests
- Update monitoring supervisor documentation

Benefits:
- Simpler architecture (no etcd cluster to manage)
- PostgreSQL-based (uses existing database)
- Built-in uniqueness prevents duplicate jobs cluster-wide
- Better observability with Oban Web UI
- Automatic job rescue on node crashes
- Easier local development (no etcd required)

What was removed:
- etcd StatefulSet (3 pods)
- EtcdCoordinator module (320 lines)
- EtcdLock module (158 lines)
- eetcd dependency

All 3,686 tests passing.
2026-01-24 16:12:27 -06:00
f084e09ea1
add multiple cloud agents 2026-01-23 18:04:01 -06:00
2b78b1a2d3
snmpkit overhaul and etcd 2026-01-23 16:23:57 -06:00
518b49318c
routeros fix 2026-01-23 13:16:02 -06:00
a0ba9285dd
bring in snmpkit 2026-01-23 12:52:17 -06:00
7c660169b7
better mib handling and tests 2026-01-21 10:07:07 -06:00
1f123bbeb9
add honeybadger and snmp overhaul 2026-01-18 16:16:08 -06:00
e4b3778da4
exq improvements 2026-01-18 11:50:19 -06:00