Splits the runtime apt installs (gdal-bin, snmp, libsnmp40, locales,
BEAM runtime libs) into k8s/Dockerfile.base, hosted at
codeberg.org/gmcintire/towerops-base:latest. The app Dockerfile now
does FROM that base instead of re-installing gdal (~500 MB) on every
push.
The new build-base workflow rebuilds the base image only when
k8s/Dockerfile.base or the workflow itself changes, weekly via cron
(Sundays 06:00 UTC, with CACHE_BUST=<ISO week> to force apt-get update
on a week boundary), or via workflow_dispatch.
Production workflow now uses buildx + does docker login before the
build so it can pull the private base image.
Catalog-only system for LIDAR-derived 1m DEMs covering Texas. We do not
mirror raster data — the DB stores tile metadata (URL + footprint
geometry) and elevation values are streamed on-demand from public USGS
3DEP / TNRIS Cloud-Optimized GeoTIFFs via GDAL /vsicurl/ HTTP byte-range
reads. Used for future tower coverage line-of-sight calculations.
- Enable PostGIS extension; tile footprints indexed via GiST
- Catalog sync from USGS 3DEP (primary) and TNRIS (fallback for gaps)
- Tile dedup by ST_Contains so TNRIS doesn't duplicate 3DEP coverage
- Single-point reads via gdallocationinfo with tile fallthrough on nodata
- Grid retrieval via gdal_translate AAIGrid with multi-tile mosaicing
- Monthly Oban cron worker keeps catalog fresh
- gdal-bin added to runtime image; geo_postgis dep added
Applies the 11 recommendations from the codebase optimization review.
Module decomposition:
- Extract Towerops.Topology.InferenceEngine: pulls device-role inference
(capability rules, vendor matching, evidence merging) out of the topology
context. Topology now delegates merge_confidence/2,
infer_role_from_capabilities/1, and infer_device_role/1.
- Extract Towerops.Accounts.TOTP: TOTP secret/QR generation, multi-device
verification, recovery code lifecycle, and sudo MFA logic. Accounts
delegates the public TOTP API for backwards compatibility.
- Extract Towerops.Accounts.Sessions: browser session create/list/touch/
revoke/anonymize/expire. Accounts delegates the public session API.
- Extract Towerops.Snmp.Queries: time-series read paths
(sensor readings, interface stats, latest-per-id batches).
- Extract Towerops.Snmp.Monitoring: time-series write paths
(sensor/interface/processor reading inserts, batch inserts via
Repo.insert_all). Snmp.ex shrinks from 2985 to 2742 lines.
DRY refactoring:
- Add per-schema Query modules (Devices.DeviceQuery, Alerts.AlertQuery)
with composable for_organization/for_site/with_status/etc filters;
refactor list_site_devices, count_organization_devices,
count_site_devices, count_site_devices_down, count_organization_alerts
to compose them.
- Introduce create_discovered_checks/6 helper in Snmp; collapses 4 nearly
identical Enum.reduce blocks for sensors/interfaces/processors/storage.
- Unify MAC and ARP evidence collectors behind collect_lookup_evidence/3
(lldp/cdp were already factored).
Pattern matching:
- Replace cond block in infer_role_from_capabilities/1 with a declarative
@role_rules table consulted via Enum.find_value (Elixir 1.19 forbids `in`
with runtime lists in guards, so multi-clause functions weren't an option).
- Replace inline `if since` in get_sensor_readings/2 with a maybe_filter_since
pipe helper.
Performance:
- Replace Enum.each + Repo.insert per evidence row in upsert_link with a
single Repo.insert_all batch (truncates :observed_at to seconds).
Human-centric:
- Decompose build_device_lookup/1 into fetch_lookup_devices, map_ips_to_ids,
map_names_to_ids, map_macs_to_ids.
- Adopt Towerops.Result.map/2 in Snmp.Client.do_get_multiple_sequential
(kept narrow; `with` is preferred over Result.and_then for chained
operations and is already used widely).
CLAUDE.md updates:
- Replace stale main→staging / production-branch deployment notes with the
current flow: PRs deploy to Dokku staging, push to main runs the
ExUnit gate then builds an image; argocd-image-updater rolls production.
- Refresh data-model diagram and references from "Equipment" to "Device"
(post equipment→devices rename migration).
- Correct stale architecture facts: Hammer → in-tree Towerops.RateLimit,
Rust NIF → C NIF (libnetsnmp), gitlab-registry → forgejo-registry,
expand the Oban queue and cron lists, list actual implemented Ecto types.
All 10,228 tests pass.
Adds Forgejo Actions CI/CD workflows:
Production deployment (.forgejo/workflows/production.yml):
- Runs on push to 'production' branch
- Test gates (must pass before deploy):
* All ExUnit tests with coverage
* All e2e Playwright tests
- Only deploys if both test suites pass
- Builds Docker image
- Pushes to git.mcintire.me registry
- Updates k8s/deployment.yaml with new image tag
- FluxCD auto-applies changes
Main branch testing (.forgejo/workflows/test.yml):
- Runs on push to 'main' or pull requests
- Runs ExUnit tests with coverage
- Runs e2e tests
- No deployment (staging still via Dokku)
Updated CLAUDE.md with CI/CD pipeline documentation.
Updated priv/static/changelog.txt with all recent changes from CHANGELOG.txt
in concise, user-facing format. Enhanced CLAUDE.md to document the dual
changelog system (technical CHANGELOG.txt vs user-facing priv/static/changelog.txt)
with clear guidelines on what to include in each and when to update them.
Reduced file size by 62% while preserving all critical information:
- Simplified data model diagram to hierarchical text format
- Removed verbose code examples, keeping essential patterns
- Condensed sections while retaining actionable information
- Streamlined Custom Ecto Types, Browser Navigation, and Memory Leak sections
The condensed version maintains all necessary guidance for AI assistants
working on the project without overwhelming the context window.
Add comprehensive documentation for idiomatic Phoenix LiveView URL state
management to ensure browser back/forward buttons work correctly.
Changes:
- Add design document: docs/plans/2026-02-12-browser-navigation-fix-design.md
- Documents the problem (18 LiveViews missing handle_params/3)
- Provides idiomatic Phoenix patterns using push_patch/2
- Includes migration strategy for updating all LiveViews
- Contains testing protocols and success criteria
- Update CLAUDE.md: Add "Browser Navigation and URL State Management" section
- Documents the three core patterns: tabs, modals, filters
- Establishes critical rules for all future LiveView development
- Provides working code examples for each pattern
- Links to official Phoenix documentation
Key Principles:
- URL is single source of truth for visible state
- Use push_patch/2 in event handlers, never assign for URL-backed state
- Always implement handle_params/3 to validate and process URL params
- Use Phoenix built-ins (no custom helpers needed)
This ensures future LiveView development follows the correct pattern
and browser navigation works as users expect.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Move Oban Web from /oban to /admin/oban
- Move LiveDashboard from /dashboard to /admin/dashboard
- Both remain protected by :require_superuser plug
- Update documentation to reflect new paths
Replaced GenServer-based periodic workers with Oban Cron jobs to improve
pod rollover resilience and simplify architecture.
Worker Changes:
- NeighborCleanupWorker: GenServer → Oban Cron (hourly)
- Cleans stale neighbors, ARP entries, and MAC addresses
- Runs every hour via Oban.Plugins.Cron
- StaleAgentWorker: GenServer → Oban Cron (every minute)
- Detects agents that haven't checked in for 10+ minutes
- Refactored to reduce nesting (extracted helper functions)
- Removed stateful tracking (now stateless, re-evaluates each run)
- AgentLatencyEvaluator: GenServer → Oban Cron (every 5 minutes)
- Latency-based agent reassignment with 20% threshold
- Removed trigger_evaluation/0 (no longer needed)
- JobHealthCheckWorker: NEW Oban Cron worker (every 10 minutes)
- Safety net to recover missing monitor/poller jobs
- Auto-creates jobs for devices with monitoring/SNMP enabled
Infrastructure Changes:
- Removed Monitoring.Supervisor (no longer needed)
- Updated application.ex to remove GenServer workers from supervision tree
- Added Oban.Plugins.Cron to dev.exs and runtime.exs
- All workers now run cluster-wide via PostgreSQL-backed coordination
Test Updates:
- Updated all worker tests to call perform(%Oban.Job{args: %{}})
- Removed GenServer lifecycle tests (start_link, send messages, etc.)
- Removed async sleep calls (no longer needed)
Benefits:
- Better pod rollover resilience (Cron jobs run cluster-wide)
- Simpler architecture (no GenServers for periodic tasks)
- Better observability (all jobs visible in Oban dashboard)
- Safety net for missing jobs (JobHealthCheckWorker)
- Stateless workers (easier to reason about and test)
Documentation:
- Updated CLAUDE.md with Background Job Architecture section
- Documented job types, queues, and resilience features
All tests passing (3,686 tests, 0 failures).
Fixed two critical issues preventing agent communication:
1. Fix Mix.env() runtime error in production
- Replace Mix.env() with Application.get_env(:towerops, :env)
- Add :env config to test.exs
- Mix module not available in production releases
2. Add Protocol Buffers support to agent API endpoints
- GET /api/v1/agent/config now accepts application/x-protobuf
- POST /api/v1/agent/heartbeat now accepts application/x-protobuf
- Added conversion functions for config/equipment/sensors/interfaces
- Maintains JSON backward compatibility as fallback
All agent controller tests passing (14 tests, 0 failures)
Document complete setup process for deploying Towerops to a Talos
Kubernetes cluster, including:
- All required infrastructure prerequisites
- GitLab agent for CI/CD
- Newt for Pangolin forwarding
- Secrets management with 1Password
- FluxCD GitOps configuration
- Deployment process and troubleshooting
