towerops/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

CRITICAL: Read AGENTS.md First

Before starting any work in this repository, you MUST read AGENTS.md in the project root.

• AGENTS.md contains mandatory Phoenix/LiveView/Elixir coding guidelines AND project-specific patterns for this codebase
• Covers: Elixir/OTP conventions, Phoenix/LiveView best practices, browser navigation URL state, JS hook memory management, form handling patterns, test guidelines, quality/security checks
• These guidelines take precedence over general development practices when there's a conflict
• Always read AGENTS.md at the start of a new conversation or when resuming work

Project Overview

Towerops is a Phoenix 1.8 web application built with Elixir, using Ecto for database operations (PostgreSQL), LiveView for real-time interactions, and Tailwind CSS v4 for styling.

Data Model Relationships

User (Accounts) → owns/member of → Organization (Organizations)
                                    ├─ Site (Sites) → Equipment
                                    ├─ Equipment → SNMPDevice, MonitoringCheck, Alert
                                    └─ AgentToken → AgentAssignment → Equipment

Key Relationships:
- User can own/belong to multiple Organizations
- Organization has default_agent_token_id (optional)
- Equipment belongs to both Site and Organization (denormalized)
- Equipment can be assigned to one AgentToken via AgentAssignment
- Equipment has one SNMPDevice with Sensors and Interfaces
- Equipment has many MonitoringChecks (polling results) and Alerts
- AgentToken authenticates remote agents for local SNMP polling

Note: Update this diagram when making data model changes.

Essential Commands

Setup and Development

• mix setup - Install dependencies, create/migrate database, build assets
• mix phx.server - Start Phoenix server (http://localhost:4000)
• iex -S mix phx.server - Start server with IEx shell

Testing and Quality

• mix test - Run all tests
• mix test --failed - Re-run failed tests
• mix test --cover - Run with coverage (target: 90% minimum)
• mix precommit - Run before committing: compiles with warnings as errors, formats, runs tests
• mix dialyzer - Static type analysis

Database

• mix ecto.create/migrate/reset - Database operations
• mix ecto.gen.migration name_using_underscores - Generate migration

Assets

• mix assets.build/deploy - Build CSS/JS assets (auto-rebuilds on save in dev)

Architecture

Application Structure

Standard Phoenix conventions: business logic in lib/towerops/, web interface in lib/towerops_web/.

Key Configuration:

• Binary IDs: UUID primary keys by default (binary_id: true)
• Timestamps: :utc_datetime for all timestamps
• Web server: Bandit adapter
• HTTP client: :req library (Req module) - ONLY approved client
• Ecto repos: [Towerops.Repo]

Web Layer

All LiveViews get these imports via html_helpers/0:

• ToweropsWeb.CoreComponents - Core UI (<.button>, <.input>, <.form>)
• ToweropsWeb.Layouts, Phoenix.LiveView.JS, Gettext, verified routes (~p)

Rate Limiting

Uses Hammer (ETS-backed):

• Auth endpoints: 10 req/min per IP (/users/log-in, /users/register, TOTP)
• API v1: 1000 req/min per IP (/api/v1/*)
• Admin API: Not rate limited (superuser only)
• Returns 429 Too Many Requests with Retry-After header
• Disabled in test: config :towerops, :rate_limiting_enabled, false

Asset Pipeline

• Tailwind CSS v4: Uses @import "tailwindcss" in assets/css/app.css (no config file)
• esbuild: Bundles assets/js/app.js
• Import all vendor assets into app.js/app.css - no external src/href in layouts
• No inline <script> tags - use LiveView hooks

Custom Ecto Types

Avoid "primitive obsession" by using custom types:

Available Types:

1. Towerops.EctoTypes.IpAddress - IPv4/IPv6 validation, struct with version/tuple
2. Towerops.EctoTypes.EmailAddress - Normalized emails (Planned)
3. Towerops.EctoTypes.MacAddress - MAC address normalization (Planned)

Pattern: Implement Ecto.Type with type/0, cast/1, load/1, dump/1.

When to use:

• ✅ Domain concepts with validation rules (emails, IPs, phone numbers)
• ✅ Values needing normalization (case-insensitive emails, MAC addresses)
• ✅ Multiple representations (IP strings vs tuples)
• ❌ Simple strings, primitives, one-off validations

Migration: No database changes needed - column stays primitive, Ecto handles conversion.

Background Jobs (Oban)

PostgreSQL-backed with cluster-wide coordination.

Queues:

• default (10) - General tasks
• discovery (10) - SNMP discovery
• pollers (50) - SNMP polling (per-device)
• monitors (50) - Health checks (per-device)
• maintenance (5) - Periodic cleanup

Key Workers:

1. Self-Scheduling (per-device):

   • DeviceMonitorWorker - Health checks (60s default)
   • DevicePollerWorker - SNMP data collection (60s default)
   • Auto-created/cancelled when device settings change

2. Oban Cron (cluster-wide):

   • NeighborCleanupWorker - Hourly stale data cleanup
   • StaleAgentWorker - Detect agents offline 10+ minutes (every minute)
   • AgentLatencyEvaluator - Latency-based reassignment (every 5 minutes)
   • JobHealthCheckWorker - Recover missing jobs (every 10 minutes)

Resilience: Oban Cron uses PostgreSQL locking, self-scheduling recovered every 10 minutes. Dashboard: /admin/oban (superuser), /dev/dashboard → Oban tab (dev)

SNMP Polling

Two mechanisms:

1. Discovery (Towerops.Snmp.Discovery) - One-time/manual, collects full device info
2. Polling (DevicePollerWorker) - Continuous (60s default), time-series data

Load Distribution: Staggered polling using hash-based offsets:

• offset = :erlang.phash2(device_id) |> rem(interval_seconds)
• Prevents thundering herd, stable across restarts
• Implementation: lib/towerops/workers/polling_offset.ex

MikroTik API Integration

RouterOS API access alongside SNMP.

Architecture:

• 3-tier credential cascade: Organization → Site → Device (like SNMP)
• Encryption: Passwords encrypted at rest (Cloak AES-256-GCM)
• Detection: Auto-detected from SNMP manufacturer field
• Transport: API-SSL (8729, default) or plain API (8728, insecure)
• Security: Plain API blocked for cloud pollers, passwords require CLOAK_KEY env var

Key Files:

• lib/towerops/vault.ex - Cloak vault
• lib/towerops/ecto_types/encrypted_binary.ex - Encrypted field type
• lib/towerops/devices.ex - Config resolution, propagation
• priv/proto/agent.proto - MikrotikDevice protobuf messages

Credential Resolution: Each field resolves independently up the hierarchy.

MIB Name Resolution (Rust NIF)

Uses Rust NIF via Rustler for fast MIB resolution (replaces unreliable Erlang SNMP).

• NIF Module: lib/towerops_native.ex
• Rust: native/towerops_native/src/mib.rs (uses snmptranslate command)
• MIB Files: priv/mibs/ (570+ vendor and standard MIBs)
• Dependencies: brew install net-snmp (macOS), apt-get install libsnmp-dev snmp-mibs-downloader (Docker)
• Usage: Try ToweropsNative.resolve_oid/1 first, fallback to SnmpKit.resolve/1

Note: SnmpKit still used for SNMP protocol operations (get, walk, etc.).

Admin Features

User Impersonation

• Location: /admin/users (superuser only)
• Superusers CAN impersonate other superusers (feature, not bug)
• All events logged to audit_logs
• Stop link appears in user menu when active

GeoIP Database

• Import MaxMind GeoLite2-City for IP-based country/city detection (GDPR cookies)
• Local: make geoip-import DIR=~/Downloads/GeoLite2-City-CSV_20260127/
• Production: make geoip-import-prod DIR=... (requires TOWEROPS_KEY)
• API: POST /admin/api/geoip/import (superuser only)

Project-Specific Constraints

See AGENTS.md for the full set of coding constraints. Key reminders:

• Use mix precommit before committing (formats, compiles with warnings-as-errors, runs tests)
• Use :req (Req) for HTTP — never :httpoison, :tesla, :httpc
• Never use daisyUI — custom Tailwind components only
• Never put Ecto queries directly in LiveViews — use context modules
• Always run mix format after Elixir changes

API Documentation

• Available at /docs/api (Tailwind UI Protocol template)
• Update controller @doc comments and /docs/api template when changing endpoints
• Wrap examples in <%= raw(~S"""...""") %> to prevent HEEx parsing

Kubernetes Deployment

Prerequisites: cert-manager, Traefik, MetalLB, FluxCD, GitLab Agent, NFS Provisioner

Secrets (1Password, towerops namespace):

• gitlab-registry - Docker credentials
• towerops-secrets - RELEASE_COOKIE, SECRET_KEY_BASE, CLOAK_KEY
• towerops-db - PostgreSQL connection
• towerops-aws - AWS credentials

Create CLOAK_KEY:

# Generate and store in 1Password first
CLOAK_KEY=$(openssl rand -base64 32)

# Create secret
kubectl create secret generic towerops-secrets \
  --from-literal=RELEASE_COOKIE=$(openssl rand -base64 32) \
  --from-literal=SECRET_KEY_BASE=$(mix phx.gen.secret) \
  --from-literal=CLOAK_KEY="$CLOAK_KEY" \
  -n towerops

# Or patch existing
kubectl patch secret towerops-secrets -n towerops \
  --type='json' \
  -p="[{'op': 'add', 'path': '/data/CLOAK_KEY', 'value': '$(echo -n "$CLOAK_KEY" | base64)'}]"

# Restart pods
kubectl rollout restart deployment/towerops -n towerops

Important: Losing CLOAK_KEY makes encrypted data unrecoverable.

Deployment: FluxCD auto-applies k8s/ manifests. Manual: kubectl apply -k k8s/

Deployment

Branch-Based Deployment Strategy

Towerops uses branch-based deployment with Forgejo CI/CD:

Staging (Dokku):

• Push to main branch triggers automatic deploy to Dokku staging server
• Direct git push to Dokku (no Docker build)
• Buildpack-based deployment (detects Elixir/Phoenix)
• URL: staging.towerops.app (or configured Dokku domain)

Production (Kubernetes):

• Push to production branch triggers Docker build + k8s deployment
• Builds Docker image from k8s/Dockerfile
• Pushes to container registry
• Updates k8s/deployment.yaml with new image tag
• FluxCD detects change and applies to cluster

Deploying Changes

To staging:

git push origin main
# CI automatically deploys to Dokku
# Watch: https://git.mcintire.me/graham/towerops-web/actions

To production:

# Merge main → production
git checkout production
git merge main
git push origin production

# Or fast-forward if no conflicts
git push origin main:production

# CI builds Docker image and updates k8s manifests
# FluxCD applies within ~5 minutes

Direct Dokku deploy (bypass CI):

# Add Dokku remote (one-time)
git remote add dokku dokku@204.110.191.231:towerops

# Force push to deploy
git push dokku main:main --force

Manual production deploy:

# Apply k8s manifests directly
kubectl apply -k k8s/

# Or trigger rollout restart
kubectl rollout restart deployment/towerops -n towerops

Deployment Verification

Staging:

# Check Dokku logs
ssh dokku@204.110.191.231 logs towerops --tail 100

# Check app status
ssh dokku@204.110.191.231 ps:report towerops

Production:

# Check pod status
kubectl get pods -n towerops

# Check logs
kubectl logs -n towerops -l app=towerops --tail=100 -f

# Check deployment status
kubectl rollout status deployment/towerops -n towerops

Rollback

Staging (Dokku):

# Revert git commit, push to main
git revert <bad-commit>
git push origin main

# Or rebuild previous release
ssh dokku@204.110.191.231 ps:rebuild towerops <release-id>

Production (Kubernetes):

# Rollback deployment
kubectl rollout undo deployment/towerops -n towerops

# Or revert git commit on production branch
git checkout production
git revert <bad-commit>
git push origin production

Common Patterns

Pagination in LiveView

Use <.pagination> component from CoreComponents:

def handle_params(params, _url, socket) do
  page = params |> Map.get("page", "1") |> String.to_integer()
  per_page = 20

  all_items = MyContext.list_items(org_id)
  total_count = length(all_items)
  total_pages = ceil(total_count / per_page)
  page = max(1, min(page, max(1, total_pages)))

  offset = (page - 1) * per_page
  items = Enum.slice(all_items, offset, per_page)

  {:noreply, socket
   |> assign(:items, items)
   |> assign(:pagination, %{page: page, per_page: per_page,
                             total_count: total_count, total_pages: total_pages})}
end

Template:

<.pagination meta={@pagination} path={~p"/devices"} params={%{"tab" => @tab}} />

For large datasets (>10K), use database-level limit/offset instead of in-memory slicing.

Testing Notes

• Never use npm — esbuild is built into Phoenix
• Run cargo fmt before committing Rust changes
• Never open test coverage HTML files — read results in terminal
• See AGENTS.md for full test guidelines, SNMP mocking patterns, and LiveView test helpers

Dialyzer

• mix dialyzer - Run analysis (builds PLT first time)
• mix dialyzer --format dialyzer - Detailed error locations
• Never suppress valid warnings — fix root cause
• PLT files in priv/plts/ — don't commit

Changelog

Two changelog files to maintain:

CHANGELOG.txt (Technical/Internal)

After every code change, append to CHANGELOG.txt:

• Date (YYYY-MM-DD)
• Short description (e.g. "fix: update last_snmp_poll_at for agent-polled devices")
• Files changed and brief explanation
• Technical details about implementation

Keep reverse chronological (newest at top). Never remove entries.

priv/static/changelog.txt (User-Facing)

After significant changes, update priv/static/changelog.txt:

• Group changes by date (YYYY-MM-DD)
• Brief bullet points without code specifics
• Generic descriptions (no function names, modules, file paths, or language details)
• Focus on what changed for the user, not how it was implemented
• Follow existing pattern: "* Feature description" or "* Bug fix: brief description"

Example conversions:

• ❌ "fix: update last_snmp_poll_at in agent_channel.ex"

• ✅ "Poll time tracking improvements"

• ❌ "feat: add discover_wireless_sensors/1 to MikroTik vendor module"

• ✅ "Enhanced wireless monitoring for MikroTik devices"

When to update user-facing changelog:

• User-visible features or improvements
• Bug fixes affecting user experience
• New vendor/device support
• Performance improvements
• Security improvements
• Skip: Test-only changes, internal refactoring, documentation updates