towerops/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## CRITICAL: Read AGENTS.md First

**Before starting any work in this repository**, you MUST read `AGENTS.md` in the project root.

- `AGENTS.md` contains comprehensive Phoenix/LiveView/Elixir guidelines that are mandatory for this project
- These guidelines take precedence over general development practices when there's a conflict
- Always read `AGENTS.md` at the start of a new conversation or when resuming work
- Follow the project-specific patterns, conventions, and constraints documented there

## Project Overview

Towerops is a Phoenix 1.8 web application built with Elixir, using Ecto for database operations (PostgreSQL), LiveView for real-time interactions, and Tailwind CSS v4 for styling.

### Data Model Relationships

```
User (Accounts) → owns/member of → Organization (Organizations)
                                    ├─ Site (Sites) → Equipment
                                    ├─ Equipment → SNMPDevice, MonitoringCheck, Alert
                                    └─ AgentToken → AgentAssignment → Equipment

Key Relationships:
- User can own/belong to multiple Organizations
- Organization has default_agent_token_id (optional)
- Equipment belongs to both Site and Organization (denormalized)
- Equipment can be assigned to one AgentToken via AgentAssignment
- Equipment has one SNMPDevice with Sensors and Interfaces
- Equipment has many MonitoringChecks (polling results) and Alerts
- AgentToken authenticates remote agents for local SNMP polling
```

**Note**: Update this diagram when making data model changes.

## Essential Commands

### Setup and Development
- `mix setup` - Install dependencies, create/migrate database, build assets
- `mix phx.server` - Start Phoenix server (http://localhost:4000)
- `iex -S mix phx.server` - Start server with IEx shell

### Testing and Quality
- `mix test` - Run all tests
- `mix test --failed` - Re-run failed tests
- `mix test --cover` - Run with coverage (target: 90% minimum)
- `mix precommit` - **Run before committing**: compiles with warnings as errors, formats, runs tests
- `mix dialyzer` - Static type analysis

### Database
- `mix ecto.create/migrate/reset` - Database operations
- `mix ecto.gen.migration name_using_underscores` - Generate migration

### Assets
- `mix assets.build/deploy` - Build CSS/JS assets (auto-rebuilds on save in dev)

## Architecture

### Application Structure

Standard Phoenix conventions: business logic in `lib/towerops/`, web interface in `lib/towerops_web/`.

**Key Configuration**:
- **Binary IDs**: UUID primary keys by default (`binary_id: true`)
- **Timestamps**: `:utc_datetime` for all timestamps
- **Web server**: Bandit adapter
- **HTTP client**: `:req` library (Req module) - ONLY approved client
- **Ecto repos**: `[Towerops.Repo]`

### Web Layer

All LiveViews get these imports via `html_helpers/0`:
- `ToweropsWeb.CoreComponents` - Core UI (`<.button>`, `<.input>`, `<.form>`)
- `ToweropsWeb.Layouts`, `Phoenix.LiveView.JS`, Gettext, verified routes (`~p`)

### Rate Limiting

Uses Hammer (ETS-backed):
- **Auth endpoints**: 10 req/min per IP (`/users/log-in`, `/users/register`, TOTP)
- **API v1**: 1000 req/min per IP (`/api/v1/*`)
- **Admin API**: Not rate limited (superuser only)
- Returns `429 Too Many Requests` with `Retry-After` header
- Disabled in test: `config :towerops, :rate_limiting_enabled, false`

### Asset Pipeline

- **Tailwind CSS v4**: Uses `@import "tailwindcss"` in `assets/css/app.css` (no config file)
- **esbuild**: Bundles `assets/js/app.js`
- Import all vendor assets into app.js/app.css - no external src/href in layouts
- No inline `<script>` tags - use LiveView hooks

### Custom Ecto Types

Avoid "primitive obsession" by using custom types:

**Available Types**:
1. **`Towerops.EctoTypes.IpAddress`** - IPv4/IPv6 validation, struct with version/tuple
2. **`Towerops.EctoTypes.EmailAddress`** - Normalized emails _(Planned)_
3. **`Towerops.EctoTypes.MacAddress`** - MAC address normalization _(Planned)_

**Pattern**: Implement `Ecto.Type` with `type/0`, `cast/1`, `load/1`, `dump/1`.

**When to use**:
- ✅ Domain concepts with validation rules (emails, IPs, phone numbers)
- ✅ Values needing normalization (case-insensitive emails, MAC addresses)
- ✅ Multiple representations (IP strings vs tuples)
- ❌ Simple strings, primitives, one-off validations

**Migration**: No database changes needed - column stays primitive, Ecto handles conversion.

### Background Jobs (Oban)

PostgreSQL-backed with cluster-wide coordination.

**Queues**:
- `default` (10) - General tasks
- `discovery` (10) - SNMP discovery
- `pollers` (50) - SNMP polling (per-device)
- `monitors` (50) - Health checks (per-device)
- `maintenance` (5) - Periodic cleanup

**Key Workers**:
1. **Self-Scheduling** (per-device):
   - `DeviceMonitorWorker` - Health checks (60s default)
   - `DevicePollerWorker` - SNMP data collection (60s default)
   - Auto-created/cancelled when device settings change

2. **Oban Cron** (cluster-wide):
   - `NeighborCleanupWorker` - Hourly stale data cleanup
   - `StaleAgentWorker` - Detect agents offline 10+ minutes (every minute)
   - `AgentLatencyEvaluator` - Latency-based reassignment (every 5 minutes)
   - `JobHealthCheckWorker` - Recover missing jobs (every 10 minutes)

**Resilience**: Oban Cron uses PostgreSQL locking, self-scheduling recovered every 10 minutes.
**Dashboard**: `/admin/oban` (superuser), `/dev/dashboard` → Oban tab (dev)

### SNMP Polling

**Two mechanisms**:
1. **Discovery** (`Towerops.Snmp.Discovery`) - One-time/manual, collects full device info
2. **Polling** (`DevicePollerWorker`) - Continuous (60s default), time-series data

**Load Distribution**: Staggered polling using hash-based offsets:
- `offset = :erlang.phash2(device_id) |> rem(interval_seconds)`
- Prevents thundering herd, stable across restarts
- Implementation: `lib/towerops/workers/polling_offset.ex`

### MikroTik API Integration

RouterOS API access alongside SNMP.

**Architecture**:
- **3-tier credential cascade**: Organization → Site → Device (like SNMP)
- **Encryption**: Passwords encrypted at rest (Cloak AES-256-GCM)
- **Detection**: Auto-detected from SNMP manufacturer field
- **Transport**: API-SSL (8729, default) or plain API (8728, insecure)
- **Security**: Plain API blocked for cloud pollers, passwords require `CLOAK_KEY` env var

**Key Files**:
- `lib/towerops/vault.ex` - Cloak vault
- `lib/towerops/ecto_types/encrypted_binary.ex` - Encrypted field type
- `lib/towerops/devices.ex` - Config resolution, propagation
- `priv/proto/agent.proto` - MikrotikDevice protobuf messages

**Credential Resolution**: Each field resolves independently up the hierarchy.

### MIB Name Resolution (Rust NIF)

Uses Rust NIF via Rustler for fast MIB resolution (replaces unreliable Erlang SNMP).

- **NIF Module**: `lib/towerops_native.ex`
- **Rust**: `native/towerops_native/src/mib.rs` (uses `snmptranslate` command)
- **MIB Files**: `priv/mibs/` (570+ vendor and standard MIBs)
- **Dependencies**: `brew install net-snmp` (macOS), `apt-get install libsnmp-dev snmp-mibs-downloader` (Docker)
- **Usage**: Try `ToweropsNative.resolve_oid/1` first, fallback to `SnmpKit.resolve/1`

Note: SnmpKit still used for SNMP protocol operations (get, walk, etc.).

## Admin Features

### User Impersonation
- Location: `/admin/users` (superuser only)
- Superusers CAN impersonate other superusers (feature, not bug)
- All events logged to audit_logs
- Stop link appears in user menu when active

### GeoIP Database
- Import MaxMind GeoLite2-City for IP-based country/city detection (GDPR cookies)
- Local: `make geoip-import DIR=~/Downloads/GeoLite2-City-CSV_20260127/`
- Production: `make geoip-import-prod DIR=...` (requires TOWEROPS_KEY)
- API: `POST /admin/api/geoip/import` (superuser only)

## Project-Specific Constraints

From AGENTS.md (see that file for details):
- Use `mix precommit` before committing
- Use `:req` (Req) for HTTP - never `:httpoison`, `:tesla`, `:httpc`
- Never use `daisyUI` - custom Tailwind components
- LiveView templates start with `<Layouts.app flash={@flash}>`
- Use `<.icon name="hero-x-mark">` for icons
- Use LiveView streams for large collections
- Forms: `to_form/2` in LiveView, `<.form for={@form}>` in templates
- Never access changesets directly in templates - use form assign
- Tailwind v4: new import syntax, never `@apply`
- Always run `mix format` after Elixir changes

### Database Schema Critical Notes

**Binary UUID Primary Keys**: All tables MUST use `:binary_id`, not `bigint`:
```elixir
create table(:table_name, primary_key: false) do
  add :id, :binary_id, primary_key: true
end
```

**SNMP Binary Data**: Convert to printable strings before saving. Non-printable → colon-separated hex.
See `lib/towerops/snmp/neighbor_discovery.ex:sanitize_string_field/1`.

### API Documentation

- Available at `/docs/api` (Tailwind UI Protocol template)
- Update controller `@doc` comments and `/docs/api` template when changing endpoints
- Wrap examples in `<%= raw(~S"""...""") %>` to prevent HEEx parsing

## Kubernetes Deployment

**Prerequisites**: cert-manager, Traefik, MetalLB, FluxCD, GitLab Agent, NFS Provisioner

**Secrets** (1Password, `towerops` namespace):
- `gitlab-registry` - Docker credentials
- `towerops-secrets` - RELEASE_COOKIE, SECRET_KEY_BASE, CLOAK_KEY
- `towerops-db` - PostgreSQL connection
- `towerops-aws` - AWS credentials

**Create CLOAK_KEY**:
```bash
# Generate and store in 1Password first
CLOAK_KEY=$(openssl rand -base64 32)

# Create secret
kubectl create secret generic towerops-secrets \
  --from-literal=RELEASE_COOKIE=$(openssl rand -base64 32) \
  --from-literal=SECRET_KEY_BASE=$(mix phx.gen.secret) \
  --from-literal=CLOAK_KEY="$CLOAK_KEY" \
  -n towerops

# Or patch existing
kubectl patch secret towerops-secrets -n towerops \
  --type='json' \
  -p="[{'op': 'add', 'path': '/data/CLOAK_KEY', 'value': '$(echo -n "$CLOAK_KEY" | base64)'}]"

# Restart pods
kubectl rollout restart deployment/towerops -n towerops
```

**Important**: Losing CLOAK_KEY makes encrypted data unrecoverable.

**Deployment**: FluxCD auto-applies `k8s/` manifests. Manual: `kubectl apply -k k8s/`

## Common Patterns

### Pagination in LiveView

Use `<.pagination>` component from `CoreComponents`:

```elixir
def handle_params(params, _url, socket) do
  page = params |> Map.get("page", "1") |> String.to_integer()
  per_page = 20

  all_items = MyContext.list_items(org_id)
  total_count = length(all_items)
  total_pages = ceil(total_count / per_page)
  page = max(1, min(page, max(1, total_pages)))

  offset = (page - 1) * per_page
  items = Enum.slice(all_items, offset, per_page)

  {:noreply, socket
   |> assign(:items, items)
   |> assign(:pagination, %{page: page, per_page: per_page,
                             total_count: total_count, total_pages: total_pages})}
end
```

Template:
```heex
<.pagination meta={@pagination} path={~p"/devices"} params={%{"tab" => @tab}} />
```

For large datasets (>10K), use database-level `limit/offset` instead of in-memory slicing.

### Browser Navigation and URL State

**Critical**: All LiveViews MUST sync UI state to URL for browser back/forward buttons.

**Core Principle**: URL is single source of truth for visible state (tabs, modals, filters, pagination).

**Pattern**: Use `push_patch/2` and `handle_params/3`:

```elixir
# Tab navigation
def handle_params(params, _url, socket) do
  tab = case params["tab"] do
    tab when tab in ~w(overview sensors) -> tab
    _ -> "overview"  # Safe default
  end
  {:noreply, assign(socket, :active_tab, tab) |> load_tab_data(tab)}
end

def handle_event("select_tab", %{"tab" => tab}, socket) do
  {:noreply, push_patch(socket, to: ~p"/devices/#{socket.assigns.device.id}?tab=#{tab}")}
end
```

Template:
```heex
<.link patch={~p"/devices/#{@device.id}?tab=overview"}
       class={if @active_tab == "overview", do: "active"}>Overview</.link>
```

**Modal state**:
```elixir
def handle_params(params, _url, socket) do
  modal = params["modal"]
  {:noreply, assign(socket, show_modal: modal != nil, modal_name: modal)}
end

def handle_event("show_modal", _, socket) do
  {:noreply, push_patch(socket, to: ~p"/settings?modal=add_token")}
end

# Close modal by removing ?modal param
{:noreply, push_patch(socket, to: ~p"/settings")}
```

**Rules**:
1. Every LiveView implements `handle_params/3` - ONLY place to read URL params
2. Validate all params, provide defaults
3. Never use `assign` for URL-backed state - use `push_patch`
4. Test browser back/forward buttons

**Achieves**: Browser navigation, bookmarkable URLs, deep linking, refresh preservation.

Reference: `docs/plans/2026-02-12-browser-navigation-fix-design.md`

## Testing Patterns

### SNMP Mocking with Mox

- `snmp_adapter().get/3` returns `{:ok, value}` or `{:error, reason}`
- `snmp_adapter().walk/3` returns `{:ok, [%{oid: "...", value: ...}]}` (list, not map)
- Values already extracted (no type wrapper needed)

**Common Pitfalls**:
- ❌ `walk` returning `{:ok, %{}}` instead of `{:ok, []}`
- ❌ Returning type wrappers like `{:integer, 123}`
- ❌ Not matching OIDs in get expectations

### Test Organization

- `DataCase` for database tests, `ConnCase` for controllers/LiveView
- `async: true` when possible
- Never open test coverage HTML files
- Never use npm - esbuild is built into Phoenix
- Run `cargo fmt` before committing Rust changes

## Dialyzer

- `mix dialyzer` - Run analysis (builds PLT first time)
- `mix dialyzer --format dialyzer` - Detailed error locations
- Start with broad types, narrow gradually
- Never suppress valid warnings - fix root cause
- PLT files in `priv/plts/` - don't commit

## Memory Leak Prevention

### JavaScript Hook Cleanup

Always remove event listeners in `destroyed()`:

```typescript
const MyHook = {
  handleClick: null as ((e: Event) => void) | null,

  mounted(this: any) {
    this.handleClick = (e: Event) => { /* logic */ }
    this.el.addEventListener("click", this.handleClick)
  },

  destroyed(this: any) {
    if (this.handleClick) {
      this.el.removeEventListener("click", this.handleClick)
      this.handleClick = null
    }
  }
}
```

Clean up third-party instances (Chart.js, Cytoscape) in `destroyed()`.

### LiveView Patterns

**Use streams for**:
- Lists with hundreds/thousands of items
- Lists with frequent PubSub updates
- When only adding/updating/removing individual items

**Example**:
```elixir
# Mount
{:ok, stream(socket, :alerts, load_all_alerts())}

# Add item
stream_insert(socket, :alerts, alert, at: 0)

# Remove item
stream_delete(socket, :alerts, alert)
```

Template:
```heex
<div id="alerts" phx-update="stream">
  <%= for {id, alert} <- @streams.alerts do %>
    <div id={id}>{alert.message}</div>
  <% end %>
</div>
```

**Use pagination for**: Audit logs, historical alerts, any list that could grow to hundreds of items.

### LiveView Form Data Access

**Problem**: `form.params` is empty until `phx-change="validate"` fires.

**Solution**: Access current values from changeset:
```elixir
def handle_event("test_connection", _params, socket) do
  changeset = socket.assigns.form.source
  form_data = Ecto.Changeset.apply_changes(changeset)  # Returns struct with all values

  # Convert to map if needed
  device_map = %{"ip_address" => form_data.ip_address, ...}
end
```

Combines: DB values + default values + user changes.

### Phoenix.PubSub

Phoenix automatically unsubscribes when LiveView terminates - no manual cleanup needed.

### Global Event Listeners

Safe patterns (already correct):
- `document.addEventListener('DOMContentLoaded', ...)` - Once per page load
- `window.addEventListener("phx:page-loading-start", ...)` - Global app events

LiveView patch navigation doesn't reload page, so no duplicates.

### Checklist

**LiveView**:
- [ ] Use `stream/3` for large/frequently-updated lists
- [ ] Use pagination for lists that could grow large
- [ ] PubSub auto-cleans (no action needed)

**JavaScript Hooks**:
- [ ] Every `addEventListener` has matching `removeEventListener` in `destroyed()`
- [ ] Store event handler references (no anonymous functions)
- [ ] Clean up third-party instances in `destroyed()`
- [ ] Set hook properties to `null` in `destroyed()`

## Changelog

**Two changelog files to maintain:**

### CHANGELOG.txt (Technical/Internal)
After every code change, append to `CHANGELOG.txt`:
- Date (YYYY-MM-DD)
- Short description (e.g. "fix: update last_snmp_poll_at for agent-polled devices")
- Files changed and brief explanation
- Technical details about implementation

Keep reverse chronological (newest at top). Never remove entries.

### priv/static/changelog.txt (User-Facing)
After significant changes, update `priv/static/changelog.txt`:
- Group changes by date (YYYY-MM-DD)
- Brief bullet points without code specifics
- Generic descriptions (no function names, modules, file paths, or language details)
- Focus on what changed for the user, not how it was implemented
- Follow existing pattern: "* Feature description" or "* Bug fix: brief description"

**Example conversions:**
- ❌ "fix: update last_snmp_poll_at in agent_channel.ex"
- ✅ "Poll time tracking improvements"

- ❌ "feat: add discover_wireless_sensors/1 to MikroTik vendor module"
- ✅ "Enhanced wireless monitoring for MikroTik devices"

**When to update user-facing changelog:**
- User-visible features or improvements
- Bug fixes affecting user experience
- New vendor/device support
- Performance improvements
- Security improvements
- Skip: Test-only changes, internal refactoring, documentation updates