916 lines
No EOL
34 KiB
Markdown
916 lines
No EOL
34 KiB
Markdown
# CLAUDE.md
|
|
|
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
|
|
## CRITICAL: Read AGENTS.md First
|
|
|
|
**Before starting any work in this repository**, you MUST read `AGENTS.md` in the project root.
|
|
|
|
- `AGENTS.md` contains comprehensive Phoenix/LiveView/Elixir guidelines that are mandatory for this project
|
|
- These guidelines take precedence over general development practices when there's a conflict
|
|
- Always read `AGENTS.md` at the start of a new conversation or when resuming work
|
|
- Follow the project-specific patterns, conventions, and constraints documented there
|
|
|
|
## Project Overview
|
|
|
|
Towerops is a Phoenix 1.8 web application built with Elixir, using Ecto for database operations (PostgreSQL), LiveView for real-time interactions, and Tailwind CSS v4 for styling.
|
|
|
|
### Data Model Relationships
|
|
|
|
```
|
|
┌─────────────┐
|
|
│ User │
|
|
│ (Accounts) │
|
|
└──────┬──────┘
|
|
│ owns/member of (many-to-many)
|
|
│
|
|
↓
|
|
┌──────────────────────────────────────────────────────┐
|
|
│ Organization │
|
|
│ (Organizations) │
|
|
└────┬────────────┬─────────────┬──────────────────────┘
|
|
│ │ │
|
|
│ has many │ has many │ has many
|
|
│ │ │
|
|
↓ ↓ ↓
|
|
┌─────────┐ ┌──────────┐ ┌────────────────┐
|
|
│ Site │ │Equipment │ │ AgentToken │
|
|
│ (Sites) │ │(Equipment│ │ (Agents) │
|
|
└────┬────┘ └────┬─────┘ └────────┬───────┘
|
|
│ │ │
|
|
│ has many │ has one │ assigned via
|
|
│ │ │
|
|
↓ ↓ ↓
|
|
┌──────────┐ ┌──────────────┐ ┌──────────────────┐
|
|
│Equipment │ │ SNMPDevice │ │ AgentAssignment │◀──┐
|
|
│(Equipment│ │(Snmp.Devices)│ │ (Agents) │ │
|
|
└────┬─────┘ └──────────────┘ └──────────────────┘ │
|
|
│ │ │
|
|
│ has many └───────────────┘
|
|
├────────────────┬─────────────────┘
|
|
│ │ assigns equipment
|
|
↓ ↓ to agent
|
|
┌──────────────┐ ┌─────────┐
|
|
│MonitoringCheck│ │ Alert │
|
|
│ (Monitoring) │ │ (Alerts)│
|
|
└──────────────┘ └─────────┘
|
|
|
|
Key Relationships:
|
|
- User can own/belong to multiple Organizations
|
|
- Organization has default_agent_token_id (optional)
|
|
- Equipment belongs to both Site and Organization (denormalized)
|
|
- Equipment can be assigned to one AgentToken via AgentAssignment
|
|
- Equipment has one SNMPDevice with Sensors and Interfaces
|
|
- Equipment has many MonitoringChecks (polling results)
|
|
- Equipment has many Alerts (equipment_down, equipment_up)
|
|
- AgentToken authenticates remote agents for local SNMP polling
|
|
```
|
|
|
|
**Note to AI assistants**: When you make changes to the data model (migrations, schema additions, new contexts), update this diagram to reflect the current state.
|
|
|
|
## Essential Commands
|
|
|
|
### Setup and Development
|
|
- `mix setup` - Install dependencies, create/migrate database, setup and build assets
|
|
- `mix phx.server` - Start the Phoenix server (accessible at http://localhost:4000)
|
|
- `iex -S mix phx.server` - Start server with interactive Elixir shell
|
|
|
|
### Testing and Quality
|
|
- `mix test` - Run all tests (automatically creates and migrates test database)
|
|
- `mix test test/path/to/specific_test.exs` - Run a specific test file
|
|
- `mix test --failed` - Re-run only previously failed tests
|
|
- `mix test --cover` - Run tests with coverage report (view: `open cover/modules_*.html`)
|
|
- `mix precommit` - **Run before committing**: compiles with warnings as errors, unlocks unused deps, formats code, and runs tests
|
|
- `mix dialyzer` - Run static type analysis
|
|
|
|
**Coverage Notes**:
|
|
- Target: 90% minimum
|
|
- Protobuf-generated modules (Towerops.Agent.*) show 0% but don't need tests
|
|
- TimescaleDB aggregate tests should be tagged `@tag :skip` (test DB lacks continuous aggregates)
|
|
|
|
### Database
|
|
- `mix ecto.create` - Create the database
|
|
- `mix ecto.migrate` - Run pending migrations
|
|
- `mix ecto.reset` - Drop, create, and migrate database
|
|
- `mix ecto.gen.migration migration_name_using_underscores` - Generate a new migration file
|
|
|
|
### Assets
|
|
- `mix assets.build` - Build CSS and JS assets (Tailwind + esbuild)
|
|
- `mix assets.deploy` - Build minified assets for production
|
|
|
|
## Architecture
|
|
|
|
### Application Structure
|
|
|
|
Standard Phoenix conventions with separation between business logic (`lib/towerops/`) and web interface (`lib/towerops_web/`).
|
|
|
|
**Key Configuration**:
|
|
- **Binary IDs**: Generators use binary (UUID) primary keys by default (`binary_id: true`)
|
|
- **Timestamps**: Use `:utc_datetime` for all timestamps
|
|
- **Web server**: Bandit adapter (not Cowboy)
|
|
- **HTTP client**: `:req` library (Req module) - this is the only approved HTTP client
|
|
- **Ecto repos**: `[Towerops.Repo]`
|
|
|
|
### Web Layer Patterns
|
|
|
|
All LiveViews, LiveComponents, and HTML modules automatically get these imports/aliases via `html_helpers/0`:
|
|
- `ToweropsWeb.CoreComponents` - Core UI components (`<.button>`, `<.input>`, `<.form>`, etc.)
|
|
- `ToweropsWeb.Layouts` - Layout components
|
|
- `Phoenix.LiveView.JS` - Client-side JS commands
|
|
- Gettext for translations
|
|
- Verified routes with `~p` sigil
|
|
|
|
### Asset Pipeline
|
|
|
|
- **Tailwind CSS v4**: Uses new `@import "tailwindcss"` syntax in `assets/css/app.css`, no `tailwind.config.js` needed
|
|
- **esbuild**: Bundles JavaScript from `assets/js/app.js`
|
|
- All vendor scripts/styles must be imported into app.js/app.css - cannot reference external src/href in layouts
|
|
- No inline `<script>` tags in templates - use colocated LiveView hooks instead
|
|
- Assets are rebuilt on save automatically
|
|
|
|
### Development Environment
|
|
|
|
- Dev routes: LiveDashboard at `/dev/dashboard`, Swoosh mailbox at `/dev/mailbox`
|
|
- Phoenix LiveReload watches for file changes
|
|
- Code reloader enabled
|
|
|
|
### Custom Ecto Types
|
|
|
|
Uses custom Ecto types to create rich domain objects and avoid "primitive obsession" - the anti-pattern of representing domain concepts as primitive types (strings, integers) instead of structured data.
|
|
|
|
**Pattern**: Implement `Ecto.Type` behavior with 4 required callbacks:
|
|
- `type/0` - Database storage type (e.g., `:string`, `:jsonb`)
|
|
- `cast/1` - Transform user input to domain struct (validation happens here)
|
|
- `load/1` - Convert database value to domain struct
|
|
- `dump/1` - Convert domain struct back to database primitive
|
|
|
|
**Benefits**:
|
|
- **Type Safety**: Work with `%IPAddress{}` instead of raw strings
|
|
- **Centralized Validation**: One place for format checking and normalization
|
|
- **Automatic Conversion**: Happens transparently in changesets
|
|
- **Rich Domain Behavior**: Pattern matching, helper functions, computed properties
|
|
- **Zero Migration Cost**: Column types stay as primitives (varchar, jsonb, etc.)
|
|
|
|
**Available Custom Types**:
|
|
|
|
1. **`Towerops.EctoTypes.IpAddress`** - IPv4/IPv6 address handling
|
|
- Validates format using `:inet.parse_address/1`
|
|
- Struct: `%IPAddress{address: String.t, version: :ipv4 | :ipv6, tuple: ip_tuple}`
|
|
- Helpers: `ipv4?/1`, `ipv6?/1`, `to_tuple/1`, `to_string/1`
|
|
- Handles edge cases: nil, IPv6 zone IDs, rejects CIDR notation
|
|
|
|
2. **`Towerops.EctoTypes.EmailAddress`** - Normalized email addresses _(Planned - Phase 2)_
|
|
- Normalizes to lowercase
|
|
- Splits into local + domain parts
|
|
- Consistent validation across all user-facing schemas
|
|
|
|
3. **`Towerops.EctoTypes.MacAddress`** - MAC address handling _(Planned - Phase 3)_
|
|
- Handles multiple input formats (colon, hyphen, dot-separated)
|
|
- Normalizes to colon-separated lowercase
|
|
- SNMP binary conversion support
|
|
|
|
**Usage Example**:
|
|
|
|
```elixir
|
|
# Schema definition
|
|
defmodule Towerops.Devices.Device do
|
|
use Ecto.Schema
|
|
|
|
schema "devices" do
|
|
field :name, :string
|
|
field :ip_address, Towerops.EctoTypes.IpAddress # Custom type
|
|
end
|
|
end
|
|
|
|
# Changeset - validation handled automatically by custom type
|
|
def changeset(device, attrs) do
|
|
device
|
|
|> cast(attrs, [:name, :ip_address])
|
|
|> validate_required([:name, :ip_address])
|
|
# No need for manual IP validation - handled in IPAddress.cast/1
|
|
end
|
|
|
|
# Pattern matching in functions
|
|
def ipv4_only?(%Device{ip_address: %IPAddress{version: :ipv4}}), do: true
|
|
def ipv4_only?(_), do: false
|
|
```
|
|
|
|
**When to Create a Custom Type**:
|
|
|
|
✅ **Good candidates**:
|
|
- Domain concepts with specific validation rules (emails, phone numbers, URLs)
|
|
- Values that benefit from normalization (case-insensitive emails, MAC addresses)
|
|
- Data with multiple representations (IP addresses as strings vs tuples)
|
|
- Types that need computed properties (IP version derived from format)
|
|
|
|
❌ **Avoid for**:
|
|
- Simple strings with no validation (names, descriptions)
|
|
- Values that are truly primitive (booleans, simple integers)
|
|
- One-off validations (use changeset validation instead)
|
|
|
|
**Implementation Pattern**:
|
|
|
|
```elixir
|
|
defmodule Towerops.EctoTypes.YourType do
|
|
use Ecto.Type
|
|
|
|
# Define your domain struct
|
|
defstruct [:field1, :field2]
|
|
|
|
@impl Ecto.Type
|
|
def type, do: :string # Database storage type
|
|
|
|
@impl Ecto.Type
|
|
def cast(nil), do: {:ok, nil}
|
|
def cast(%__MODULE__{} = value), do: {:ok, value} # Idempotence
|
|
def cast(value) when is_binary(value) do
|
|
# Validation and transformation logic here
|
|
case validate_and_parse(value) do
|
|
{:ok, parsed} -> {:ok, struct(__MODULE__, parsed)}
|
|
{:error, _} -> :error
|
|
end
|
|
end
|
|
def cast(_), do: :error
|
|
|
|
@impl Ecto.Type
|
|
def load(value) when is_binary(value), do: cast(value)
|
|
def load(nil), do: {:ok, nil}
|
|
|
|
@impl Ecto.Type
|
|
def dump(%__MODULE__{} = struct), do: {:ok, struct.field1} # Extract primitive
|
|
def dump(nil), do: {:ok, nil}
|
|
def dump(_), do: :error
|
|
|
|
# Helper functions
|
|
def new(value), do: cast(value)
|
|
def new!(value) do
|
|
case cast(value) do
|
|
{:ok, result} -> result
|
|
:error -> raise ArgumentError, "invalid value"
|
|
end
|
|
end
|
|
end
|
|
```
|
|
|
|
**Migration Notes**:
|
|
|
|
Adopting custom types requires NO database migrations:
|
|
|
|
1. Create custom type module in `lib/towerops/ecto_types/`
|
|
2. Create comprehensive tests in `test/towerops/ecto_types/`
|
|
3. Change schema field type: `field :ip_address, :string` → `field :ip_address, Towerops.EctoTypes.IpAddress`
|
|
4. Remove manual validation from changeset (now handled in `cast/1`)
|
|
5. Update tests to assert on struct fields instead of raw strings
|
|
6. Database column stays as varchar/jsonb - Ecto handles conversion transparently
|
|
|
|
**Testing Pattern**:
|
|
|
|
```elixir
|
|
defmodule Towerops.EctoTypes.YourTypeTest do
|
|
use ExUnit.Case, async: true
|
|
|
|
alias Towerops.EctoTypes.YourType
|
|
|
|
describe "type/0" do
|
|
test "returns storage type" do
|
|
assert YourType.type() == :string
|
|
end
|
|
end
|
|
|
|
describe "cast/1" do
|
|
test "accepts valid values" do
|
|
assert {:ok, %YourType{}} = YourType.cast("valid-value")
|
|
end
|
|
|
|
test "rejects invalid values" do
|
|
assert :error = YourType.cast("invalid")
|
|
end
|
|
|
|
test "handles nil" do
|
|
assert {:ok, nil} = YourType.cast(nil)
|
|
end
|
|
end
|
|
|
|
describe "roundtrip" do
|
|
test "cast -> dump -> load preserves value" do
|
|
{:ok, original} = YourType.cast("value")
|
|
{:ok, dumped} = YourType.dump(original)
|
|
{:ok, loaded} = YourType.load(dumped)
|
|
assert loaded == original
|
|
end
|
|
end
|
|
end
|
|
```
|
|
|
|
### Background Job Architecture (Oban)
|
|
|
|
Uses PostgreSQL-backed Oban for all background processing with cluster-wide coordination.
|
|
|
|
**Queues:**
|
|
- `default` (10 workers) - General background tasks
|
|
- `discovery` (10 workers) - SNMP discovery operations
|
|
- `pollers` (50 workers) - SNMP polling jobs (one per device)
|
|
- `monitors` (50 workers) - Health check jobs (one per device)
|
|
- `maintenance` (5 workers) - Periodic cleanup and health check workers
|
|
|
|
**Key Workers:**
|
|
1. **Self-Scheduling Workers** (per-device recurring jobs):
|
|
- `DeviceMonitorWorker` - Health check polling (60s default)
|
|
- `DevicePollerWorker` - SNMP data collection (60s default)
|
|
- Schedule next run before completing, one job per enabled device
|
|
- Automatically created/cancelled when device settings change
|
|
|
|
2. **Oban Cron Workers** (cluster-wide periodic maintenance):
|
|
- `NeighborCleanupWorker` - Hourly cleanup of stale neighbors/ARP/MAC entries
|
|
- `StaleAgentWorker` - Detects agents that haven't checked in for 10+ minutes (every minute)
|
|
- `AgentLatencyEvaluator` - Latency-based agent reassignment (every 5 minutes)
|
|
- `JobHealthCheckWorker` - Safety net to recover missing monitor/poller jobs (every 10 minutes)
|
|
|
|
**Resilience:**
|
|
- Oban Cron jobs run cluster-wide via PostgreSQL-based locking
|
|
- Self-scheduling workers recovered by `JobHealthCheckWorker` every 10 minutes
|
|
- All jobs visible in Oban dashboard at `/dev/dashboard` → Oban tab
|
|
- Failed jobs automatically retried with exponential backoff
|
|
|
|
**Configuration:**
|
|
- Dev: `config/dev.exs` - Oban config with cron plugin
|
|
- Prod: `config/runtime.exs` - Same cron schedule as dev
|
|
- Application: `lib/towerops/application.ex` - Oban supervision
|
|
|
|
**Oban Web Dashboard**:
|
|
- Location: `/admin/oban` (superuser access required)
|
|
- Vendored dependencies in `vendor/`: `oban_pro`, `oban_web`, `oban_met`
|
|
- See `vendor/README.md` for update instructions
|
|
|
|
### LiveDashboard and Telemetry
|
|
|
|
Available at `/admin/dashboard` (or `/dev/dashboard` in development).
|
|
|
|
**Metrics**: Phoenix requests, DB queries, VM stats, Oban queue sizes, Redis/Valkey stats
|
|
- Collected every 10 seconds via `telemetry_poller`
|
|
- Access: Navigate to dashboard → "Metrics" tab
|
|
|
|
### SNMP Polling Architecture
|
|
|
|
Two separate SNMP collection mechanisms:
|
|
|
|
**1. Discovery (`Towerops.Snmp.Discovery`)** - One-time or manual
|
|
- Triggered manually or via `DiscoveryWorker` Oban job
|
|
- Collects: system info, device identification, interfaces, sensors, neighbors
|
|
- Creates/updates device record and associated data
|
|
|
|
**2. Polling (`DevicePollerWorker`)** - Continuous time-series
|
|
- Runs automatically every 60 seconds (configurable per equipment)
|
|
- Collects: sensor readings, interface stats, neighbor topology
|
|
- Updates time-series data, keeps neighbor data fresh
|
|
- Direct Oban worker (no GenServer layer)
|
|
|
|
**Load Distribution - Staggered Polling**:
|
|
- Uses deterministic hash-based offsets via `PollingOffset.calculate_offset/2`
|
|
- Formula: `offset = :erlang.phash2(device_id) |> rem(interval_seconds)`
|
|
- Distributes devices evenly across full interval (prevents thundering herd)
|
|
- Same device always gets same offset (stable across restarts)
|
|
- Applied to both `DevicePollerWorker` and `DeviceMonitorWorker`
|
|
- Implementation: `lib/towerops/workers/polling_offset.ex`
|
|
|
|
### MIB Name Resolution (Rust NIF)
|
|
|
|
Uses Rust NIF via Rustler for fast SNMP MIB name resolution (replaces unreliable Erlang SNMP modules).
|
|
|
|
**Architecture:**
|
|
- NIF Module: `lib/towerops_native.ex`
|
|
- Rust Implementation: `native/towerops_native/src/mib.rs` (uses `snmptranslate` command)
|
|
- Build Integration: `mix.exs` - Rustler compiler configured via `rustler_crates`
|
|
|
|
**How It Works:**
|
|
1. Application startup calls `ToweropsNative.load_mib_directory/1` with `priv/mibs` path
|
|
2. Client calls `resolve_mib_name/1` before SNMP operations
|
|
3. Tries `ToweropsNative.resolve_oid/1` first, falls back to `SnmpKit.resolve/1`
|
|
4. Uses `snmptranslate -On` command-line tool with auto-detected system MIB directories
|
|
|
|
**System Dependencies:**
|
|
- **Development** (macOS): `brew install net-snmp` (keg-only, requires PKG_CONFIG_PATH)
|
|
- **Production** (Docker): `apt-get install libsnmp-dev snmp-mibs-downloader`
|
|
- **Runtime**: `snmptranslate` command must be in PATH
|
|
|
|
**MIB Files:**
|
|
- Located in `priv/mibs/` directory (570+ vendor and standard MIB files)
|
|
- Standard MIBs: SNMPv2-MIB, IF-MIB, etc.
|
|
- Vendor MIBs: Cisco, Ubiquiti, IEEE, etc.
|
|
|
|
**Important**:
|
|
- SnmpKit still used for SNMP protocol operations (get, walk, get_next, get_bulk)
|
|
- NIF only handles MIB name resolution
|
|
- Rust compilation happens automatically via Mix
|
|
- Clean Rust builds: `cd native/towerops_native && cargo clean`
|
|
|
|
## Admin Features
|
|
|
|
### User Impersonation
|
|
|
|
**Location**: `/admin/users` (superuser only)
|
|
|
|
**Files**: `lib/towerops_web/live/admin/user_live/index.ex`, `lib/towerops_web/user_auth.ex`
|
|
|
|
**Notes**:
|
|
- Superusers CAN impersonate other superusers (security feature, not bug)
|
|
- Backend checks prevent self-impersonation
|
|
- All events logged to audit_logs table
|
|
- Stop impersonation link appears in user menu when active
|
|
|
|
### GeoIP Database Management
|
|
|
|
**Purpose**: Import MaxMind GeoLite2-City database for IP-based country/city detection (GDPR cookie consent).
|
|
|
|
**Files**: `lib/towerops_web/controllers/api/v1/geoip_controller.ex`, `lib/mix/tasks/geoip.import.ex`
|
|
|
|
**Import Methods**:
|
|
```bash
|
|
# Local Import (Development)
|
|
make geoip-import DIR=~/Downloads/GeoLite2-City-CSV_20260127/
|
|
|
|
# Production Import via API (requires TOWEROPS_KEY env var)
|
|
make geoip-import-prod DIR=~/Downloads/GeoLite2-City-CSV_20260127/
|
|
```
|
|
|
|
**Implementation**:
|
|
- Local mode: Direct database import via Ecto
|
|
- Production mode: Processes CSVs locally, sends data to API in 5,000 record batches
|
|
- API endpoint: `POST /admin/api/geoip/import` (superuser only)
|
|
- Download from: https://dev.maxmind.com/geoip/geolite2-free-geolocation-data
|
|
|
|
## Project-Specific Constraints
|
|
|
|
Key constraints from AGENTS.md (see that file for complete details):
|
|
|
|
- Use `mix precommit` before committing changes
|
|
- Use `:req` (Req) for all HTTP requests - never use `:httpoison`, `:tesla`, or `:httpc`
|
|
- Never use `daisyUI` - write custom Tailwind components
|
|
- LiveView templates must start with `<Layouts.app flash={@flash}>`
|
|
- Use `<.icon name="hero-x-mark">` component for icons, never Heroicons modules
|
|
- Use LiveView streams for collections to avoid memory issues
|
|
- Forms must use `to_form/2` in LiveView and `<.form for={@form}>` in templates
|
|
- Never access changesets directly in templates - always use the form assign
|
|
- Tailwind v4 uses new import syntax - never use `@apply` in CSS
|
|
- When adding/updating hex modules, only use https://hex.pm
|
|
- Always run `mix format` after modifying Elixir files (don't specify file, format everything)
|
|
|
|
### Database Schema Critical Notes
|
|
|
|
**Binary UUID Primary Keys**: All tables MUST use `:binary_id` for primary keys, not `bigint`.
|
|
|
|
```elixir
|
|
create table(:table_name, primary_key: false) do
|
|
add :id, :binary_id, primary_key: true
|
|
# ...
|
|
end
|
|
```
|
|
|
|
**Issue History**: The `snmp_neighbors` table was initially created with `bigint` instead of `:binary_id`, causing `DBConnection.EncodeError`. Always verify actual schema matches migration:
|
|
```bash
|
|
psql towerops_dev -c "\d table_name" # Check actual schema
|
|
```
|
|
|
|
**SNMP Binary Data Handling**: All SNMP values must be converted to printable strings before saving. Non-printable binaries should be converted to colon-separated hex format. See `lib/towerops/snmp/neighbor_discovery.ex:sanitize_string_field/1`.
|
|
|
|
### API Documentation
|
|
|
|
Available at `/docs/api` (adapted from Tailwind UI Protocol template).
|
|
|
|
**Files**: `lib/towerops_web/controllers/api_docs_controller.ex`, `lib/towerops_web/controllers/api_docs_html/index.html.heex`
|
|
|
|
**When updating API endpoints**:
|
|
1. Update controller documentation (`@doc` comments)
|
|
2. Update `/docs/api` template
|
|
3. Wrap code examples in `<%= raw(~S"""...""") %>` to prevent HEEx parsing curly braces
|
|
|
|
**Current Documented Endpoints**: Sites API, Devices API (GET/POST/PATCH/DELETE)
|
|
|
|
## Kubernetes Deployment
|
|
|
|
**Prerequisites**: cert-manager, Traefik, MetalLB, FluxCD, GitLab Agent, NFS Provisioner
|
|
**Secrets**: All stored in 1Password, created manually in `towerops` namespace
|
|
- `gitlab-registry` - Docker registry credentials
|
|
- `towerops-secrets` - RELEASE_COOKIE, SECRET_KEY_BASE
|
|
- `towerops-db` - PostgreSQL connection details
|
|
- `towerops-aws` - AWS credentials
|
|
|
|
**Deployment**: FluxCD monitors Git repository, automatically applies manifests in `k8s/` directory
|
|
**Manual**: `kubectl apply -k k8s/`
|
|
|
|
**Troubleshooting**:
|
|
- FluxCD: `kubectl get kustomization -n flux-system`
|
|
- Certificates: `kubectl describe certificate towerops-net-cert -n towerops`
|
|
- Pods: `kubectl describe pod -n towerops -l app=towerops`
|
|
|
|
## Common Patterns
|
|
|
|
### Pagination in LiveView
|
|
|
|
Simple offset-based pagination pattern for lists. Example from `DeviceLive.Index` (discovered devices tab):
|
|
|
|
**1. Add pagination to socket assigns in `handle_params` or `apply_action`:**
|
|
|
|
```elixir
|
|
def handle_params(params, _url, socket) do
|
|
tab = Map.get(params, "tab", "existing")
|
|
{:noreply, apply_action(socket, socket.assigns.live_action, params, tab)}
|
|
end
|
|
|
|
defp apply_action(socket, :index, params, "discovered") do
|
|
organization = socket.assigns.current_scope.organization
|
|
|
|
# Get page from params, default to 1
|
|
page = params |> Map.get("page", "1") |> String.to_integer()
|
|
per_page = 20
|
|
|
|
# Fetch all items (or use a paginated query for large datasets)
|
|
all_items = MyContext.list_items(organization.id)
|
|
total_count = length(all_items)
|
|
total_pages = ceil(total_count / per_page)
|
|
|
|
# Ensure page is within valid range
|
|
page = max(1, min(page, max(1, total_pages)))
|
|
|
|
# Slice items for current page
|
|
offset = (page - 1) * per_page
|
|
items = Enum.slice(all_items, offset, per_page)
|
|
|
|
socket
|
|
|> assign(:items, items)
|
|
|> assign(:pagination, %{
|
|
page: page,
|
|
per_page: per_page,
|
|
total_count: total_count,
|
|
total_pages: total_pages
|
|
})
|
|
end
|
|
```
|
|
|
|
**2. Add pagination helper function to LiveView module:**
|
|
|
|
```elixir
|
|
# Generate pagination range with ellipsis for large page counts
|
|
# Shows: [1] ... [4] [5] [6] ... [20] when on page 5 of 20
|
|
defp pagination_range(current_page, total_pages) do
|
|
cond do
|
|
total_pages <= 7 ->
|
|
# Show all pages if 7 or fewer
|
|
Enum.to_list(1..total_pages)
|
|
|
|
current_page <= 4 ->
|
|
# Near the start
|
|
[1, 2, 3, 4, 5, :ellipsis, total_pages]
|
|
|
|
current_page >= total_pages - 3 ->
|
|
# Near the end
|
|
[1, :ellipsis, total_pages - 4, total_pages - 3, total_pages - 2, total_pages - 1, total_pages]
|
|
|
|
true ->
|
|
# In the middle
|
|
[1, :ellipsis, current_page - 1, current_page, current_page + 1, :ellipsis, total_pages]
|
|
end
|
|
end
|
|
```
|
|
|
|
**3. Add pagination controls to template:**
|
|
|
|
```heex
|
|
<%= if @pagination.total_pages > 1 do %>
|
|
<div class="mt-6 flex items-center justify-between border-t border-gray-200 dark:border-white/10 pt-6">
|
|
<!-- Mobile: Previous/Next only -->
|
|
<div class="flex flex-1 justify-between sm:hidden">
|
|
<.link
|
|
:if={@pagination.page > 1}
|
|
patch={~p"/path?page=#{@pagination.page - 1}"}
|
|
class="relative inline-flex items-center rounded-md border border-gray-300 bg-white px-4 py-2 text-sm font-medium text-gray-700 hover:bg-gray-50 dark:border-white/10 dark:bg-gray-800 dark:text-gray-300 dark:hover:bg-gray-700"
|
|
>
|
|
Previous
|
|
</.link>
|
|
<.link
|
|
:if={@pagination.page < @pagination.total_pages}
|
|
patch={~p"/path?page=#{@pagination.page + 1}"}
|
|
class="relative ml-3 inline-flex items-center rounded-md border border-gray-300 bg-white px-4 py-2 text-sm font-medium text-gray-700 hover:bg-gray-50 dark:border-white/10 dark:bg-gray-800 dark:text-gray-300 dark:hover:bg-gray-700"
|
|
>
|
|
Next
|
|
</.link>
|
|
</div>
|
|
|
|
<!-- Desktop: Full pagination with page numbers -->
|
|
<div class="hidden sm:flex sm:flex-1 sm:items-center sm:justify-between">
|
|
<div>
|
|
<p class="text-sm text-gray-700 dark:text-gray-400">
|
|
Showing
|
|
<span class="font-medium">{(@pagination.page - 1) * @pagination.per_page + 1}</span>
|
|
to
|
|
<span class="font-medium">{min(@pagination.page * @pagination.per_page, @pagination.total_count)}</span>
|
|
of
|
|
<span class="font-medium">{@pagination.total_count}</span>
|
|
results
|
|
</p>
|
|
</div>
|
|
<div>
|
|
<nav class="isolate inline-flex -space-x-px rounded-md shadow-sm" aria-label="Pagination">
|
|
<!-- Previous button -->
|
|
<.link
|
|
:if={@pagination.page > 1}
|
|
patch={~p"/path?page=#{@pagination.page - 1}"}
|
|
class="relative inline-flex items-center rounded-l-md px-2 py-2 text-gray-400 ring-1 ring-inset ring-gray-300 hover:bg-gray-50 dark:ring-white/10 dark:hover:bg-gray-800"
|
|
>
|
|
<span class="sr-only">Previous</span>
|
|
<.icon name="hero-chevron-left" class="h-5 w-5" />
|
|
</.link>
|
|
|
|
<!-- Page numbers with ellipsis -->
|
|
<%= for page_num <- pagination_range(@pagination.page, @pagination.total_pages) do %>
|
|
<%= if page_num == :ellipsis do %>
|
|
<span class="relative inline-flex items-center px-4 py-2 text-sm font-semibold text-gray-700 ring-1 ring-inset ring-gray-300 dark:text-gray-400 dark:ring-white/10">
|
|
...
|
|
</span>
|
|
<% else %>
|
|
<.link
|
|
patch={~p"/path?page=#{page_num}"}
|
|
class={[
|
|
"relative inline-flex items-center px-4 py-2 text-sm font-semibold ring-1 ring-inset ring-gray-300 dark:ring-white/10",
|
|
if @pagination.page == page_num do
|
|
"z-10 bg-blue-600 text-white focus-visible:outline focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-600"
|
|
else
|
|
"text-gray-900 hover:bg-gray-50 dark:text-gray-300 dark:hover:bg-gray-800"
|
|
end
|
|
]}
|
|
>
|
|
{page_num}
|
|
</.link>
|
|
<% end %>
|
|
<% end %>
|
|
|
|
<!-- Next button -->
|
|
<.link
|
|
:if={@pagination.page < @pagination.total_pages}
|
|
patch={~p"/path?page=#{@pagination.page + 1}"}
|
|
class="relative inline-flex items-center rounded-r-md px-2 py-2 text-gray-400 ring-1 ring-inset ring-gray-300 hover:bg-gray-50 dark:ring-white/10 dark:hover:bg-gray-800"
|
|
>
|
|
<span class="sr-only">Next</span>
|
|
<.icon name="hero-chevron-right" class="h-5 w-5" />
|
|
</.link>
|
|
</nav>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<% end %>
|
|
```
|
|
|
|
**Notes**:
|
|
- Use `patch` instead of `navigate` to keep LiveView mounted and avoid full page reload
|
|
- Preserve other query params when changing pages (e.g., `~p"/path?tab=#{@tab}&page=#{page}"`)
|
|
- For very large datasets (>10,000 items), use database-level pagination with `limit/offset` instead of in-memory slicing
|
|
- The `pagination_range/2` function creates smart ellipsis to avoid showing too many page numbers
|
|
- Mobile shows simple Previous/Next, desktop shows full pagination controls
|
|
|
|
## Testing Patterns
|
|
|
|
### SNMP Mocking with Mox
|
|
|
|
**SNMP Adapter Mock Format**:
|
|
- `snmp_adapter().get/3` returns `{:ok, value}` or `{:error, reason}`
|
|
- `snmp_adapter().walk/3` returns `{:ok, [%{oid: "...", value: ...}]}` (list of maps, NOT a map)
|
|
- Values from `get/3` are already extracted (no type wrapper needed)
|
|
|
|
**Common Pitfalls**:
|
|
- ❌ Returning `{:ok, %{}}` from walk - should be `{:ok, []}`
|
|
- ❌ Returning type wrappers like `{:integer, 123}` - values are already extracted
|
|
- ❌ Not matching OIDs in get expectations - use pattern matching on OID string
|
|
|
|
### Test Organization
|
|
|
|
- Use `DataCase` for database tests, `ConnCase` for controller/LiveView tests
|
|
- Use `async: true` for tests that can run in parallel
|
|
- Use `async: false` for tests with shared state
|
|
- Never open test coverage HTML files
|
|
- Never try to use npm to install JS things, use esbuild built in to Phoenix
|
|
- When working in Rust, always run `cargo fmt` before committing
|
|
|
|
## Dialyzer: Static Type Analysis
|
|
|
|
**Commands:**
|
|
- `mix dialyzer` - Run analysis (builds PLT on first run)
|
|
- `mix dialyzer --format dialyzer` - Show detailed error locations
|
|
- `mix dialyzer --format ignore_file` - Generate `.dialyzer_ignore.exs` for gradual adoption
|
|
|
|
**Configuration** (`mix.exs`):
|
|
```elixir
|
|
defp dialyzer do
|
|
[
|
|
plt_add_deps: :apps_direct, # Faster incremental analysis
|
|
plt_file: {:no_warn, "priv/plts/dialyzer.plt"},
|
|
flags: [:error_handling, :underspecs]
|
|
]
|
|
end
|
|
```
|
|
|
|
**Common Warning Types**:
|
|
1. Invalid Contracts - Return type doesn't match all code paths
|
|
2. Function Application Arguments - Type alignment issues
|
|
3. Opaque Type Mismatches - External module accessing opaque type internals
|
|
4. Range Errors - Incomplete specs (e.g., not accounting for empty list case)
|
|
|
|
**Workflow**:
|
|
- Start with broad types, narrow gradually
|
|
- Use ignore file for gradual adoption in legacy code
|
|
- Combine with runtime type checking for complete safety
|
|
- PLT files in `priv/plts/` - don't commit
|
|
- Never suppress valid warnings - fix the root cause
|
|
|
|
## Memory Leak Prevention
|
|
|
|
### JavaScript Hook Event Listeners
|
|
|
|
**Problem**: Event listeners added in LiveView hooks without cleanup cause memory leaks when the hook is destroyed.
|
|
|
|
**Solution**: Always store event listener references and remove them in `destroyed()`:
|
|
|
|
```typescript
|
|
const MyHook = {
|
|
handleClick: null as ((e: Event) => void) | null,
|
|
handleDrag: null as ((e: DragEvent) => void) | null,
|
|
|
|
mounted(this: any) {
|
|
// Store reference to handler
|
|
this.handleClick = (e: Event) => {
|
|
// Handler logic
|
|
}
|
|
|
|
this.el.addEventListener("click", this.handleClick)
|
|
},
|
|
|
|
destroyed(this: any) {
|
|
// Clean up event listeners
|
|
if (this.handleClick) {
|
|
this.el.removeEventListener("click", this.handleClick)
|
|
this.handleClick = null
|
|
}
|
|
|
|
// Clean up any other state
|
|
this.someProperty = null
|
|
}
|
|
}
|
|
```
|
|
|
|
**Examples Fixed**:
|
|
- `CopyToClipboard` - Added `destroyed()` hook to remove click listener
|
|
- `DeviceListReorder` - Added `destroyed()` to remove dragstart, dragend, dragover, drop listeners
|
|
|
|
**Chart.js Cleanup** (already correct):
|
|
```typescript
|
|
const SensorChart = {
|
|
destroyed() {
|
|
if (this.chart) {
|
|
this.chart.destroy() // Prevents canvas memory leaks
|
|
this.chart = null
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
**Cytoscape Cleanup** (already correct):
|
|
```typescript
|
|
const NetworkMap = {
|
|
destroyed(this: any) {
|
|
if (this.cy) {
|
|
this.cy.destroy() // Cleans up Cytoscape instance
|
|
this.cy = null
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
### LiveView Assigns vs Streams
|
|
|
|
**Problem**: Large collections in assigns are kept in memory for the entire LiveView session.
|
|
|
|
**When to use streams instead of assigns**:
|
|
- Lists with hundreds or thousands of items
|
|
- Lists that receive frequent updates via PubSub
|
|
- Lists where you only need to add/update/remove individual items
|
|
|
|
**Example - AlertLive.Index should use streams**:
|
|
|
|
❌ **Current (memory inefficient)**:
|
|
```elixir
|
|
def mount(_params, _session, socket) do
|
|
{:ok, socket |> assign(:alerts, load_all_alerts())}
|
|
end
|
|
|
|
def handle_info({:new_alert, _device_id, _alert}, socket) do
|
|
# Reloads entire list into memory
|
|
{:noreply, assign(socket, :alerts, load_all_alerts())}
|
|
end
|
|
```
|
|
|
|
✅ **Better (use streams)**:
|
|
```elixir
|
|
def mount(_params, _session, socket) do
|
|
{:ok, socket |> stream(:alerts, load_all_alerts())}
|
|
end
|
|
|
|
def handle_info({:new_alert, _device_id, alert}, socket) do
|
|
# Only adds one alert to the stream
|
|
{:noreply, stream_insert(socket, :alerts, alert, at: 0)}
|
|
end
|
|
|
|
def handle_info({:alert_resolved, _device_id, alert}, socket) do
|
|
# Only removes one alert from the stream
|
|
{:noreply, stream_delete(socket, :alerts, alert)}
|
|
end
|
|
```
|
|
|
|
Template change:
|
|
```heex
|
|
<!-- Old -->
|
|
<%= for alert <- @alerts do %>
|
|
<div id={"alert-#{alert.id}"}>{alert.message}</div>
|
|
<% end %>
|
|
|
|
<!-- New with streams -->
|
|
<div id="alerts" phx-update="stream">
|
|
<%= for {id, alert} <- @streams.alerts do %>
|
|
<div id={id}>{alert.message}</div>
|
|
<% end %>
|
|
</div>
|
|
```
|
|
|
|
### Pagination for Large Lists
|
|
|
|
**Problem**: Loading thousands of items into memory for display.
|
|
|
|
**Solution**: Use pagination (see "Pagination in LiveView" section above).
|
|
|
|
**When pagination is appropriate**:
|
|
- Discovered devices (already paginated - 20 per page)
|
|
- Audit logs
|
|
- Historical alerts
|
|
- Any list that could grow to hundreds of items
|
|
|
|
**Example** (from DeviceLive.Index):
|
|
```elixir
|
|
# Don't keep all items in assigns
|
|
all_items = Context.list_all_items() # Temporary variable
|
|
total_count = length(all_items)
|
|
|
|
# Only store current page in assigns
|
|
items_page = Enum.slice(all_items, offset, per_page)
|
|
|
|
socket
|
|
|> assign(:items, items_page) # Only current page
|
|
|> assign(:pagination, %{...}) # Metadata
|
|
```
|
|
|
|
### Phoenix.PubSub Subscriptions
|
|
|
|
**Good news**: Phoenix automatically unsubscribes when a LiveView process terminates - no manual cleanup needed.
|
|
|
|
**Pattern** (already correct in codebase):
|
|
```elixir
|
|
def mount(_params, _session, socket) do
|
|
if connected?(socket) do
|
|
Phoenix.PubSub.subscribe(Towerops.PubSub, "topic")
|
|
end
|
|
# No need for terminate/2 callback
|
|
end
|
|
```
|
|
|
|
### Global Event Listeners (app.ts)
|
|
|
|
**Safe patterns** (already correct):
|
|
- `document.addEventListener('DOMContentLoaded', ...)` - Only fires once, remains for page lifetime
|
|
- `window.addEventListener("phx:page-loading-start", ...)` - Global events for the application
|
|
- `window.addEventListener("phx:copy", ...)` - Phoenix JS commands
|
|
|
|
These are acceptable because:
|
|
1. They're registered once when the page loads
|
|
2. They remain active for the entire page lifetime
|
|
3. LiveView patch navigation doesn't reload the page, so no duplicates
|
|
|
|
### Memory Leak Checklist
|
|
|
|
When writing new LiveView code or JavaScript hooks:
|
|
|
|
**LiveView (Elixir)**:
|
|
- [ ] Use `stream/3` for large or frequently-updated lists
|
|
- [ ] Use pagination for lists that could grow large
|
|
- [ ] Avoid storing entire datasets in assigns when only a subset is displayed
|
|
- [ ] PubSub subscriptions clean up automatically (no action needed)
|
|
|
|
**JavaScript Hooks**:
|
|
- [ ] Every `addEventListener` in `mounted()` has a matching `removeEventListener` in `destroyed()`
|
|
- [ ] Store event handler references (don't use anonymous functions)
|
|
- [ ] Clean up third-party library instances (Chart.js, Cytoscape, etc.) in `destroyed()`
|
|
- [ ] Set hook properties to `null` in `destroyed()` to release references
|
|
|
|
**Testing for Memory Leaks**:
|
|
- Navigate to a page with a large list
|
|
- Use browser DevTools → Memory → Take heap snapshot
|
|
- Navigate away and back multiple times
|
|
- Take another snapshot
|
|
- Compare - memory should not continuously grow
|
|
- assets are rebuilt automatically on filesystem change |