From 0e322a2403e8a367506590264230bb8159fecd53 Mon Sep 17 00:00:00 2001 From: Graham McInitre Date: Tue, 21 Jul 2026 11:01:27 -0500 Subject: [PATCH] docs: add structured documentation covering architecture, contexts, web, infra, and dev setup --- docs/README.md | 41 ++++++++ docs/architecture/ingestion-pipeline.md | 66 ++++++++++++ docs/architecture/overview.md | 92 +++++++++++++++++ docs/contexts/accounts.md | 73 ++++++++++++++ docs/contexts/devices.md | 51 ++++++++++ docs/contexts/packets.md | 100 ++++++++++++++++++ docs/development/setup.md | 129 ++++++++++++++++++++++++ docs/infrastructure/deployment.md | 73 ++++++++++++++ docs/infrastructure/observability.md | 65 ++++++++++++ docs/web/api-rest.md | 56 ++++++++++ docs/web/channels.md | 28 +++++ docs/web/liveview-map.md | 81 +++++++++++++++ docs/web/liveview-pages.md | 88 ++++++++++++++++ 13 files changed, 943 insertions(+) create mode 100644 docs/README.md create mode 100644 docs/architecture/ingestion-pipeline.md create mode 100644 docs/architecture/overview.md create mode 100644 docs/contexts/accounts.md create mode 100644 docs/contexts/devices.md create mode 100644 docs/contexts/packets.md create mode 100644 docs/development/setup.md create mode 100644 docs/infrastructure/deployment.md create mode 100644 docs/infrastructure/observability.md create mode 100644 docs/web/api-rest.md create mode 100644 docs/web/channels.md create mode 100644 docs/web/liveview-map.md create mode 100644 docs/web/liveview-pages.md diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..1095def --- /dev/null +++ b/docs/README.md @@ -0,0 +1,41 @@ +# aprs.me Documentation + +APRS packet visualization and tracking platform built with Phoenix LiveView + PostGIS. + +## Architecture + +- [Architecture Overview](architecture/overview.md) — System components, data flow, and design decisions +- [Ingestion Pipeline](architecture/ingestion-pipeline.md) — APRS-IS → GenStage → Postgres → PubSub broadcast + +## Domain Contexts + +- [Accounts](contexts/accounts.md) — User registration, authentication, session management +- [Packets](contexts/packets.md) — Packet storage, querying, replay, weather, and spatial search +- [Devices](contexts/devices.md) — APRS device identification and registry + +## Web Layer + +- [LiveView Map](web/liveview-map.md) — MapLive architecture, real-time streaming, components +- [LiveView Pages](web/liveview-pages.md) — Packet streams, weather, status, info, auth pages +- [REST API](web/api-rest.md) — API v1 endpoints (callsign lookup, weather nearby) +- [Mobile WebSocket](web/channels.md) — MobileChannel protocol for iOS/Android apps + +## Infrastructure + +- [Deployment](infrastructure/deployment.md) — Kubernetes (ArgoCD), Fly.io, Docker, Nix dev shell +- [Observability](infrastructure/observability.md) — PromEx metrics, error tracking, telemetry + +## Development + +- [Setup Guide](development/setup.md) — Local dev environment, required services, tooling + +## Features & Plans + +- [Tracked Callsign Behavior](tracked-callsign-behavior.md) +- [Trail Line Visualization](features/trail-line-visualization.md) +- [Plans](plans/) — Design documents and implementation plans +- [Improvement Todos](improvement-todos.md) + +## API + +- [Mobile API Reference](mobile-api.md) — WebSocket message format for mobile clients diff --git a/docs/architecture/ingestion-pipeline.md b/docs/architecture/ingestion-pipeline.md new file mode 100644 index 0000000..0aa5074 --- /dev/null +++ b/docs/architecture/ingestion-pipeline.md @@ -0,0 +1,66 @@ +# Ingestion Pipeline + +The packet ingestion pipeline moves raw APRS data from the APRS-IS network into Postgres and broadcasts to connected clients. + +## Components + +### Aprsme.Is (TCP Connection) + +- **File:** `lib/aprsme/is/is.ex` +- **Type:** GenServer +- **Role:** Maintains TCP connection to APRS-IS, authenticates with callsign + passcode, receives raw APRS frames +- **Parsing:** Uses `aprs.parse/1` (external Gleam library) to decode APRS frames +- **Filtering:** Configurable APRS-IS filter (by callsign, area, distance) via `Aprsme.Is.LoginParams` +- **Backpressure:** Responds to `{:backpressure, :activate | :deactivate}` signals from `PacketProducer` — sets TCP socket to `:passive` mode when buffer is full + +### PacketProducer (GenStage) + +- **File:** `lib/aprsme/packet_producer.ex` +- **Type:** GenStage producer +- **Buffer:** Erlang `:queue` with water-mark backpressure + - **High water** (>80% capacity): signals `Aprsme.Is` to activate backpressure + - **Low water** (<30% capacity): signals `Aprsme.Is` to deactivate backpressure +- **Demand-driven:** Only emits packets when consumers are ready + +### PacketConsumer (GenStage Consumer) + +- **File:** `lib/aprsme/packet_consumer.ex` +- **Type:** GenStage consumer (pooled via `PacketConsumerPool`) +- **Processing pipeline per packet:** + 1. Sanitize — `PacketSanitizer.sanitize_packet/1` (overflow prevention) + 2. Normalize — `EncodingUtils` conversions (UTF-8, latin1, floats, decimals) + 3. Extract position — lat/lon → PostGIS geometry + 4. Extract device — `DeviceParser.extract_device_id/1` + 5. Filter fields — `PacketFieldWhitelist` trims to schema columns + 6. Insert — `Repo.insert_all` with `on_conflict: :nothing` (idempotent) + 7. Broadcast — async dispatch to PubSub systems + +### Broadcasting After Insert + +After successful insert, each packet is broadcast to three PubSub layers: + +1. **StreamingPacketsPubSub** — ETS-based, clients subscribe with bounds +2. **SpatialPubSub** — Grid-indexed (1° cells), viewport-filtered delivery +3. **Phoenix.PubSub** — Legacy per-topic broadcasts: + - `"postgres:aprsme_packets"` — global packet stream + - `"packets:#{callsign}"` — per-callsign updates + - `"weather:#{callsign}"` — per-weather-station updates + +Broadcast execution is async via `BroadcastTaskSupervisor`. + +## Error Handling + +Packets that fail parsing or validation are stored in `badpackets` table via `Packets.store_bad_packet/2` with error type/message, viewable at `/badpackets` page. + +## Configuration + +- APRS-IS server, callsign, passcode set via application config +- Consumer pool size configurable per environment +- Retention period configurable via `CleanupScheduler` + +## Performance Notes + +- `PreparedQueries` module caches frequently-used query plans +- `InsertOptimizer` batch-tunes insert operations +- `PacketConsumer` uses `insert_all` (not individual inserts) for throughput +- Daily table partitioning via `PartitionManager` keeps index sizes manageable diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md new file mode 100644 index 0000000..36f9cde --- /dev/null +++ b/docs/architecture/overview.md @@ -0,0 +1,92 @@ +# Architecture Overview + +## System Purpose + +Real-time visualization of APRS (Automatic Packet Reporting System) packets on an interactive map. Tracks amateur radio stations, weather data, and station telemetry via APRS-IS network feed. + +## Technology Stack + +| Layer | Technology | +|---|---| +| Language | Elixir 1.17+ | +| Web Framework | Phoenix 1.8 + LiveView 1.2 | +| Database | PostgreSQL + PostGIS | +| APRS Parsing | `aprs` (Gleam BEAM library) | +| Map Client | Leaflet (lazy-loaded) | +| Charts | Chart.js (lazy-loaded) | +| Metrics | PromEx + Telemetry | +| Deployment | Kubernetes (ArgoCD), Fly.io, Docker | + +## High-Level Data Flow + +``` +APRS-IS Network (TCP) + │ + ▼ +Aprsme.Is (GenServer — TCP connection manager) + │ raw APRS frames parsed via aprs.parse/1 + ▼ +PacketProducer (GenStage — water-mark backpressure) + │ buffers incoming packets, signals backpressure upstream + ▼ +PacketConsumer pool (GenStage consumers) + │ sanitize → normalize → insert into Postgres + │ broadcast to PubSub + ├──► Postgres packets table (daily-partitioned) + └──► Broadcasting + ├── SpatialPubSub (viewport-filtered, grid-indexed) + ├── StreamingPacketsPubSub (ETS-based, bounds-filtered) + └── Phoenix.PubSub (per-callsign topics) + │ + ▼ + LiveView clients (MapLive, PacketsLive, InfoLive) + Mobile WebSocket clients (MobileChannel) +``` + +## Supervision Tree + +``` +Aprsme.Application +├── Aprsme.Repo +├── Phoenix.PubSub +├── Aprsme.Cache / DeviceCache / WeatherCache / RegexCache +├── Aprsme.RateLimiter +├── Aprsme.PacketPipelineSupervisor +│ ├── PacketProducer +│ └── PacketConsumerPool +│ └── PacketConsumer (×N) +├── Aprsme.BroadcastTaskSupervisor +├── Aprsme.Is.IsSupervisor +│ └── Aprsme.Is +├── Aprsme.SpatialPubSub +├── Aprsme.StreamingPacketsPubSub +├── Aprsme.CleanupScheduler +├── Aprsme.PartitionManager +├── Aprsme.PostgresNotifier +├── Aprsme.ConnectionMonitor +├── Aprsme.SignalHandler +├── Aprsme.ShutdownHandler +├── Aprsme.PromEx +├── Cluster modules (optional, via libcluster) +└── AprsmeWeb.Endpoint (Bandit) +``` + +## Key Design Decisions + +### Database Partitioning +The `packets` table is **daily-partitioned** via `PartitionManager`. High ingestion volume makes partitioning essential for query performance and retention cleanup. + +### Spatial Filtering +Rather than broadcasting every packet to every client, `SpatialPubSub` maintains a 1° grid spatial index. Clients register their map viewport bounds; only packets intersecting those bounds are delivered. + +### Backpressure +`PacketProducer` uses water-mark backpressure (>80% buffer → pause TCP; <30% → resume) to prevent memory exhaustion under load spikes. + +### Symbol System +APRS symbols are rendered client-side from sprite sheets. `AprsSymbol` provides server-side symbol table lookups for info pages. + +### Clustering +Optional multi-node deployment via `libcluster`. Uses leader election to designate the node that maintains the APRS-IS connection. Packets are relayed to other nodes via `PacketDistributor`. + +### Lazy Asset Loading +The map bundle (Leaflet + plugins) and chart bundle (Chart.js) are loaded on-demand only for pages that need them, keeping initial page load small. diff --git a/docs/contexts/accounts.md b/docs/contexts/accounts.md new file mode 100644 index 0000000..f8529ae --- /dev/null +++ b/docs/contexts/accounts.md @@ -0,0 +1,73 @@ +# Accounts Context + +## Overview + +**Module:** `Aprsme.Accounts` (`lib/aprsme/accounts.ex`) + +Handles user registration, authentication, email confirmation, password reset, and session management. + +## Schemas + +### User (`lib/aprsme/accounts/user.ex`) + +| Field | Type | Notes | +|---|---|---| +| `id` | binary_id | Primary key | +| `email` | citext | Case-insensitive, unique | +| `callsign` | string | Optional amateur radio callsign | +| `hashed_password` | string | Bcrypt-hashed | +| `confirmed_at` | utc_datetime | Email confirmation timestamp | +| `password` | virtual | Transient, used for changesets only | + +### UserToken (`lib/aprsme/accounts/user_token.ex`) + +| Field | Type | Notes | +|---|---|---| +| `id` | binary_id | Primary key | +| `token` | binary | Bcrypt-hashed token | +| `context` | string | `"session"`, `"confirm"`, `"reset_password"`, `"change_email"` | +| `sent_to` | string | Email or identifier | +| `user_id` | binary_id | FK → `users.id` | + +## Public API + +### Registration +- `register_user/1` — Create user from registration params +- `change_user_registration/2` — Changeset for registration form + +### Authentication +- `get_user_by_email_and_password/2` — Verify credentials +- `generate_user_session_token/1` — Create session token +- `get_user_by_session_token/1` — Lookup user from token +- `delete_user_session_token/1` — Logout + +### Email Confirmation +- `deliver_user_confirmation_instructions/2` — Send confirmation email +- `confirm_user/1` — Mark user as confirmed + +### Password Reset +- `deliver_user_reset_password_instructions/2` — Send reset email +- `get_user_by_reset_password_token/1` — Validate reset token +- `reset_user_password/2` — Update password + +### Account Settings +- `change_user_email/2`, `update_user_email/2` — Email change flow +- `change_user_callsign/2`, `update_user_callsign/3` — Callsign management +- `change_user_password/2`, `update_user_password/3` — Password change + +## Email Delivery + +Uses `Aprsme.Accounts.UserNotifier` which delegates to `Aprsme.Mailer` (Swoosh with custom Resend adapter). + +## Web Integration + +- `AprsmeWeb.UserAuth` module provides plug-based and LiveView `on_mount` hooks +- Session stored in signed+encrypted cookie (`_aprs_key`) +- "Remember me" cookie (60-day expiry, signed) +- Token-based session with fixation prevention (renew on login/logout) + +## Notes + +- No FK relationship between `users` ↔ `packets` +- Authentication is optional; most pages are publicly accessible without login +- Authenticated pages: `/users/settings`, `/dashboard` (LiveDashboard), `/errors` (ErrorTracker) diff --git a/docs/contexts/devices.md b/docs/contexts/devices.md new file mode 100644 index 0000000..a648e0d --- /dev/null +++ b/docs/contexts/devices.md @@ -0,0 +1,51 @@ +# Devices Context + +## Overview + +Identifies APRS equipment from packet data, maintaining a local registry of known devices. + +## Schema: `Aprsme.Devices` (`lib/aprsme/devices.ex`) + +Stores known APRS device information in the `devices` table: + +| Field | Type | Notes | +|---|---|---| +| `identifier` | string | Unique device identifier | +| `class` | string | Device class (e.g., "tracker", "handheld") | +| `model` | string | Model name | +| `vendor` | string | Manufacturer name | +| `os` | string | Operating system | +| `contact` | string | Vendor/author contact | +| `features` | {:array, :string} | Feature tags | + +No FK constraint: `device_identifier` on the `packets` table is a free-text string, not a DB-level reference. + +## DeviceIdentification (`lib/aprsme/device_identification.ex`) + +### Static Identification +Matches known MIC-E symbol patterns (regex-based) to identify ~20 common device types from packet data. + +### Remote Registry +Fetches device database from `aprs-deviceid.aprsfoundation.org`: +- `maybe_refresh_devices/0` — Weekly refresh cycle +- `fetch_and_upsert_devices/0` — HTTP fetch wrapped in `CircuitBreaker`, upserts results into `devices` table +- `lookup_device_by_identifier/1` — Wildcard-matching lookup, delegates to `DeviceCache` + +### Caching +- `DeviceCache` GenServer caches device data, refreshed daily +- Backed by `Aprsme.Cache` abstraction + +## DeviceParser (`lib/aprsme/device_parser.ex`) + +Extracts device identifiers from incoming packets by probing: +1. `data_extended.device_identifier` field +2. `destination` field +3. `data_extended.symbol_table_id` + `symbol_code` combination + +Called during packet ingestion pipeline in `PacketConsumer`. + +## Resilience + +- `CircuitBreaker` wraps remote HTTP fetch, preventing cascading failures +- Cache ensures device lookups work even when remote source is unavailable +- Static patterns provide fallback identification without network calls diff --git a/docs/contexts/packets.md b/docs/contexts/packets.md new file mode 100644 index 0000000..37500b4 --- /dev/null +++ b/docs/contexts/packets.md @@ -0,0 +1,100 @@ +# Packets Context + +## Overview + +**Module:** `Aprsme.Packets` (`lib/aprsme/packets.ex`) +**Behaviour:** `Aprsme.PacketsBehaviour` + +Core domain for APRS packet storage, retrieval, querying, and management. + +## Schema: `Aprsme.Packet` (`lib/aprsme/packet.ex`) + +The `packets` table is **daily-partitioned** on `received_at`. Composite primary key: `{id, received_at}`. + +### Key Field Groups + +| Group | Fields | +|---|---| +| **Identity** | `id`, `sender` (callsign), `base_callsign`, `ssid`, `destination`, `path` | +| **Position** | `lat` (decimal), `lon` (decimal), `location` (PostGIS geometry), `has_position` (boolean), `dao` (map) | +| **Symbol** | `symbol_code`, `symbol_table_id` | +| **Weather** | `temperature`, `humidity`, `wind_speed`, `wind_direction`, `wind_gust`, `pressure`, `rain_1h`, `rain_24h`, `rain_since_midnight`, `snow`, `has_weather` | +| **Equipment** | `manufacturer`, `equipment_type`, `course`, `speed`, `altitude` | +| **Messaging** | `addressee`, `message_text`, `message_number` | +| **Other** | `data_type`, `region`, `comment`, `raw_packet`, `device_identifier`, `data` (JSONB) | + +### Changeset Pipeline + +The `changeset/2` function auto-computes: +- `has_position` and `has_weather` booleans +- `location` PostGIS point from lat/lon +- Symbol normalization +- Course/wind_direction normalization +- Display-only fields swept into `data` JSONB column + +## Schema: `Aprsme.BadPacket` (`lib/aprsme/bad_packet.ex`) + +Failed packets stored in `badpackets` table: +- `raw_packet` — original APRS frame +- `error_message`, `error_type` — parsing/storage error details +- `attempted_at` — timestamp + +## Public API + +### Storage +- `store_packet/1` — Sanitize → build attrs → insert; errors → `badpackets` +- `store_bad_packet/2` — Record parse/insert failures + +### Querying +- `get_recent_packets/1` — Cursor-paginated, latest first +- `get_recent_packets_for_map/1` — Lightweight map-optimized query (~22 columns) +- `get_latest_packet_for_callsign/1` — Single callsign latest +- `get_latest_positions_for_callsigns/1` — Batch latest positions +- `get_nearby_stations/3,4` — KNN spatial query via PostGIS `<->` operator +- `get_weather_packets/4` — Time-bound weather data +- `get_other_ssids/1` — Find SSID variants of a base callsign +- `get_total_packet_count/0` — Via PostgreSQL function + +### Replay +- `get_packets_for_replay/1` — Historical packets filtered by callsign/bounds/region/time +- `stream_packets_for_replay/1` — Lazy stream with timing preservation +- `get_historical_packet_count/1` — Area packet density + +### Weather +- `get_latest_weather_packet/1` — Latest weather for station +- `has_weather_packets?/1`, `weather_callsigns/1` — Weather presence queries + +### Maintenance +- `clean_old_packets/0`, `clean_packets_older_than/1` — Retention cleanup +- `get_oldest_packet_timestamp/0` — Oldest data timestamp + +## Submodules + +### QueryBuilder (`lib/aprsme/packets/query_builder.ex`) +Composable Ecto query helpers: `with_position/1`, `for_callsign/2`, `within_bounds/2`, `weather_only/1`, `chronological/1`, `recent_first/1`, `select_map_fields/1`, etc. + +### PreparedQueries (`lib/aprsme/packets/prepared_queries.ex`) +High-performance prepared statements for hot paths: KNN queries, weather queries, bounds-filtered queries, callsign lookups. + +### Clustering (`lib/aprsme/packets/clustering.ex`) +Grid-based heatmap clustering for low-zoom map views (zoom ≤ 8). Reduces thousands of points to manageable grid cells. + +## Supporting Utilities + +| Module | Purpose | +|---|---| +| `Aprsme.PacketSanitizer` | Prevents DB field overflow | +| `Aprsme.PacketFieldWhitelist` | Filters fields for `insert_all` | +| `Aprsme.Encoding` / `EncodingUtils` | String/data normalization | +| `Aprsme.Convert` | Unit conversions | +| `Aprsme.Callsign` | Callsign parsing, normalization, SSID extraction | +| `Aprsme.Maidenhead` | Maidenhead grid locator conversion | +| `Aprsme.GeoUtils` | Geographic calculations | +| `Aprsme.WeatherCache` | ETS cache for weather callsign presence (TTL 5 min) | + +## Database + +- `packets` table: daily-partitioned, managed by `Aprsme.PartitionManager` +- `badpackets` table: standard table for error logging +- PostGIS extensions required for spatial queries +- `get_packet_count()` PostgreSQL function for efficient counting diff --git a/docs/development/setup.md b/docs/development/setup.md new file mode 100644 index 0000000..7a234f9 --- /dev/null +++ b/docs/development/setup.md @@ -0,0 +1,129 @@ +# Development Setup + +## Prerequisites + +- Elixir 1.17+ +- Erlang/OTP 26+ +- PostgreSQL 16+ with PostGIS extension +- Node.js 20+ (for asset compilation) + +## Quick Start + +```bash +# Clone and cd +git clone +cd aprs.me + +# Install dependencies and set up database +mix setup + +# Start dev server +mix phx.server +# or with IEx: +iex -S mix phx.server +``` + +The application will be available at `http://localhost:4000`. + +## Nix Development Environment (Optional) + +A reproducible development environment is available via Nix: + +```bash +nix develop +# or with direnv: +direnv allow +``` + +This provides all required tooling (Elixir, Erlang, PostgreSQL, Node.js) without manual installation. + +## Database Setup + +The `mix setup` command: +1. Installs Hex and Rebar dependencies +2. Fetches Elixir dependencies +3. Creates the development database +4. Runs migrations +5. Seeds device data + +### Manual Database Setup + +```bash +mix ecto.create +mix ecto.migrate +mix run priv/repo/seeds.exs +``` + +**Important:** PostGIS extension must be enabled on the database. The migration pipeline handles this, but ensure your PostgreSQL installation includes PostGIS. + +## Running Tests + +```bash +# Full test suite +mix test + +# Watch mode (auto-rerun on changes) +mix test.watch + +# Coverage report +mix coveralls.html +# View at cover/excoveralls.html +``` + +## Code Quality + +```bash +# Format code +mix format + +# Static analysis +mix credo --strict + +# Type checking +mix dialyzer + +# Security scan +mix sobelow +``` + +## Asset Compilation + +Frontend assets use ESBuild + Tailwind CSS v4: + +```bash +# Development (watch mode) +mix assets.build + +# Production +MIX_ENV=prod mix assets.deploy +``` + +JavaScript entry points are in `assets/js/`: +- `app.ts` — Core application (always loaded) +- `map.ts` — Map functionality (lazy-loaded) +- `map_helpers.ts` — Leaflet utilities +- `map_fixes.ts` — Leaflet bug workarounds + +CSS: Tailwind v4 in `assets/css/app.css` with custom dark mode and Leaflet overrides. + +## Useful Commands + +| Command | Purpose | +|---|---| +| `mix phx.server` | Start dev server | +| `iex -S mix phx.server` | Start with interactive shell | +| `mix test` | Run test suite | +| `mix test.watch` | Auto-rerun tests on changes | +| `mix format` | Format Elixir code | +| `mix credo --strict` | Lint check | +| `mix dialyzer` | Type checking | +| `mix sobelow` | Security scan | +| `mix coveralls.html` | Test coverage report | +| `mix phx.routes` | List all routes | +| `mix ecto.migrate` | Run pending migrations | +| `mix ecto.rollback` | Rollback last migration | +| `mix parse_file ` | Parse raw APRS log file | + +## Project Structure + +See [AGENTS.md](../../AGENTS.md) for coding conventions, and [Architecture Overview](../architecture/overview.md) for system design. diff --git a/docs/infrastructure/deployment.md b/docs/infrastructure/deployment.md new file mode 100644 index 0000000..3339bb0 --- /dev/null +++ b/docs/infrastructure/deployment.md @@ -0,0 +1,73 @@ +# Deployment + +## Platforms + +### Kubernetes (ArgoCD) + +Primary production deployment target. Manifests in `k8s/`. + + + +### Fly.io + +Alternative deployment option. Configuration in `fly.toml`. + + + +### Docker + +Multi-stage Docker build (`Dockerfile`). +- Build stage: Elixir compilation + asset bundling +- Release stage: Minimal runtime with Erlang/Elixir release + +## Release Management + +### Building + +```bash +MIX_ENV=prod mix assets.deploy +mix release +``` + +### Database Migrations + +`Aprsme.Release` module handles migrations during release startup. + + + +## Environment Configuration + +Configuration files in `config/`: +- `config.exs` — Base configuration +- `dev.exs` — Development overrides +- `prod.exs` — Production defaults +- `runtime.exs` — Runtime configuration (from env vars) + +### Key Environment Variables + + + +## Development Environment + +### Nix Shell + +Nix-based reproducible dev environment via `flake.nix` / `shell.nix`. Provides Elixir, Erlang, PostgreSQL, and all tooling. + +### Dev Containers + +`.devcontainer/` configuration for VS Code / GitHub Codespaces. + +## Clustering + +Optional multi-node deployment via `libcluster`: +- Leader election determines which node maintains APRS-IS connection +- Packet distribution relayed across nodes +- Configuration in cluster modules under `lib/aprsme/cluster/` + +## Health Checks + +- `/health` — Kubernetes liveness (basic) and readiness (DB + PubSub, shutdown-aware) +- `/ready` — App readiness check +- `/status.json` — JSON status endpoint + +Handled by `HealthCheck` plug before router. diff --git a/docs/infrastructure/observability.md b/docs/infrastructure/observability.md new file mode 100644 index 0000000..bf5ddc4 --- /dev/null +++ b/docs/infrastructure/observability.md @@ -0,0 +1,65 @@ +# Observability + +## Metrics + +### PromEx (`Aprsme.PromEx`) + +Prometheus-compatible metrics exposed at `/metrics` (PromEx plugin). +Custom plugin: `Aprsme.PromEx.Plugins.Aprsme` + + + +### Telemetry + +`telemetry_metrics` + `telemetry_poller` for application-level metrics: +- `Aprsme.Telemetry.DatabaseMetrics` — DB query performance +- `Aprsme.ApiMetrics` — API usage tracking + +## Error Tracking + +### ErrorTracker + +Self-hosted error tracking dashboard at `/errors` (authenticated). +Configuration: `error_tracker` hex package. + +### Error Handling + +- `Aprsme.ErrorHandler` — Application error handling +- `Aprsme.ErrorNotifier` — Telemetry-based error notification + +## Logging + +### Log Sanitization + +`Aprsme.LogSanitizer` — Strips PII from log output. + +### Log Filtering + +`AprsmeWeb.Plugs.LogFilter` — Filters noisy paths (`/health`, `/ready`, `/`) from access logs. + +## Monitoring + +### Connection Monitor + +`Aprsme.ConnectionMonitor` — Monitors APRS-IS connection health. + +### Status Page + +`/status` (StatusLive.Index) — Real-time system status dashboard: +- APRS-IS connection state +- Uptime +- Health score +- Cluster status (when clustered) + +### Analytics + +`Aprsme.PlausibleAnalytics` — Plausible analytics integration for page views. + +## Signal Handling + +- `Aprsme.SignalHandler` — OS signal handling (SIGTERM, SIGINT) +- `Aprsme.ShutdownHandler` — Graceful shutdown coordination + +## Deployment Notifications + +`Aprsme.DeploymentNotifier` — Notifies external systems of deploy events. diff --git a/docs/web/api-rest.md b/docs/web/api-rest.md new file mode 100644 index 0000000..d5e3945 --- /dev/null +++ b/docs/web/api-rest.md @@ -0,0 +1,56 @@ +# REST API v1 + +## Overview + +Public REST API for programmatic access to APRS packet data. + +**Base URL:** `/api/v1` +**Content-Type:** `application/json` +**Auth Pipeline:** `:api` — JSON accepts, rate-limited (100 req/min), API CSRF protection + +## Endpoints + +### `GET /api/v1/callsign/:callsign` + +Returns the latest packet for a given amateur radio callsign. + +**Controller:** `AprsmeWeb.Api.V1.CallsignController` +**View:** `AprsmeWeb.Api.V1.CallsignJSON` + +**Parameters:** +| Param | Type | Required | Description | +|---|---|---|---| +| `callsign` | string | yes | Valid amateur radio callsign (path param) | + + + +### `GET /api/v1/weather/nearby` + +Returns nearby weather station data. + +**Controller:** `AprsmeWeb.Api.V1.WeatherController` +**View:** `AprsmeWeb.Api.V1.WeatherJSON` + + + +## Error Handling + +API errors are rendered via `ErrorJSON` and `ChangesetJSON` view modules. The `FallbackController` handles generic error cases. + + + +## Rate Limiting + +API endpoints are rate-limited to 100 requests per minute per client via `RateLimiter` plug (ETS-based). + +## CSRF Protection + +API requests must include either: +- `X-Requested-With: XMLHttpRequest` header, or +- Valid CSRF token + +Enforced by `ApiCSRF` plug. + +## Interactive Docs + +A live API documentation page with interactive testing form is available at `/api` (`ApiDocsLive`). diff --git a/docs/web/channels.md b/docs/web/channels.md new file mode 100644 index 0000000..fad14eb --- /dev/null +++ b/docs/web/channels.md @@ -0,0 +1,28 @@ +# Mobile WebSocket Channel + +## Overview + +Real-time packet streaming for iOS/Android mobile applications. + +**Socket:** `MobileUserSocket` (`/mobile`, WebSocket only) +**Channel:** `MobileChannel` + +## Connection + +Mobile clients connect via WebSocket to `/mobile`: +``` +ws://host/mobile/websocket?vsn=2.0.0 +``` + +## Protocol + +See [mobile-api.md](../mobile-api.md) for the complete message format and protocol specification. + +## Architecture + +The mobile channel receives real-time packet broadcasts from the same PubSub sources as the web LiveView clients: +- `StreamingPacketsPubSub` — bounds-filtered +- `SpatialPubSub` — viewport-filtered +- Per-callsign Phoenix PubSub topics + +Packets are formatted into a mobile-optimized JSON structure and pushed to connected clients. diff --git a/docs/web/liveview-map.md b/docs/web/liveview-map.md new file mode 100644 index 0000000..a7ef709 --- /dev/null +++ b/docs/web/liveview-map.md @@ -0,0 +1,81 @@ +# LiveView Map + +## Overview + +**Module:** `AprsmeWeb.MapLive.Index` (~2050 lines, the largest LiveView) +**Routes:** `/` (main map), `/:callsign` (tracked station) + +The map is the primary user interface — a real-time Leaflet-based map displaying live APRS station positions, trails, weather data, and station information. + +## MapLive Architecture + +`MapLive.Index` is supported by 12 submodules under `lib/aprsme_web/live/map_live/`: + +``` +MapLive.Index +├── Components — Reusable UI (map container, slideover panel, packet list) +├── PopupComponent — Marker popup rendering (callsign, weather, timestamp) +├── DataBuilder — Constructs map data payloads for JS hooks +├── DisplayManager — Manages what's shown on map (stations, trails, clusters) +├── HistoricalLoader — Loads historical packet data for selected station +├── Navigation — Determines map center/zoom from URL params +├── PacketBatcher — Batches incoming real-time packets for rendering +├── PacketProcessor — Processes raw packets into display-ready data +├── RfPath — RF path visualization (digipeater hops) +├── UrlParams — URL parameter sync with map state +└── (Plus shared modules in live/shared/) +``` + +## Real-Time Updates + +MapLive subscribes to: +- `StreamingPacketsPubSub` — viewport-filtered new packets +- `SpatialPubSub` — grid-indexed spatial delivery +- Phoenix PubSub topics for callsign-specific updates + +Updates are batched via `PacketBatcher` to reduce DOM operations, then pushed to the Leaflet client via `push_event` (e.g., `"update_markers"`, `"update_clusters"`). + +## Client-Side + +The `APRSMap` JavaScript hook (`assets/js/map.ts`) handles: +- Leaflet map initialization with multiple tile layers +- Marker management (add, update, remove) +- Marker clustering at lower zoom levels +- Trail line rendering (`trail_manager.ts`) +- Viewport change tracking (pushed back to server for subscription bounds) +- URL parameter sync (center, zoom) + +## Lazy Loading + +The map bundle (Leaflet + plugins, ~300KB) is loaded only when a map page is visited, via `VendorLoader.loadMap()` in `assets/js/app.ts`. Non-map pages never download it. + +## Key Features + +### Station Tracking +When visiting `/:callsign`, the map centers on that station and subscribes to its specific PubSub topic. A slideover panel shows station details. + +### Trail Lines +Historical path of a station rendered as a polyline on the map. See [trail-line-visualization.md](../features/trail-line-visualization.md). + +### RF Path +Digipeater hop visualization showing the path an APRS packet took through the network. + +### Heatmap Clustering +At low zoom levels (≤8), individual markers are replaced by grid-based heatmap clusters via `Packets.Clustering` for performance. + +### Weather Overlay +Weather station markers with temperature/wind indicators. Clicking opens a popup with current conditions, linking to the full weather page. + +## Slideover Panel + +When a station is selected, a slideover panel displays: +- Latest position and timestamp +- Course, speed, altitude +- Device/manufacturer info +- Link to full info page (`/info/:callsign`) +- Link to packet history (`/packets/:callsign`) +- Weather data (if weather station) + +## URL State + +Map state (center, zoom, selected callsign) is synced to URL query parameters via `UrlParams`, enabling shareable map views and browser back/forward navigation. diff --git a/docs/web/liveview-pages.md b/docs/web/liveview-pages.md new file mode 100644 index 0000000..6c06ca9 --- /dev/null +++ b/docs/web/liveview-pages.md @@ -0,0 +1,88 @@ +# LiveView Pages + +Non-map LiveView pages in the application. + +## Packet Streams + +### PacketsLive.Index (`/packets`) +- Real-time stream of all incoming packets (max 100) +- Subscribes to `"postgres:aprsme_packets"` Phoenix PubSub topic +- Uses `stream/4` for efficient DOM updates + +### PacketsLive.CallsignView (`/packets/:callsign`) +- Filtered packet stream for a single callsign +- Subscribes to `"packets:#{callsign}"` topic +- Includes device information parsed per-packet + +## Station Info + +### InfoLive.Show (`/info/:callsign`) +- Comprehensive station information page +- Latest packet details, device identification +- Nearby stations via spatial query (neighbors) +- APRS path decoding — "heard by" and "stations heard" analysis +- Digipeater relationships +- APRS symbol rendering +- Embedded single-station map via `InfoMapComponent` (JS hook) +- Subscribes to `"packets:#{callsign}"` for live updates + +## Weather + +### WeatherLive.CallsignView (`/weather/:callsign`) +- Weather station data visualization with Chart.js charts +- Temperature, humidity, wind, pressure, rainfall trends +- Locale-aware unit conversions (metric/imperial) +- Subscribes to `"weather:#{callsign}"` for live updates +- Pushes `"update_weather_charts"` events to refresh Chart.js + +## Status + +### StatusLive.Index (`/status`) +- System status dashboard +- APRS-IS connection state, uptime, health score +- Cluster node information (when clustered) +- Polls every 5 seconds via `Process.send_after` +- Inline `render/1` (no template file) +- Subscribes to `"aprs_status"` topic + +## Bad Packets + +### BadPacketsLive.Index (`/badpackets`) +- Malformed/rejected packet viewer +- Shows raw packet data + error type/message +- Debounced refresh (2s delay) +- Subscribes to `"postgres:aprsme_events"` + +## Static Pages + +### AboutLive (`/about`) +- Simple about page, title assign only + +### ApiDocsLive (`/api`) +- API documentation with interactive testing form +- Inline render (~970 lines) +- Self-documenting with example requests/responses + +## Authentication Pages + +All auth LiveViews use inline `render/1`: + +| LiveView | Route | Auth Pipeline | +|---|---|---| +| `UserLoginLive` | `/users/log_in` | `redirect_if_user_is_authenticated` | +| `UserRegistrationLive` | `/users/register` | `redirect_if_user_is_authenticated` | +| `UserForgotPasswordLive` | `/users/reset_password` | `redirect_if_user_is_authenticated` | +| `UserResetPasswordLive` | `/users/reset_password/:token` | `redirect_if_user_is_authenticated` | +| `UserConfirmationLive` | `/users/confirm/:token` | `:current_user` | +| `UserConfirmationInstructionsLive` | `/users/confirm` | `:current_user` | +| `UserSettingsLive` | `/users/settings` | `require_authenticated_user` | + +## Shared LiveView Modules + +Located in `lib/aprsme_web/live/shared/`: + +- `BoundsUtils` — Map boundary calculations +- `PacketHandler` — Shared packet enrichment +- `ParamUtils` — URL parameter parsing +- `PacketUtils` — Shared packet querying +- `CoordinateUtils` — Coordinate math