100 lines
4.2 KiB
Markdown
100 lines
4.2 KiB
Markdown
# Packets Context
|
|
|
|
## Overview
|
|
|
|
**Module:** `Aprsme.Packets` (`lib/aprsme/packets.ex`)
|
|
**Behaviour:** `Aprsme.PacketsBehaviour`
|
|
|
|
Core domain for APRS packet storage, retrieval, querying, and management.
|
|
|
|
## Schema: `Aprsme.Packet` (`lib/aprsme/packet.ex`)
|
|
|
|
The `packets` table is **daily-partitioned** on `received_at`. Composite primary key: `{id, received_at}`.
|
|
|
|
### Key Field Groups
|
|
|
|
| Group | Fields |
|
|
|---|---|
|
|
| **Identity** | `id`, `sender` (callsign), `base_callsign`, `ssid`, `destination`, `path` |
|
|
| **Position** | `lat` (decimal), `lon` (decimal), `location` (PostGIS geometry), `has_position` (boolean), `dao` (map) |
|
|
| **Symbol** | `symbol_code`, `symbol_table_id` |
|
|
| **Weather** | `temperature`, `humidity`, `wind_speed`, `wind_direction`, `wind_gust`, `pressure`, `rain_1h`, `rain_24h`, `rain_since_midnight`, `snow`, `has_weather` |
|
|
| **Equipment** | `manufacturer`, `equipment_type`, `course`, `speed`, `altitude` |
|
|
| **Messaging** | `addressee`, `message_text`, `message_number` |
|
|
| **Other** | `data_type`, `region`, `comment`, `raw_packet`, `device_identifier`, `data` (JSONB) |
|
|
|
|
### Changeset Pipeline
|
|
|
|
The `changeset/2` function auto-computes:
|
|
- `has_position` and `has_weather` booleans
|
|
- `location` PostGIS point from lat/lon
|
|
- Symbol normalization
|
|
- Course/wind_direction normalization
|
|
- Display-only fields swept into `data` JSONB column
|
|
|
|
## Schema: `Aprsme.BadPacket` (`lib/aprsme/bad_packet.ex`)
|
|
|
|
Failed packets stored in `badpackets` table:
|
|
- `raw_packet` — original APRS frame
|
|
- `error_message`, `error_type` — parsing/storage error details
|
|
- `attempted_at` — timestamp
|
|
|
|
## Public API
|
|
|
|
### Storage
|
|
- `store_packet/1` — Sanitize → build attrs → insert; errors → `badpackets`
|
|
- `store_bad_packet/2` — Record parse/insert failures
|
|
|
|
### Querying
|
|
- `get_recent_packets/1` — Cursor-paginated, latest first
|
|
- `get_recent_packets_for_map/1` — Lightweight map-optimized query (~22 columns)
|
|
- `get_latest_packet_for_callsign/1` — Single callsign latest
|
|
- `get_latest_positions_for_callsigns/1` — Batch latest positions
|
|
- `get_nearby_stations/3,4` — KNN spatial query via PostGIS `<->` operator
|
|
- `get_weather_packets/4` — Time-bound weather data
|
|
- `get_other_ssids/1` — Find SSID variants of a base callsign
|
|
- `get_total_packet_count/0` — Via PostgreSQL function
|
|
|
|
### Replay
|
|
- `get_packets_for_replay/1` — Historical packets filtered by callsign/bounds/region/time
|
|
- `stream_packets_for_replay/1` — Lazy stream with timing preservation
|
|
- `get_historical_packet_count/1` — Area packet density
|
|
|
|
### Weather
|
|
- `get_latest_weather_packet/1` — Latest weather for station
|
|
- `has_weather_packets?/1`, `weather_callsigns/1` — Weather presence queries
|
|
|
|
### Maintenance
|
|
- `clean_old_packets/0`, `clean_packets_older_than/1` — Retention cleanup
|
|
- `get_oldest_packet_timestamp/0` — Oldest data timestamp
|
|
|
|
## Submodules
|
|
|
|
### QueryBuilder (`lib/aprsme/packets/query_builder.ex`)
|
|
Composable Ecto query helpers: `with_position/1`, `for_callsign/2`, `within_bounds/2`, `weather_only/1`, `chronological/1`, `recent_first/1`, `select_map_fields/1`, etc.
|
|
|
|
### PreparedQueries (`lib/aprsme/packets/prepared_queries.ex`)
|
|
High-performance prepared statements for hot paths: KNN queries, weather queries, bounds-filtered queries, callsign lookups.
|
|
|
|
### Clustering (`lib/aprsme/packets/clustering.ex`)
|
|
Grid-based heatmap clustering for low-zoom map views (zoom ≤ 8). Reduces thousands of points to manageable grid cells.
|
|
|
|
## Supporting Utilities
|
|
|
|
| Module | Purpose |
|
|
|---|---|
|
|
| `Aprsme.PacketSanitizer` | Prevents DB field overflow |
|
|
| `Aprsme.PacketFieldWhitelist` | Filters fields for `insert_all` |
|
|
| `Aprsme.Encoding` / `EncodingUtils` | String/data normalization |
|
|
| `Aprsme.Convert` | Unit conversions |
|
|
| `Aprsme.Callsign` | Callsign parsing, normalization, SSID extraction |
|
|
| `Aprsme.Maidenhead` | Maidenhead grid locator conversion |
|
|
| `Aprsme.GeoUtils` | Geographic calculations |
|
|
| `Aprsme.WeatherCache` | ETS cache for weather callsign presence (TTL 5 min) |
|
|
|
|
## Database
|
|
|
|
- `packets` table: daily-partitioned, managed by `Aprsme.PartitionManager`
|
|
- `badpackets` table: standard table for error logging
|
|
- PostGIS extensions required for spatial queries
|
|
- `get_packet_count()` PostgreSQL function for efficient counting
|