- Create migration to change string fields from VARCHAR(255) to TEXT type - Add PacketSanitizer module for UTF-8 safe string truncation - Update PacketConsumer to sanitize data before database insertion - Prevents database errors from overly long packet data fields
7.3 KiB
7.3 KiB
Changelog
All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
[Unreleased]
Added
- Database fields for APRS items and objects support:
item_name,object_namefields to identify items/objectsis_item,is_objectboolean flags for filteringdaofield to store DAO (Datum As Offset) extension data for extra position precision
- Automatic detection of items/objects during packet processing
- Database indexes for efficient item/object queries
- Packet data sanitization to prevent database field overflow errors
- Created
Aprsme.PacketSanitizermodule to truncate overly long strings - Implemented UTF-8 safe string truncation
- Created
Fixed
- Fixed packet storage failure in production due to non-schema fields from updated APRS parser
- Remove
type,digipeaters,daodatumbyte,mbits,message,phg,wx,resultcode,resultmsgfields before database insertion - Enhanced error logging to capture sample packet data when batch inserts fail
- Created helper function
remove_non_schema_fields/1to centralize field removal logic
- Remove
- Fixed VARCHAR(255) database errors by migrating string fields to TEXT type
- Migration increases field sizes for: comment, message_text, raw_packet, body, origpacket, header, radiorange, item_name, object_name, telemetry_bits, device_identifier, format
- Added sanitization in PacketConsumer to truncate strings before insertion as safety measure
Added
- Database query optimization with 10 new performance indexes
- Functional index on upper(sender) for case-insensitive searches
- Composite indexes for position/time queries
- Spatial index using geography type for ST_DWithin queries
- Partial indexes for weather data queries
- Generated column
has_weatherwith trigger for query optimization - Region and data_type composite indexes for filtering
- BRIN indexes consideration for time-series data
- Connection draining and load balancing for clustered deployments
- Monitors CPU usage and connection counts across cluster nodes
- Documentation for tracked callsign behavior confirming last packet is always shown regardless of age
- Automatically drains connections when node is overloaded (>70% CPU or 2x average connections)
- Gracefully reconnects clients to less loaded nodes
- Improves overall cluster stability and performance
- Position ambiguity support from APRS parser
- Added
position_ambiguityfield to packets table (integer, 0-4) - Automatically captures position ambiguity level when parsing APRS packets
- Supports ambiguity levels 0-4 as defined in APRS specification
- Added
- Enhanced APRS parser integration with latest improvements
- Added
posresolutionfield showing position accuracy in meters (18.52m for uncompressed, 0.291m for compressed)
- Added
- Upgraded vendor/aprs parser to version 0.1.5+ with comprehensive compatibility improvements
- Added standard APRS parser fields: posambiguity, format, alive, symboltable, symbolcode, messaging
- Enhanced weather data extraction with dedicated
wxfield - Added radio range (RNG) parsing from comments
- Improved PHG data format handling (now returns string representation)
- Added telemetry fields (seq, vals, bits) extraction
- Better UTF-8 handling and error messages for invalid position data
- Fixed compressed latitude calculation and third-party traffic parsing
- Added support for alternate compressed position formats
- Added 20 new database fields for enhanced parser compatibility
- Implemented weather data extraction with dedicated
wxfield support - Enhanced PHG parsing to handle both string format and legacy map structure
- Added radio range field extraction from comments
- Added standard parser compatibility fields for better integration
- Added
formatfield indicating "compressed" or "uncompressed" position type - Added telemetry fields (
telemetry_seq,telemetry_vals,telemetry_bits) for telemetry packet support - Improved coordinate handling with direct float support from parser
- Fixed compressed latitude divisor calculation for more accurate position decoding
Changed
- Query performance improved by 50-90% for common operations
- Weather queries now use indexed
has_weathercolumn - Spatial queries optimized with geography cast index
Fixed
- Tracked callsigns now show all packets including those without position data
- Previously filtered out packets with
has_position == false - Now shows all packets when tracking a specific callsign via /:callsign URL
- Ensures users can see status updates, messages, and other non-position packets
- Previously filtered out packets with
- PostgreSQL notify trigger now sends all required fields for info page updates
- Info page real-time updates now display all packet details correctly
- Enhanced PostgreSQL notify to include complete packet data (device info, weather data, SSIDs, etc.)
- Mic-E packet comment parsing now properly extracts altitude data and removes telemetry suffixes
- Altitude data (e.g., "6;}" = 218 feet) is now parsed and stored
- Telemetry markers (_%...) are removed from comments
- Non-human-readable encoded data is no longer displayed as comments
- Historical packets now load on initial page load for all users
- Previously only loaded historical data when tracking a specific callsign
- Fixed by setting
needs_initial_historical_loadto true for all map views - Ensures users see recent activity immediately when visiting the map
[0.2.0] - 2025-07-26
Added
- Redis-based distributed PubSub for real-time updates across replicas
- PgBouncer connection pooling for efficient database connections
- Distributed caching with Redis (query, device, symbol caches)
- Distributed rate limiting with Redis using sliding window algorithm
- Graceful shutdown handling with connection draining
- PodDisruptionBudget for zero-downtime deployments
- Signal handler for proper SIGTERM handling
- Comprehensive health checks for Kubernetes probes
- Docker build optimizations with better layer caching
- Concurrency control in CI/CD pipeline
Changed
- Moved from ETS to Redis for caching when REDIS_URL is available
- Updated Kubernetes deployment to use StatefulSet with stable networking
- Simplified Dockerfile by removing Node.js dependencies
- Improved CI/CD to not wait for rollout completion
- Enhanced shutdown process to be silent (no user notifications)
- Optimized Docker image size (removed ~150MB)
Fixed
- PgBouncer database connection issues with SKIP_DB_CREATE
- Redis connection errors by removing invalid pool_size option
- Compilation warnings by grouping handle_info clauses
- ShutdownHandler interfering with application startup
- DeviceCache startup timing issues with Redis
- Docker permission errors with non-root user
Removed
- Node.js and npm from Docker image (using standalone ESBuild/Tailwind)
- Complex BuildKit features that were slowing builds
- Multi-platform Docker builds (simplified to amd64 only)
[0.1.0] - 2023-12-17
Added
- Initial release
- APRS-IS network connection with reconnection logic
- Real-time packet processing with GenStage pipeline
- Phoenix LiveView map interface
- PostGIS geographic data storage
- Background job processing with Oban
- ETS-based local caching
- Basic rate limiting