Add posresolution and rain_midnight to the list of float fields in
normalize_numeric_types to ensure integer values are converted to floats
before database insertion. This fixes the Ecto.ChangeError that was
preventing many APRS packets from being stored in the database.
- Create migration to change string fields from VARCHAR(255) to TEXT type
- Add PacketSanitizer module for UTF-8 safe string truncation
- Update PacketConsumer to sanitize data before database insertion
- Prevents database errors from overly long packet data fields
The APRS parser is sending timestamp as integer (Unix timestamp) but our
schema expects a string. This was causing Ecto.ChangeError in production
preventing packet insertion.
Convert integer timestamps to strings in the field conversion pipeline.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Instead of playing whack-a-mole with individual fields, this implements a
proper whitelist approach that only allows fields defined in the Packet schema.
- Created PacketFieldWhitelist module with all 78 valid packet fields
- Filters out ANY field not in the schema, preventing future errors
- Handles both atom and string keys
- Includes debug helpers to identify filtered fields
This definitively fixes packet insertion failures by ensuring only valid
database fields are passed to Repo.insert_all, regardless of what the
APRS parser sends.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
The parser is sending fields with different names than our schema:
- aprs_messaging? (with question mark) -> convert to aprs_messaging
- raw_data -> remove (not in schema)
This should fix the remaining packet insertion failures in production.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Added item_name, object_name fields to store names
- Added is_item, is_object boolean flags for filtering
- Added dao field to store DAO extension data for extra position precision
- Automatic detection during packet processing to populate these fields
- Added database indexes for efficient queries
- Now preserving itemname as item_name and dao data instead of discarding
This enables future features like:
- Showing/hiding items and objects separately from stations
- Querying for specific objects (events, repeaters, etc.)
- Using DAO data for high-precision applications
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Production logs showed additional fields causing insertion failures:
- dao: DAO extension data
- itemname: Item name field
- longitude/latitude: Parser uses these but our schema uses lon/lat
Added conversion step to map latitude->lat and longitude->lon before processing.
This ensures packets can be properly saved to the database.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
The updated APRS parser (commit 69d0ab5) adds several fields that don't exist in our database schema:
- type, digipeaters, daodatumbyte, mbits, message (duplicate of message_text)
- phg (we store individual PHG fields), wx (weather object)
- resultcode, resultmsg, gpsfixstatus
Created a centralized helper function to remove all non-schema fields before database insertion.
Also enhanced error logging to capture sample packet data for easier debugging.
This fixes the production issue where packets were failing to save to the database.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
The recent APRS parser update added a gpsfixstatus field that doesn't exist
in our database schema. This was causing all packet insertions to fail in
production with ArgumentError, preventing any packets from being stored and
making historical packet loading impossible.
Changes:
- Added explicit removal of :gpsfixstatus field in packet_consumer.ex
- Added both atom and string key deletion for safety
- Placed removal alongside other field removals like raw_weather_data
This fixes the critical production issue where no packets were being stored,
which prevented historical packet loading from working.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Optimized list concatenation in PacketConsumer from O(n) to O(1) by prepending events
- Refactored coordinate rounding to avoid recreating anonymous functions on each call
- Added database indexes for case-insensitive searches on upper(sender) and upper(base_callsign)
- Implemented RegexCache to avoid recompiling regex patterns for wildcard device matching
- Added compound index on upper(sender) with received_at for efficient sorted queries
These changes improve performance in hot paths:
- Packet batching is now more efficient with better list operations
- Coordinate processing avoids function allocation overhead
- Database queries using UPPER() now use functional indexes
- Device wildcard matching no longer recompiles regex on every lookup
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
The map wasn't receiving real-time packet updates because:
1. LiveView wasn't subscribed to StreamingPacketsPubSub
2. LiveView wasn't handling {:streaming_packet, packet} messages
3. Packets weren't being broadcast to both PubSub systems
Changes:
- Added StreamingPacketsPubSub subscription with viewport bounds filtering
- Added handler for {:streaming_packet, packet} messages
- Update subscription bounds when map viewport changes
- Properly unsubscribe on LiveView termination
- Also broadcast packets to SpatialPubSub (in addition to StreamingPacketsPubSub)
Now the map only receives packets within its current viewport bounds,
reducing unnecessary data transfer and improving performance.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit addresses two major areas:
1. Crash Prevention & Error Handling:
- Added error handling to all database queries to prevent crashes
- Implemented circuit breaker pattern for APRS-IS connection
- Added timeouts to all Redis operations (5 second timeout)
- Fixed unsafe pattern matches that could cause MatchError crashes
- Added try/rescue blocks around background tasks
- Improved socket operation error handling with proper cleanup
- Enhanced packet cleanup worker with comprehensive error handling
2. RF Path Station Markers:
- Fixed issue where digipeater/iGate icons weren't shown when searching for callsigns
- Now loads and displays position markers for all stations in the RF path
- Extracts RF path stations from tracked callsign packets
- Fetches latest position for each digipeater/iGate
- Properly displays complete RF path visualization with station positions
Technical details:
- Added Process.alive? checks before sending messages
- Fixed resource cleanup in terminate functions
- Added proper error handling to File.read operations
- Implemented safe wrapper for ShutdownHandler health checks
- All changes ensure the app continues running even when errors occur
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Fixed File.read crash in devices_seeder.ex with proper error handling
- Added timeout to TCP connect in aprs_is_connection.ex (10s timeout)
- Fixed unsafe socket send operations with error handling
- Added Process.alive? check before sending to PIDs in streaming_packets_pubsub.ex
- Fixed stream materialization memory issue in packet_consumer.ex
- Added error handling for datetime parsing errors
- Prevented crashes from dead processes and closed sockets
These fixes ensure the application is more resilient to:
- Missing files
- Network timeouts
- Dead processes
- Closed sockets
- Invalid data parsing
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implement leader election to ensure only one APRS-IS connection across
multiple Kubernetes replicas. This prevents duplicate packet processing
and respects APRS-IS usage policies.
Key changes:
- Add leader election using :global registry
- Create connection manager for dynamic APRS-IS management
- Implement packet distribution from leader to all nodes
- Add Kubernetes headless service for node discovery
- Configure DNS-based clustering with libcluster
- Update deployment to support 3 replicas with clustering
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add nil check before canceling timer in PacketConsumer to prevent errors
when timer reference doesn't exist (fixes test failures)
- Simplify packet counter reset migration by removing unsupported event
triggers, keeping only manual reset function
- All tests now passing
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Fixed "ParseError does not implement the Access behaviour" error that
occurred during packet batch insert when packets contained ParseError
structs in their data_extended field.
Added special handling for Aprs.Types.ParseError structs in the
struct_to_map function to convert them to simple maps with their
error_code and message fields, preventing the Access behavior error
when the struct is accessed during processing.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
The memory monitoring was incorrectly using total VM memory instead of
per-process memory, causing false positives whenever any process in the
system allocated memory.
Changes:
- Changed memory monitoring to per-process instead of total VM memory
- Reduced thresholds to reasonable per-process values (50MB diff, 100MB total)
- Increased minor GC trigger from 100 to 1000 packets to prevent GC after
every normal batch
This prevents detecting memory changes from other processes and eliminates
the constant GC warnings while still maintaining proper memory management.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Reduced batch processing parameters to prevent excessive GC:
- Reduced batch_size from 500 to 100 packets
- Adjusted batch_timeout from 2000ms to 1000ms for faster processing
- Reduced max_demand from 750 to 300 to match smaller batches
- Updated hardcoded fallback from 500 to 100 in packet_consumer.ex
- Changed minor GC trigger from 500 to 100 packets to match batch size
These smaller batch sizes were proven to work well in production and
should significantly reduce the frequency of garbage collection while
maintaining good performance.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Increased memory thresholds from 10MB/20MB to 500MB/1GB since the server
has 15GB available memory. This prevents constant GC warnings while
maintaining memory safety.
Configured VM garbage collection settings in vm.args.eex:
- Enabled concurrent ports/sockets limit (+Q 65536)
- Set heap sizes and GC parameters optimized for better performance
- Added scheduler and binary heap settings
These changes will reduce unnecessary GC overhead and improve overall
application performance.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Create PacketConsumerPool to run 3 parallel consumers (configurable)
- Increase max_demand from 250 to 750 total (250 per consumer)
- Fix timer management to properly cancel/restart on batch processing
- Configure proper GenStage subscriptions with backpressure control
- Allow unnamed consumers for pool usage
This 3x increase in processing capacity prevents the "Packet buffer full"
warnings by ensuring consumers can keep up with incoming packet rate.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit introduces several major performance improvements and adds
robust error handling for malformed HTTP requests:
Performance Optimizations:
- Add database migration with BRIN indexes for time-series data and partial
indexes for recent queries, significantly improving query performance
- Create global StreamingPacketsPubSub system for real-time packet distribution
with geographic bounds filtering using ETS for fast lookups
- Refactor PacketConsumer to use Stream module for memory-efficient processing,
preventing memory accumulation during batch operations
- Implement dedicated BroadcastTaskSupervisor pool for async broadcast operations,
preventing GenServer blocking on I/O
- Increase database connection pool from 25 to 45 (production) and 15 to 30 (dev)
for better concurrency support
Error Handling:
- Add SentryFilter to prevent Bandit.HTTPError from missing Host headers from
cluttering Sentry (common with bots/scanners)
- Configure custom 400 Bad Request error handling
- Filter out common bot/scanner paths from error reporting
All changes include comprehensive test coverage following TDD practices.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Restored original PacketConsumer implementation from before optimizations
- Reverted to fixed batch size of 50 packets (original value)
- Removed SystemMonitor and InsertOptimizer from application startup
- Restored original config with buffer size 1000 and batch timeout 1000ms
- Removed all dynamic batch sizing and optimization logic
The recent performance optimizations were causing packet buffer overflows
because the dynamic batch sizing was actually slowing down processing.
Reverting to the simpler, working implementation that processes packets
consistently without buffer overflows.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Increased packet buffer size from 1,000 to 10,000 to handle traffic spikes
- Removed InsertOptimizer dynamic batch sizing that was slowing processing
- Reverted to fixed batch size of 200 for consistent performance
- Simplified insert options to use static configuration
- Added telemetry metrics for buffer overflow and utilization monitoring
- Fixed unused variable warning in packet_consumer.ex
The root cause was the recent performance optimizations that introduced
dynamic batch sizing, which actually degraded performance and caused
the producer buffer to overflow when the consumer couldn't keep up.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Fixed InsertOptimizer returning map instead of keyword list for Ecto
- Increased batch size ranges to handle high load (100-800 packets)
- Improved consumer responsiveness:
- Reduced batch timeout from 1000ms to 500ms
- Process batches at 80% capacity for better throughput
- Reduced adjustment interval from 10s to 5s
- Added comprehensive logging for debugging buffer status
- Created integration tests for packet pipeline
- Coordinated batch sizing between SystemMonitor and InsertOptimizer
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
