Simplify complex functions to improve readability and maintainability:
- clustering.ex: Convert case statement to map lookup for cluster radii
- encoding.ex: Extract codepoint validation into separate function clauses
- leader_election.ex: Extract PID liveness checking and cleanup logic into helpers
Co-Authored-By: Graham <noreply@anthropic.com>
Extract nested logic into helper functions to improve readability:
- regex_cache.ex: Extract compile_and_cache/1 for regex compilation and caching
- connection_monitor.ex: Extract calculate_scheduler_utilization/1 for CPU stat calculation
- device_identification.ex: Extract fetch_devices_from_url/0 for HTTP request handling
Co-Authored-By: Graham <noreply@anthropic.com>
- Fix length/1 check in mobile_channel.ex
- Add LeaderElection alias in connection_manager.ex
All warnings and design suggestions now complete!
Remaining: 40 refactoring opportunities (complex/nested functions)
Check if Repo process is started before attempting to collect PostgreSQL
metrics. This prevents the 'could not lookup Ecto repo' error message
during application startup when the periodic metrics collector runs
before the Repo is fully initialized.
The metrics collection now silently skips when the Repo isn't ready
and will succeed on subsequent runs once the Repo is started.
- Add Phoenix.LiveViewTest import to ConnCase to fix warnings
- Disable AppSignal in dev and test (only enable in prod)
- Explicitly disable clustering in test environment
- Fix PacketDistributor to check cluster_enabled before calling LeaderElection
- Add Phoenix.CodeReloader listener to mix.exs
This prevents tests from trying to call cluster-specific GenServers
(ConnectionMonitor, LeaderElection) that don't exist in test mode.
The host from Application.get_env returns a binary string, but :gen_tcp.connect
expects a charlist. This was causing ArgumentError on connection attempts.
Convert host to charlist if it's a binary before passing to :gen_tcp.connect.
- Fix unsubscribe_from_bounds -> unsubscribe (function doesn't exist)
- Fix search_callsign query to use 'lon' instead of 'lng' field
- Add comprehensive test suite for mobile channel:
- Join and subscription tests
- Bounds validation tests
- Callsign search tests
- Callsign tracking tests
- Streaming packet tests
- Packet data format tests
- 29 tests total, all passing
Created test/support/channel_case.ex for channel testing infrastructure.
Generated with Claude Code https://claude.com/claude-code
- Always use ETS-based RateLimiter instead of checking for REDIS_URL
- Remove RedisRateLimiter calls that were causing crashes
- Simplify wrapper to only delegate to Aprsme.RateLimiter
- Replace Cachex with native ETS (Erlang Term Storage) in Cache module
- Remove Redis-based caching logic from application.ex
- Always initialize ETS tables for device_cache, query_cache, symbol_cache
- Update StatusLive to use Aprsme.Cache instead of Cachex directly
- Fix crash on startup when REDIS_URL was set but Redis not used
This fixes the no_cache error that was causing pods to crash.
Previously, when clustering was enabled, the leader election process
would wait indefinitely for the cluster to form. If cluster formation
failed or took too long, no leader would be elected and the APRS-IS
connection would never start, leaving the system disconnected.
Changes:
- Add 30-second timeout for cluster formation
- Force leader election after timeout, even in single-node mode
- Add election_forced flag to prevent duplicate election attempts
- Improve logging to distinguish between cluster and single-node modes
This ensures that at least one node will always connect to APRS-IS,
even if clustering fails completely.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Remove Exq dependency from mix.exs and application config
- Remove all Exq configuration from config files (config.exs, dev.exs, runtime.exs, test.exs)
- Update CleanupScheduler to run cleanup tasks directly using Task.start instead of queuing jobs
- Update PacketCleanupWorker documentation to remove Exq references
- Cleanup functionality preserved but now runs in supervised Tasks instead of job queue
This resolves the Redis connection error on startup by eliminating the dependency on external job queues.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Remove phoenix_pubsub_redis dependency from mix.exs
- Update PubSub configuration to use Phoenix's native distributed capabilities
- Simplify clustering logic - no longer requires Redis URL for PubSub
- Phoenix PubSub automatically distributes messages across connected Erlang nodes
- Benefits: lower latency, automatic failure handling, simpler deployment
- Redis still available for caching and job processing when needed
- Maintains backward compatibility for both clustered and non-clustered modes
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Fix duplicate bad packet storage in store_packet function
- Changed insert_packet to return specific error tuples
- Handle validation_error and storage_exception separately
- Prevent double storage of bad packets
- Add try/catch wrapper around insert_packet for exception handling
- Properly catch and return storage_exception errors
- Test now correctly expects :storage_exception
- Add helper function for LiveView tests with duplicate IDs
- Added live_with_warn/2 to ConnCase
- Suppress duplicate ID warnings in integration tests
- Fix unused variable warning in insert_packet
All packet tests now pass without failures.
Add posresolution and rain_midnight to the list of float fields in
normalize_numeric_types to ensure integer values are converted to floats
before database insertion. This fixes the Ecto.ChangeError that was
preventing many APRS packets from being stored in the database.
- Create migration to change string fields from VARCHAR(255) to TEXT type
- Add PacketSanitizer module for UTF-8 safe string truncation
- Update PacketConsumer to sanitize data before database insertion
- Prevents database errors from overly long packet data fields
The APRS parser is sending timestamp as integer (Unix timestamp) but our
schema expects a string. This was causing Ecto.ChangeError in production
preventing packet insertion.
Convert integer timestamps to strings in the field conversion pipeline.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Instead of playing whack-a-mole with individual fields, this implements a
proper whitelist approach that only allows fields defined in the Packet schema.
- Created PacketFieldWhitelist module with all 78 valid packet fields
- Filters out ANY field not in the schema, preventing future errors
- Handles both atom and string keys
- Includes debug helpers to identify filtered fields
This definitively fixes packet insertion failures by ensuring only valid
database fields are passed to Repo.insert_all, regardless of what the
APRS parser sends.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
The parser is sending fields with different names than our schema:
- aprs_messaging? (with question mark) -> convert to aprs_messaging
- raw_data -> remove (not in schema)
This should fix the remaining packet insertion failures in production.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Added item_name, object_name fields to store names
- Added is_item, is_object boolean flags for filtering
- Added dao field to store DAO extension data for extra position precision
- Automatic detection during packet processing to populate these fields
- Added database indexes for efficient queries
- Now preserving itemname as item_name and dao data instead of discarding
This enables future features like:
- Showing/hiding items and objects separately from stations
- Querying for specific objects (events, repeaters, etc.)
- Using DAO data for high-precision applications
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Production logs showed additional fields causing insertion failures:
- dao: DAO extension data
- itemname: Item name field
- longitude/latitude: Parser uses these but our schema uses lon/lat
Added conversion step to map latitude->lat and longitude->lon before processing.
This ensures packets can be properly saved to the database.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
The updated APRS parser (commit 69d0ab5) adds several fields that don't exist in our database schema:
- type, digipeaters, daodatumbyte, mbits, message (duplicate of message_text)
- phg (we store individual PHG fields), wx (weather object)
- resultcode, resultmsg, gpsfixstatus
Created a centralized helper function to remove all non-schema fields before database insertion.
Also enhanced error logging to capture sample packet data for easier debugging.
This fixes the production issue where packets were failing to save to the database.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
The recent APRS parser update added a gpsfixstatus field that doesn't exist
in our database schema. This was causing all packet insertions to fail in
production with ArgumentError, preventing any packets from being stored and
making historical packet loading impossible.
Changes:
- Added explicit removal of :gpsfixstatus field in packet_consumer.ex
- Added both atom and string key deletion for safety
- Placed removal alongside other field removals like raw_weather_data
This fixes the critical production issue where no packets were being stored,
which prevented historical packet loading from working.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Optimized list concatenation in PacketConsumer from O(n) to O(1) by prepending events
- Refactored coordinate rounding to avoid recreating anonymous functions on each call
- Added database indexes for case-insensitive searches on upper(sender) and upper(base_callsign)
- Implemented RegexCache to avoid recompiling regex patterns for wildcard device matching
- Added compound index on upper(sender) with received_at for efficient sorted queries
These changes improve performance in hot paths:
- Packet batching is now more efficient with better list operations
- Coordinate processing avoids function allocation overhead
- Database queries using UPPER() now use functional indexes
- Device wildcard matching no longer recompiles regex on every lookup
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Added 20 new database fields for enhanced parser compatibility:
- Standard parser fields: srccallsign, dstcallsign, body, origpacket, header, alive, posambiguity, symboltable, symbolcode, messaging
- Radio range field: radiorange
- Additional weather fields: rain_midnight, has_weather
- Enhanced weather data extraction with dedicated wx field support
- Updated PHG parsing to handle both string format ("1060") and legacy map structure
- Added extraction functions for standard parser compatibility fields
- Created comprehensive test suite for enhanced parser functionality
- Updated APRS parser to latest version with improved field extraction
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit includes several critical fixes and optimizations:
## Production Buffer Overflow Fix
- Fixed telemetry_vals data type conversion from strings to integers
- Enhanced put_telemetry_fields() with robust type conversion logic
- Handles mixed data types (string, integer, float) gracefully
- Prevents PacketConsumer GenServer crashes that caused buffer overflow
- Processes telemetry from both data_extended and top-level attrs
## Test Suite Performance Optimization
- Improved test execution speed by 48% (43.2s → 24.1s)
- Increased parallelization: max_cases from 16 to 48
- Optimized database connections and reduced timeouts
- Tagged slow tests and exclude them by default for fast development
- Fixed device seeding to only run when needed
- Enhanced log capture for packet consumer tests
## Code Quality Improvements
- Fixed handle_info clause grouping in LeaderElection
- Ensures mix compile --warnings-as-errors passes
- Added comprehensive telemetry conversion tests
- Improved test reliability and reduced flakiness
## Infrastructure Updates
- Added migration init container to StatefulSet for automatic schema updates
- Disabled AUTO_MIGRATE in main container (handled by init container)
- Created .test-commands with helpful test aliases
This resolves production stability issues and significantly improves development workflow.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Fixes compilation warning by moving the catch-all handle_info/2 clause
to be immediately after the specific handle_info/2 clauses.
In Elixir, all function clauses with the same name and arity must be
grouped together for pattern matching to work correctly.
This resolves the 'mix compile --warnings-as-errors' failure.
Fixes production crashes where PacketConsumer GenServers were failing due to
attempting to insert string arrays like ['12.80', '0.00', ...] into an
integer array database field. This was causing buffer overflow as crashed
consumers couldn't process packets.
Changes:
- Enhanced put_telemetry_fields() to convert string/float telemetry values to integers
- Added robust handling for mixed data types (string, integer, float)
- Added graceful fallback for invalid values (converts to 0)
- Process telemetry from both data_extended and top-level attrs
- Added comprehensive tests for all conversion scenarios
This resolves the GenServer crashes and prevents packet processing buffer issues.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add tests for store_packet/1 functionality including validation, MicE data, and error handling
- Add tests for store_bad_packet/2 with proper error type conversion
- Add tests for get_recent_packets/1 with time, bounds, and callsign filtering
- Add tests for get_nearby_stations/4 with distance ordering and exclusions
- Add tests for weather packet queries and cleanup functions
- Add tests for packet replay, streaming, and counting operations
- Fix MicE struct access to use Access behavior instead of dot notation
- Fix error_type conversion to string for BadPacket storage
- Ensure all tests are passing (437 tests, 0 failures)
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
The APRS parser returns telemetry sequence numbers as strings (e.g., "094"),
but the database schema expects integers. This was causing Ecto.ChangeError
in production when processing telemetry packets.
Changes:
- Updated put_telemetry_fields/2 to convert string seq values to integers
- Added comprehensive tests for telemetry_seq type conversion
- Handles nil, string, integer, and invalid seq values gracefully
Production deployment notes:
1. Check if migration 20250801215236 has run in production
2. If not, run pending migrations before deploying this fix
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
The APRS parser sometimes returns :unknown as an atom for the format field,
but the database expects a string. This was causing Ecto.ChangeError when
inserting packets.
Added normalize_format_field/1 function to convert atom values to strings
before database insertion. This ensures compatibility between the parser
output and database schema requirements.
Fixes error: value `:unknown` for `Aprsme.Packet.format` in `insert_all`
does not match type :string
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Added position resolution field (posresolution) showing accuracy in meters
- 18.52m for uncompressed positions
- 0.291m for compressed positions
- Added format field indicating "compressed" or "uncompressed" position type
- Added telemetry support with three new fields:
- telemetry_seq: sequence number
- telemetry_vals: array of telemetry values
- telemetry_bits: binary telemetry bits
- Updated packet schema and database migration
- Enhanced packet extraction logic to handle new fields
- All tests pass successfully
These improvements provide better position accuracy information and
enable telemetry packet processing from APRS stations.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Added position_ambiguity field (integer, 0-4) to packets table
- Created migration to add the new database column
- Updated packet schema and processing to extract position ambiguity
- Position ambiguity levels indicate precision of reported coordinates:
- 0: No ambiguity (exact position)
- 1: ~0.1 mile ambiguity
- 2: ~1 mile ambiguity
- 3: ~10 mile ambiguity
- 4: ~60 mile ambiguity
- Automatically captures ambiguity level when parsing APRS packets
- All tests pass with no failures
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Fix invalid CSS selector in user registration test by replacing
`fl-contains` with proper href attribute selector
- Disable telemetry database metrics collection in test environment
to prevent database ownership errors from telemetry poller
This eliminates the GenServer termination errors that were occurring
during test runs due to invalid CSS selectors and database access
issues in background telemetry processes.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
