Address browser testing reliability in CI environments:
## CI/CD Improvements
- **ChromeDriver**: Use latest stable version instead of pinned version for better compatibility
- **Timeouts**: Add 10-minute timeout for integration tests to prevent CI hangs
- **Environment Variables**: Configure WALLABY_MAX_WAIT_TIME for CI-specific timeouts
## Wallaby Configuration Enhancements
- **Chrome Args**: Add CI-stable Chrome options (--no-sandbox, --disable-dev-shm-usage)
- **Window Size**: Set consistent 1280x720 window size for reproducible tests
- **Dynamic Timeouts**: Environment-configurable max wait time with sensible defaults
- **Pool Size**: Limit to single browser instance to prevent resource conflicts
## Benefits
- ✅ More reliable test execution in CI environments
- ✅ Better error handling and timeout management
- ✅ Consistent browser behavior across environments
- ✅ Reduced flakiness from resource constraints
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit implements browser-based integration testing for the Phoenix LiveView
application using Wallaby and ChromeDriver to test the complete user experience
including JavaScript interactions.
## Changes Made
### Testing Infrastructure
- Add Wallaby dependency for browser automation testing
- Configure Chrome driver with headless mode for CI/CD
- Set up Phoenix server integration for test environment
- Enable screenshot capture on test failures
### Integration Tests
- `test/integration/simple_integration_test.exs` - Basic homepage functionality
- `test/integration/map_integration_test.exs` - Map interface and routing tests
- Tests validate page loading, content rendering, and navigation
### CI/CD Integration
- Update GitHub Actions workflow to install ChromeDriver
- Add separate integration test step in CI pipeline
- Configure headless Chrome for automated testing
### Configuration Updates
- Enable Phoenix server in test environment for Wallaby
- Configure Wallaby with proper base URL and screenshot settings
- Initialize Wallaby application in test helper
## Features Tested
- ✅ Homepage loads with APRS content
- ✅ Main page structure renders correctly
- ✅ Route navigation functions properly
- ✅ Basic Phoenix LiveView functionality
## Next Steps
This provides a solid foundation for expanding to test:
- Interactive map controls and JavaScript functionality
- Real-time packet updates via LiveView PubSub
- User authentication flows and form submissions
- Mobile responsive behavior
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Fix zero coordinate handling to distinguish between invalid coordinates and valid 0.0 (equator/prime meridian)
- Add to_float_safe() function that preserves nil for proper coordinate validation
- Add comprehensive coordinate validation with descriptive logging in JavaScript
- Make position change threshold configurable via application config (default: 0.001°)
- Add extensive test coverage for edge cases including nil coordinates and equator crossing
- Improve robustness for real-world APRS coordinate edge cases
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add DATABASE_SSL and DATABASE_SSL_VERIFY environment variables
- Configure Postgrex to use SSL with optional certificate verification
- Allow connecting to PostgreSQL with self-signed certificates
- Simplify APRS_PORT parsing to avoid syntax errors
- Disable Exq properly in test environment to prevent startup issues
- Disable Prometheus telemetry in test mode to avoid port conflicts
- Update telemetry module to conditionally start Prometheus metrics
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add robust integer parsing for APRS_PORT environment variable
- Fallback to default port 14580 if parsing fails
- Prevents crashes when environment variables contain unexpected values
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Make APRS-IS port configurable via APRS_PORT environment variable
- Defaults to 14580 (filtered port) if not specified
- Allows switching to 10152 (unfiltered port) for full packet stream
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Fix race condition in map bundle loading by waiting for script to load before initializing map
- Ensure Leaflet is properly exposed as window.L in map bundle
- Add Leaflet availability check in map initialization function
- Disable Exq background jobs in development environment to remove Redis dependency
- Remove invalid TelemetryMetricsPrometheus plug from endpoint
- Application now starts successfully without Redis in development mode
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Only include password in Exq config if it's not empty
- Prevents "ERR AUTH <password>" error when Redis has no authentication
- Build config dynamically to exclude empty password field
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Wrap String.to_integer in try/rescue block properly
- Fix CompileError caused by rescue clause inside case statement
- Ensures runtime.exs can compile and load Redis configuration
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Move Redis connection settings to runtime.exs only
- Remove localhost:6379 hardcoded values that were overriding runtime config
- Keep only non-environment-dependent Exq settings in compile-time config
- Fixes Exq trying to connect to localhost instead of Kubernetes Redis service
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Parse REDIS_URL environment variable in runtime.exs
- Extract host, port, password, and database from Redis URL
- Use same Redis configuration parsing as RedisCache module
- Fixes CrashLoopBackOff caused by Exq trying to connect to localhost
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Replace Oban dependency with Exq for Redis-based background jobs
- Update PacketCleanupWorker to use Exq's perform/1 signature
- Add CleanupScheduler GenServer for periodic job scheduling
- Configure Exq with Redis connection and queue settings
- Update tests to work with Exq job format
- Maintain 6-hour packet cleanup schedule functionality
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Remove PostgreSQL session-specific parameters that are not supported
by PgBouncer connection pooling. Keep only application_name parameter
which is passed through properly.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Remove static Redis PubSub configuration from runtime.exs
- Add dynamic pubsub_config/0 function in application.ex
- Configure PubSub adapter based on cluster_enabled and REDIS_URL at startup
- Ensures proper initialization order for distributed deployments
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Configures Phoenix.PubSub.Redis adapter when clustering is enabled
and REDIS_URL environment variable is set. This allows PubSub
messages to be shared across Elixir nodes in a distributed
deployment, enabling proper real-time updates in multi-node setups.
- Add phoenix_pubsub_redis dependency (~> 3.0)
- Configure Redis adapter in runtime.exs for production
- Set pool size to 10 for better performance
- Include node name in Redis configuration
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Download external libraries locally to reduce CDN dependencies
- Create combined vendor bundles for JavaScript and CSS
- Replace CDN script tags with local bundles in layout template
- Add build scripts for vendor bundle creation and optimization
- Configure esbuild to process vendor bundles separately
- Total bundle size: ~512KB JS + 17KB CSS (optimized)
Benefits:
- Reduced external requests and CDN dependencies
- Better caching control and reliability
- Eliminated CORS and SRI integrity issues
- Faster page loads with bundled resources
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Switch to Kubernetes.DNSSRV strategy which is specifically designed
for StatefulSets and uses DNS SRV records for node discovery.
This strategy:
- Doesn't require Kubernetes API permissions (no RBAC needed)
- Automatically filters out the current node
- Works with StatefulSet stable DNS names
- Uses SRV records from the headless service
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Based on libcluster documentation, StatefulSets should use the standard
Kubernetes strategy with mode: :hostname, not the DNS strategy.
This properly discovers pods using their stable hostnames and filters
out the current node from connection attempts.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
The DNSPoll strategy was attempting to connect to IPs instead of DNS names.
Kubernetes.DNS strategy is specifically designed for StatefulSet deployments
and will properly discover pods using their stable DNS names.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implement leader election to ensure only one APRS-IS connection across
multiple Kubernetes replicas. This prevents duplicate packet processing
and respects APRS-IS usage policies.
Key changes:
- Add leader election using :global registry
- Create connection manager for dynamic APRS-IS management
- Implement packet distribution from leader to all nodes
- Add Kubernetes headless service for node discovery
- Configure DNS-based clustering with libcluster
- Update deployment to support 3 replicas with clustering
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Allow HTTP connections from internal IP 10.0.19.221 for k3s cluster access
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Replace invalid wildcard domains (*.aprs.me) with explicit origins
- Phoenix doesn't support wildcard syntax in check_origin
- Added specific domains for aprs.me, sentry.io, and openstreetmap.org
- Updated CLAUDE.md with kubectl commands for debugging k3s deployment
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Fixed the esbuild errors during Docker build where npm packages couldn't
be resolved. The issue was caused by node_modules being excluded from
the Docker build context in .dockerignore.
Implemented a proper vendor bundle strategy following Phoenix 1.7+ patterns:
- Created vendor.js that bundles all npm dependencies (Leaflet, Chart.js,
topbar) and their CSS files into a single pre-built bundle
- Added vendor esbuild profile in config.exs with proper CSS/image loaders
- Updated mix.exs to build vendor bundle before app bundle in assets.deploy
- Modified JavaScript imports to use globally loaded libraries from vendor
bundle instead of importing from node_modules
- Added vendor.js script tag to root.html.heex before app.js
This approach avoids Docker build failures by pre-bundling all vendor
dependencies, eliminating the need for node_modules during production builds.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Moved CSS imports from Tailwind config to JavaScript to let ESBuild handle them.
This fixes the Docker build error where Tailwind couldn't resolve node_modules paths.
- Removed CSS @import statements from app.css
- Added CSS imports to map.ts where Leaflet is used
- Configured ESBuild to handle CSS and image files
- Added missing topbar package to package.json
- Updated both dev and prod ESBuild configs
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
The Docker build was failing because the Chart.js date adapter was trying
to require("chart.js") but esbuild couldn't resolve it since Chart.js is
loaded as a UMD file, not a CommonJS module.
Added --external:chart.js to the vendor esbuild configuration to tell
esbuild not to try bundling chart.js when it encounters require("chart.js")
in the date adapter. This is correct because Chart.js is already loaded
globally from the UMD file.
This fixes the "Could not resolve 'chart.js'" error during production builds.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
This change eliminates all external CDN dependencies by bundling JavaScript
libraries locally using esbuild. All vendor files are now served from the
application's own assets, improving reliability and privacy.
Changes:
- Created assets/js/vendor.js to bundle all external libraries
- Added vendor bundle configuration to esbuild in config files
- Updated all templates to use local vendor.js instead of CDN scripts
- Created download-vendor-files.sh script to fetch vendor libraries
- Added vendor files to .gitignore (they're downloaded during setup)
- Updated CSS imports to include vendor stylesheets
- Added vendor build step to assets.deploy mix task
Bundled libraries:
- Leaflet 1.9.4 (mapping library)
- Leaflet.heat 0.2.0 (heatmap plugin)
- Leaflet.markercluster 1.5.3 (marker clustering)
- OverlappingMarkerSpiderfier 0.2.6 (overlapping marker handling)
- Chart.js 4.5.0 (charting library)
- chartjs-adapter-date-fns 3.0.0 (date adapter for Chart.js)
All functionality tested and working correctly with local bundles.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Reduced batch processing parameters to prevent excessive GC:
- Reduced batch_size from 500 to 100 packets
- Adjusted batch_timeout from 2000ms to 1000ms for faster processing
- Reduced max_demand from 750 to 300 to match smaller batches
- Updated hardcoded fallback from 500 to 100 in packet_consumer.ex
- Changed minor GC trigger from 500 to 100 packets to match batch size
These smaller batch sizes were proven to work well in production and
should significantly reduce the frequency of garbage collection while
maintaining good performance.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Create PacketConsumerPool to run 3 parallel consumers (configurable)
- Increase max_demand from 250 to 750 total (250 per consumer)
- Fix timer management to properly cancel/restart on batch processing
- Configure proper GenStage subscriptions with backpressure control
- Allow unnamed consumers for pool usage
This 3x increase in processing capacity prevents the "Packet buffer full"
warnings by ensuring consumers can keep up with incoming packet rate.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit introduces several major performance improvements and adds
robust error handling for malformed HTTP requests:
Performance Optimizations:
- Add database migration with BRIN indexes for time-series data and partial
indexes for recent queries, significantly improving query performance
- Create global StreamingPacketsPubSub system for real-time packet distribution
with geographic bounds filtering using ETS for fast lookups
- Refactor PacketConsumer to use Stream module for memory-efficient processing,
preventing memory accumulation during batch operations
- Implement dedicated BroadcastTaskSupervisor pool for async broadcast operations,
preventing GenServer blocking on I/O
- Increase database connection pool from 25 to 45 (production) and 15 to 30 (dev)
for better concurrency support
Error Handling:
- Add SentryFilter to prevent Bandit.HTTPError from missing Host headers from
cluttering Sentry (common with bots/scanners)
- Configure custom 400 Bad Request error handling
- Filter out common bot/scanner paths from error reporting
All changes include comprehensive test coverage following TDD practices.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Removed all console.log debugging statements from app.js (theme changes, chart rendering)
- Removed all console.log statements from map.ts (map initialization, data processing)
- Removed all console.debug statements from map_helpers.ts
- Removed all console.debug statements from map_fixes.ts
- Kept essential console.error and console.warn statements for error handling
- Reduced bundle size by ~1.4kb
This cleanup removes unnecessary debug output while maintaining error reporting
for production debugging via console.error and Sentry.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Fix "this.__view is undefined" error when calling pushEvent in event handlers
- Bind pushEvent context properly in all marker click and hover handlers
- Replace undefined sendBoundsToServer calls with saveMapState
- Ensure all event handlers check for pushEvent availability and type
- Use .bind(self) to preserve LiveView hook context in callbacks
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Restored original PacketConsumer implementation from before optimizations
- Reverted to fixed batch size of 50 packets (original value)
- Removed SystemMonitor and InsertOptimizer from application startup
- Restored original config with buffer size 1000 and batch timeout 1000ms
- Removed all dynamic batch sizing and optimization logic
The recent performance optimizations were causing packet buffer overflows
because the dynamic batch sizing was actually slowing down processing.
Reverting to the simpler, working implementation that processes packets
consistently without buffer overflows.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
