Commit graph

23 commits

Author SHA1 Message Date
aa5cfb8240
Fix startup issues with Redis and shutdown handlers
- Delay DeviceCache initial load by 1 second to allow Redis connections
- Move shutdown handlers to end of supervision tree
- This prevents shutdown handlers from interfering with startup failures

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-26 16:00:54 -05:00
eeeebef34e
Fix PgBouncer database connection and graceful shutdown
- Add SKIP_DB_CREATE env var to skip database creation when using PgBouncer
- Fix redis_children() to return proper child specs
- Implement zero-downtime graceful shutdown without user notifications
- Add health check plug for Kubernetes probes
- Remove user-facing shutdown notifications for seamless deployments

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-26 15:51:06 -05:00
226a53bf61
Add Redis-based distributed caching and rate limiting
This commit introduces a dual-mode caching and rate limiting system that can use
either Redis (for distributed deployments) or ETS/Cachex (for single-node setups).

Key changes:
- Add Cache abstraction layer that automatically switches between Redis and Cachex
- Implement RedisCache module with full distributed cache functionality
- Implement RedisRateLimiter with sliding window algorithm for accurate rate limiting
- Add RateLimiterWrapper to provide unified API for both implementations
- Update application startup to conditionally use Redis when REDIS_URL is set
- Migrate all cache operations to use the new Cache abstraction
- Support graceful fallback to ETS-based solutions when Redis is unavailable

The system automatically detects Redis availability via REDIS_URL environment
variable and switches between implementations without code changes. This enables
proper distributed caching and rate limiting in Kubernetes deployments while
maintaining backward compatibility for development environments.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-26 15:29:38 -05:00
7fa41b85b4
Fix Redis PubSub URL configuration
The phoenix_pubsub_redis adapter expects the url option at the top level,
not nested under redis_options.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-26 10:22:15 -05:00
cbbe1772bc
Add logging to PubSub configuration for debugging
🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-26 10:14:44 -05:00
c85b907ab7
Move Redis PubSub config to runtime initialization
- Remove static Redis PubSub configuration from runtime.exs
- Add dynamic pubsub_config/0 function in application.ex
- Configure PubSub adapter based on cluster_enabled and REDIS_URL at startup
- Ensures proper initialization order for distributed deployments

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-26 10:10:55 -05:00
437ccebbaa
Fix duplicate APRS-IS connections in clustered deployments
Prevent both AprsIsConnection and cluster-managed Is connection from
starting simultaneously. When clustering is enabled, only the cluster's
ConnectionManager should handle APRS-IS connections via leader election.

This resolves aprsc server disconnections caused by duplicate validated
clients with the same username (W5ISP-1).

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-25 12:10:47 -05:00
14ffefe2e9
Add Cluster.Supervisor directly to application supervision tree 2025-07-25 11:03:20 -05:00
4804bd16be
Add distributed Erlang clustering for single APRS-IS connection
Implement leader election to ensure only one APRS-IS connection across
multiple Kubernetes replicas. This prevents duplicate packet processing
and respects APRS-IS usage policies.

Key changes:
- Add leader election using :global registry
- Create connection manager for dynamic APRS-IS management
- Implement packet distribution from leader to all nodes
- Add Kubernetes headless service for node discovery
- Configure DNS-based clustering with libcluster
- Update deployment to support 3 replicas with clustering

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-25 10:30:29 -05:00
e41118a243
Remove PacketPipelineSetup to fix application startup crash
The PacketPipelineSetup was attempting to subscribe to a single
PacketConsumer process that doesn't exist. The application uses
PacketConsumerPool which automatically handles subscriptions for
multiple consumer processes, making PacketPipelineSetup redundant
and causing startup failures.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-19 15:15:06 -05:00
234fc2010f
Implement critical performance optimizations and error handling
This commit introduces several major performance improvements and adds
robust error handling for malformed HTTP requests:

Performance Optimizations:
- Add database migration with BRIN indexes for time-series data and partial
  indexes for recent queries, significantly improving query performance
- Create global StreamingPacketsPubSub system for real-time packet distribution
  with geographic bounds filtering using ETS for fast lookups
- Refactor PacketConsumer to use Stream module for memory-efficient processing,
  preventing memory accumulation during batch operations
- Implement dedicated BroadcastTaskSupervisor pool for async broadcast operations,
  preventing GenServer blocking on I/O
- Increase database connection pool from 25 to 45 (production) and 15 to 30 (dev)
  for better concurrency support

Error Handling:
- Add SentryFilter to prevent Bandit.HTTPError from missing Host headers from
  cluttering Sentry (common with bots/scanners)
- Configure custom 400 Bad Request error handling
- Filter out common bot/scanner paths from error reporting

All changes include comprehensive test coverage following TDD practices.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-18 18:19:49 -05:00
46340e008a
update deps 2025-07-16 08:20:20 -05:00
0298c8ec91
Revert all packet processing optimizations to fix buffer overflow
- Restored original PacketConsumer implementation from before optimizations
- Reverted to fixed batch size of 50 packets (original value)
- Removed SystemMonitor and InsertOptimizer from application startup
- Restored original config with buffer size 1000 and batch timeout 1000ms
- Removed all dynamic batch sizing and optimization logic

The recent performance optimizations were causing packet buffer overflows
because the dynamic batch sizing was actually slowing down processing.
Reverting to the simpler, working implementation that processes packets
consistently without buffer overflows.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-15 08:44:26 -05:00
8e507702f7
more performance improvements/cleanup 2025-07-14 16:49:55 -05:00
3730b8c848
performance improvements 2025-07-14 16:35:58 -05:00
b1ee1f92f9
dont parse device identifiers on each insert 2025-07-10 15:16:01 -05:00
711d92ebb6
performance improvements 2025-07-08 10:03:52 -05:00
2543617f16
add sentry 2025-07-07 09:52:02 -05:00
fc449ed5f9
en español 2025-07-05 15:51:39 -05:00
172da7d393
upgrade more with modern phoenix 2025-07-05 08:04:06 -05:00
f08330b46c
weather fixing 2025-06-25 11:40:04 -05:00
0beb278371
add packet batcher 2025-06-24 15:06:09 -05:00
ce7c50caa1
refactor to use external parser 2025-06-24 14:22:09 -05:00
Renamed from lib/aprs/application.ex (Browse further)