Commit graph

5 commits

Author SHA1 Message Date
4c5c730ee7
Speed up the slowest tests
- packet_replay: stop_replay test reduces fake-task receive timeout from 5s to 200ms
- broadcast_task_supervisor: scheduler_usage takes configurable sample seconds;
  test uses sample-pair API (instant) instead of blocking 1-second sample
- health_check: ask_if_alive timeout is now configurable; unresponsive-pid test
  uses 50ms timeout instead of waiting full 5-second default
- mix_unused/analyzer_test: cache analyze() result in setup_all so 4 tests share
  one analyze pass instead of running 4 separate scans
- mix/tasks/compile/unused_test: consolidate three slow severity tests into one
- log_sanitizer + packet_field_whitelist: cap property-test runs at 25 (was 100)
- historical_loading: trim 200ms post-event sleeps to 50ms
- movement: refute_push_event timeout from 200ms to 50ms

Total suite time: ~45s -> 40.6s; 2488 tests, 0 failures.
2026-05-09 10:56:03 -05:00
7897fe8a1a
perf: optimize slow tests - 40% reduction in top 10 slowest tests
- Skip expensive scheduler utilization sampling in test env (saves 1s per call)
- Reduce Process.sleep times from 100-300ms to 50-100ms
- Reduce refute_receive timeouts from 500ms to 100ms
- GPS drift test: 1371ms → 431ms (68.5% faster)
- BroadcastTaskSupervisor tests removed from slowest 10 (1000ms+ saved each)
- PacketDistributor tests: 33% faster
- StreamingPacketsPubSub tests removed from slowest 10 (400ms+ saved each)

Top 10 slowest tests reduced from 6.8s to 4.1s (40% improvement)
2026-03-03 10:47:40 -06:00
d9c04f7e0c
Fix code review findings: security, correctness, performance, dead code
Security:
- Add SQL identifier validation to DbOptimizer.analyze_table/vacuum_table
- Move health/readiness routes to non-rate-limited pipeline

Correctness:
- Fix monitor reference leak in StreamingPacketsPubSub (track refs, demonitor on unsubscribe)
- Persist circuit breaker half-open state transitions
- Simplify ShutdownHandler.terminate to avoid pointless Process.send_after

Performance:
- Batch marker removal into single WebSocket event (remove_markers_batch)
- Use Task.Supervisor.start_child for fire-and-forget broadcasts

Dead code removal:
- Remove copy_insert/regular_batch_insert/rows_to_csv from DbOptimizer
- Remove dead packet_pipeline_integration tests
- Remove unused telemetry event detach

Other:
- Fix Callsign.valid? docstring to match permissive behavior
- Fix DatabaseMetrics to delegate instead of hardcoding pool stats
- Add circuit breaker and encoding test coverage
2026-02-19 18:49:27 -06:00
185266a1f9
Refactor: reduce nesting depth in 2 functions
Extract nested logic into helper functions to improve readability:
- broadcast_task_supervisor.ex: extract broadcast_to_topics/3 helper
- log_sanitizer.ex: extract redact_match/1 helper

Progress: 2/42 refactoring opportunities fixed
2026-02-09 11:48:05 -06:00
234fc2010f
Implement critical performance optimizations and error handling
This commit introduces several major performance improvements and adds
robust error handling for malformed HTTP requests:

Performance Optimizations:
- Add database migration with BRIN indexes for time-series data and partial
  indexes for recent queries, significantly improving query performance
- Create global StreamingPacketsPubSub system for real-time packet distribution
  with geographic bounds filtering using ETS for fast lookups
- Refactor PacketConsumer to use Stream module for memory-efficient processing,
  preventing memory accumulation during batch operations
- Implement dedicated BroadcastTaskSupervisor pool for async broadcast operations,
  preventing GenServer blocking on I/O
- Increase database connection pool from 25 to 45 (production) and 15 to 30 (dev)
  for better concurrency support

Error Handling:
- Add SentryFilter to prevent Bandit.HTTPError from missing Host headers from
  cluttering Sentry (common with bots/scanners)
- Configure custom 400 Bad Request error handling
- Filter out common bot/scanner paths from error reporting

All changes include comprehensive test coverage following TDD practices.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-18 18:19:49 -05:00