Commit graph

23 commits

Author SHA1 Message Date
2fc4706d3d
fix: stop silent packet loss, timer leak, and seq-scans
- packet_consumer: when a batch exceeds max_batch_size, carry the
  excess over to the next cycle instead of dropping it. Previously
  any overage was split off into a drop list that was only logged,
  losing packets the producer had already acknowledged.

- map_live: cancel the hover_end_timer in terminate/2 so a user
  disconnecting while a marker is still highlighted doesn't leave
  an orphaned timer behind.

- packet_consumer: rescue/log exceptions inside the async broadcast
  task so PubSub serialization bugs or outages are observable
  instead of silently killing the Task.

- prepared_queries: swap ST_Y/ST_X BETWEEN predicates for the && /
  ST_MakeEnvelope pattern so bounds queries hit the GIST index on
  packets.location instead of sequentially scanning.
2026-04-21 09:29:06 -05:00
2300dcd69c
fix: stabilize moving station tracking 2026-03-26 18:48:43 -05:00
288b9fbbb2
fix: address security vulnerabilities and concurrency issues
- Fix SQL injection in partition_manager, db_optimizer, and release.ex
- Fix XSS vulnerabilities with proper HTML escaping in LiveViews
- Add proper error handling for email delivery functions
- Fix race conditions with advisory locks and atomic operations
- Replace unsupervised spawn/Task.start with supervised alternatives
- Convert ETS operations to GenServer serialization for thread safety
- Change ETS tables from :public to :protected access
- Add client limits to prevent unbounded memory growth
- Add PubSub cleanup in GenServer terminate callbacks
- Fix device upsert to use atomic Repo.insert_all
2026-03-23 16:59:09 -05:00
d234c313ca
fix: harden mobile channel numeric params 2026-03-22 17:36:09 -05:00
fdf116971d
Allow searching for objects and items by name
Updated get_latest_packet_for_callsign to search by sender, object_name,
or item_name. This allows users to search for objects like "RADIOWRLD"
or "147.06-WC" and have the map center on them, just like searching for
a station callsign.

For example:
- Searching "RADIOWRLD" now finds the GPS Central store object
- Searching "147.06-WC" finds the Calgary repeater object
- Searching "CALGRY" still finds the Calgary station

This makes objects first-class searchable entities on the map.
2026-03-22 13:07:37 -05:00
47fc071263
Add get_nearby_weather_stations/4 query function with spatial filtering
Implements PostGIS-based query to find weather stations within a radius:
- Uses ST_DWithin for efficient spatial filtering within radius (miles to meters)
- Orders results by ST_Distance (closest first)
- Filters by has_weather, has_position, and time window (default 6h)
- Deduplicates by base_callsign using DISTINCT ON (avoids duplicate SSIDs)
- Returns all weather fields: temp, humidity, pressure, wind, rain
- Configurable limit (default 50) and hours options

Comprehensive test coverage (10 tests):
- Spatial filtering within/outside radius
- Time window filtering
- Base callsign deduplication (most recent SSID)
- Distance ordering verification
- Required field validation
- Edge cases (empty results, no weather data)
2026-03-22 11:27:45 -05:00
ca05b497b9
Consolidate 24 packet columns into JSONB data field
Drop 10 dead parser-compat columns (srccallsign, dstcallsign,
origpacket, body, header, alive, posambiguity, symboltable,
symbolcode, messaging) and move 14 display-only columns into a
single `data` JSONB column (PHG, telemetry, radiorange,
information_field, format, posresolution, position_ambiguity,
luminosity, rain_midnight).

Reduces row width from ~70 to ~48 columns. Table is ephemeral
so migration uses DROP/recreate. Also fixes snow_24h -> snow bug
in weather check query and adds null byte stripping for JSONB
string sanitization.
2026-02-20 13:02:47 -06:00
84e65232e5
Batch RF path queries, batch marker removals, and JS callsign index
- Replace N+1 get_latest_packet_for_callsign calls in RF path loading
  with single get_latest_packets_for_callsigns batch query (DISTINCT ON)
- Change batch packet processing to collect marker removal IDs and push
  one remove_markers_batch event instead of individual remove_marker calls
- Add callsignIndex reverse map (callsign_group → Set<markerID>) for O(1)
  lookup in new_packet/new_packets handlers instead of scanning all markers
2026-02-20 09:04:22 -06:00
b4bbfda620
Move client-side logic to server-side LiveView
- Server tracks callsign replacements in prepare_packet_state, sends
  convert_to_historical list so JS skips full markerStates scan
- Historical packets get red dot HTML server-side in build_minimal_packet_data
  instead of JS createMarkerIcon generating it
- Add callsign_group field to all packet data output; server pre-sorts
  by group with chronological order so JS iterates directly
- Normalize heat map intensity in Clustering.cluster_packets (min/50, cap 1.0)
  instead of JS doing it per-point
- Remove buildPopupContent JS fallback and unused escapeHtml import;
  server already provides popup in all data paths
2026-02-20 08:54:34 -06:00
dcabb744df
Batch push_events, RF path queries, and ETS tuning for hot paths
- Batch real-time packet push_events: collect marker data across the
  packet batch and send a single "new_packets" WebSocket message instead
  of one per packet. JS handler processes all live→historical conversions
  before adding new markers to avoid intra-batch conflicts.
- Batch RF path station queries: replace N+1 get_latest_packet_for_callsign
  calls with a single DISTINCT ON query selecting only callsign + lat/lng.
- Add read_concurrency: true to streaming_packets_subscribers and :aprsme
  ETS tables (read-heavy, rarely written).
- Remove dead GC code in PacketConsumer (threshold > 1000 unreachable with
  max batch_size of 100).
2026-02-20 08:30:54 -06:00
8755bba9f0
Optimize historical packet loading by selecting only needed columns
Historical map loading previously fetched all 73 columns (SELECT *) from
the packets table (~2068 bytes/row). Added get_recent_packets_for_map/1
with select_map_fields/1 that fetches only the 22 columns needed for map
rendering (~200 bytes/row), yielding a ~3x query speedup.

Also removes unused all_packets tracking from PacketProcessor and
simplifies visible_packets cleanup in Index.
2026-02-20 07:41:14 -06:00
bcb3556bf6
Use PostGIS GiST index for spatial bounds queries 2026-02-20 07:29:27 -06:00
6c2b664333
Fix code review batch 3: auth, performance, dedup, cache TTL 2026-02-20 07:29:27 -06:00
7b488e9fd7
Fix code review findings batch 2: dead code, encoding, correctness
- Delete dead modules: Archiver, PacketPipelineSetup, SystemMonitor
- Encoding: fast-path for clean ASCII, fix latin1 C1 control chars (128-159)
- SpatialPubSub: fix date line grid cell wrapping
- QueryBuilder: use indexed has_weather column instead of 10-way OR
- PacketProcessor/HistoricalLoader: wire up heat map via DisplayManager
- MapLive: add PacketBatcher crash recovery with Process.monitor
- MapLive: fix time display re-render by touching assign
- DeviceCache: remove unreachable outer try/rescue
- Remove dead stubs: RateLimiterWrapper count/reset, DeviceIdentification enqueue_refresh_job
- Simplify Application.pubsub_config/0
- Add tests for SpatialPubSub, PacketBatcher, encoding, weather_only
2026-02-20 07:29:27 -06:00
0eccbd2ac7
Fix all remaining cyclomatic complexity issues
Resolves the final 5 Credo cyclomatic complexity issues by:

1. packet.ex: Create get_field_value/2 helper function
   - Replaces all `data[:field] || data["field"]` patterns
   - Reduces complexity in put_equipment_fields (11 -> 9)
   - Reduces complexity in put_phg_fields (12 -> 9)
   - Reduces complexity in put_standard_parser_fields (11 -> 9)

2. query_builder.ex: Use SQL fragment for weather_only query
   - Replaces multiple Elixir `or` operators with single SQL fragment
   - Reduces complexity from 10 to within limits

3. packet_utils.ex: Use SQL fragment for weather check query
   - Replaces multiple Elixir `or` operators with single SQL fragment
   - Reduces complexity from 11 to within limits

All 71 Credo issues now resolved:
- 14/14 warnings ✓
- 14/14 design suggestions ✓
- 40/40 refactoring issues ✓
- 3/3 code readability issues ✓

100% Credo compliance achieved!
2026-02-09 13:20:54 -06:00
d48a3a291f
Reduce cyclomatic complexity in clustering, encoding, and leader_election
Simplify complex functions to improve readability and maintainability:
- clustering.ex: Convert case statement to map lookup for cluster radii
- encoding.ex: Extract codepoint validation into separate function clauses
- leader_election.ex: Extract PID liveness checking and cleanup logic into helpers

Co-Authored-By: Graham <noreply@anthropic.com>
2026-02-09 11:57:50 -06:00
11dbf8215c
feat: Implement comprehensive performance optimizations
- Add database indexes for common query patterns (path, callsign, device, weather)
- Implement packet batching in LiveView to reduce client updates
- Create PreparedQueries module for frequently executed database queries
- Add KNN (K-nearest neighbors) spatial queries using PostGIS
- Implement configurable debouncing for map bounds and hover events
- Integrate with existing Redis cache for device lookups
- Optimize cleanup worker with batch processing (already implemented)

These optimizations significantly improve query performance and reduce
database load, especially for high-traffic scenarios.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-30 14:14:00 -05:00
48bc27e43c
fix: Resolve majority of dialyzer type errors
- Fix leader election guard test for :undefined pid
- Fix cpu_sup.util() pattern matching to handle numeric return
- Fix encoding_utils to always return explicit nil when needed
- Remove unused finite_float? function
- Fix signal handler to match :ok return from :os.set_signal/2
- Remove redundant catch-all pattern in database metrics
- Fix Gridsquare pattern matching in info_live template
- Remove unnecessary nil check for calculate_course result
- Fix get_packet_received_at to not check for nil (always returns DateTime)
- Remove redundant catch-all pattern in weather format_weather_value
- Fix query builder to use from(p in Packet) instead of bare Packet atom

Reduced dialyzer errors from 49 to 6. Remaining warnings are mostly
false positives from template compilation and overloaded function specs.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-30 13:30:21 -05:00
4348d34f62
fix weather 2025-07-12 10:53:11 -05:00
38f240b3eb
fix info/neighbor page loading 2025-07-10 14:43:00 -05:00
aaf92d9045
add heatmap on zoom out 2025-07-10 14:26:01 -05:00
d45aed256d
fix historical packet loading 2025-07-10 13:15:16 -05:00
48447ab42e
refactor and performance improvements 2025-07-10 10:53:34 -05:00