Commit graph

1040 commits

Author SHA1 Message Date
b4bbfda620
Move client-side logic to server-side LiveView
- Server tracks callsign replacements in prepare_packet_state, sends
  convert_to_historical list so JS skips full markerStates scan
- Historical packets get red dot HTML server-side in build_minimal_packet_data
  instead of JS createMarkerIcon generating it
- Add callsign_group field to all packet data output; server pre-sorts
  by group with chronological order so JS iterates directly
- Normalize heat map intensity in Clustering.cluster_packets (min/50, cap 1.0)
  instead of JS doing it per-point
- Remove buildPopupContent JS fallback and unused escapeHtml import;
  server already provides popup in all data paths
2026-02-20 08:54:34 -06:00
CI
9365fc4fad chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771597877-46e18a6 [skip ci] 2026-02-20 14:32:49 +00:00
46e18a62e4
Add TODO.md with remaining performance improvements 2026-02-20 08:30:54 -06:00
dcabb744df
Batch push_events, RF path queries, and ETS tuning for hot paths
- Batch real-time packet push_events: collect marker data across the
  packet batch and send a single "new_packets" WebSocket message instead
  of one per packet. JS handler processes all live→historical conversions
  before adding new markers to avoid intra-batch conflicts.
- Batch RF path station queries: replace N+1 get_latest_packet_for_callsign
  calls with a single DISTINCT ON query selecting only callsign + lat/lng.
- Add read_concurrency: true to streaming_packets_subscribers and :aprsme
  ETS tables (read-heavy, rarely written).
- Remove dead GC code in PacketConsumer (threshold > 1000 unreachable with
  max batch_size of 100).
2026-02-20 08:30:54 -06:00
CI
c377dfaf67 chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771596583-781fdd8 [skip ci] 2026-02-20 14:13:33 +00:00
781fdd88a3
Remove dead modules, unused deps, and commented-out code
Delete 5 unused modules: Presence (untracked), LiveView ShutdownHandler
(unused mixin), RateLimiterWrapper (pointless delegation), Passcode
(only used by own test), MapLive.Utils (pure delegates).

Fix ShutdownHandler to not call deleted Presence. Inline ParamUtils
at call sites. Remove cachex dep and 27 stale entries from mix.lock.
Clean commented-out Exq/Redis config from application.ex.
2026-02-20 08:09:22 -06:00
CI
d58043488c chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771595402-6ff8838 [skip ci] 2026-02-20 13:50:11 +00:00
6ff88380e5
Remove TODO.md — all items assessed or completed 2026-02-20 07:48:55 -06:00
4153f66b86
Remove broken dual packet state system (PacketManager/PacketStore)
System A (PacketManager + PacketStore ETS) had a fatal ID mismatch:
extract_packet_ids returned Ecto UUIDs but PacketStore stored under
MD5 hashes — cleanup could never find stored packets. Unified all
packet tracking on visible_packets map in LiveView assigns.

Deletes packet_manager.ex and packet_store.ex, removes from
supervision tree and PacketDistributor.
2026-02-20 07:47:39 -06:00
CI
b1358a8fad chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771594898-8755bba [skip ci] 2026-02-20 13:43:13 +00:00
8755bba9f0
Optimize historical packet loading by selecting only needed columns
Historical map loading previously fetched all 73 columns (SELECT *) from
the packets table (~2068 bytes/row). Added get_recent_packets_for_map/1
with select_map_fields/1 that fetches only the 22 columns needed for map
rendering (~200 bytes/row), yielding a ~3x query speedup.

Also removes unused all_packets tracking from PacketProcessor and
simplifies visible_packets cleanup in Index.
2026-02-20 07:41:14 -06:00
1832a8ef04
Restore TODO.md with remaining items at top, completed at bottom 2026-02-20 07:41:14 -06:00
CI
9886c9171f chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771594193-bcb3556 [skip ci] 2026-02-20 13:31:28 +00:00
bcb3556bf6
Use PostGIS GiST index for spatial bounds queries 2026-02-20 07:29:27 -06:00
6c2b664333
Fix code review batch 3: auth, performance, dedup, cache TTL 2026-02-20 07:29:27 -06:00
7b488e9fd7
Fix code review findings batch 2: dead code, encoding, correctness
- Delete dead modules: Archiver, PacketPipelineSetup, SystemMonitor
- Encoding: fast-path for clean ASCII, fix latin1 C1 control chars (128-159)
- SpatialPubSub: fix date line grid cell wrapping
- QueryBuilder: use indexed has_weather column instead of 10-way OR
- PacketProcessor/HistoricalLoader: wire up heat map via DisplayManager
- MapLive: add PacketBatcher crash recovery with Process.monitor
- MapLive: fix time display re-render by touching assign
- DeviceCache: remove unreachable outer try/rescue
- Remove dead stubs: RateLimiterWrapper count/reset, DeviceIdentification enqueue_refresh_job
- Simplify Application.pubsub_config/0
- Add tests for SpatialPubSub, PacketBatcher, encoding, weather_only
2026-02-20 07:29:27 -06:00
CI
b5eb59d169 chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771549280-793947a [skip ci] 2026-02-20 01:02:41 +00:00
793947a3e8
Align table headers left to match content on info page 2026-02-19 19:00:58 -06:00
CI
b2bc6e27b7 chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771548932-993af84 [skip ci] 2026-02-20 00:56:54 +00:00
993af8447e
Move APRS icons before callsigns on info page
Reorder icon and callsign in the page header, Other SSIDs table, and
Stations Near Current Position table so the icon appears first. Fix
icon clipping with flex-shrink-0 h-8 and add py-2 padding to SSIDs
table cells for proper vertical alignment.
2026-02-19 18:55:11 -06:00
CI
1caa2fe6bc chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771548585-d9c04f7 [skip ci] 2026-02-20 00:51:10 +00:00
d9c04f7e0c
Fix code review findings: security, correctness, performance, dead code
Security:
- Add SQL identifier validation to DbOptimizer.analyze_table/vacuum_table
- Move health/readiness routes to non-rate-limited pipeline

Correctness:
- Fix monitor reference leak in StreamingPacketsPubSub (track refs, demonitor on unsubscribe)
- Persist circuit breaker half-open state transitions
- Simplify ShutdownHandler.terminate to avoid pointless Process.send_after

Performance:
- Batch marker removal into single WebSocket event (remove_markers_batch)
- Use Task.Supervisor.start_child for fire-and-forget broadcasts

Dead code removal:
- Remove copy_insert/regular_batch_insert/rows_to_csv from DbOptimizer
- Remove dead packet_pipeline_integration tests
- Remove unused telemetry event detach

Other:
- Fix Callsign.valid? docstring to match permissive behavior
- Fix DatabaseMetrics to delegate instead of hardcoding pool stats
- Add circuit breaker and encoding test coverage
2026-02-19 18:49:27 -06:00
CI
0084539bb1 chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771548120-118a55b [skip ci] 2026-02-20 00:43:20 +00:00
118a55b26e
Fix APRS object name extraction for names with special characters
The object name regex only matched [A-Z0-9 ] which missed names with
hyphens (P-K5SGD), hash symbols (#146.760), and other printable ASCII.
Changed to .{9}[*_] to match all valid object names including killed
objects. Also merged the two cond branches so data_extended is tried
first with information_field parsing as fallback.

Removed dead copy_insert/3 tests from db_optimizer_test.
2026-02-19 18:41:37 -06:00
8a8e16aaba
Expand API test response to full container width
Move JSON response display outside the max-w-md form container
so API results use the full available width instead of being
constrained to the narrow form width.
2026-02-19 18:41:37 -06:00
CI
39bc7e3eaa chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771547268-27f9b32 [skip ci] 2026-02-20 00:29:11 +00:00
27f9b32dff
Fix duplicate station icons when live packet replaces historical
Historical loading overrides all markers to historical:true, including
the most-recent marker. When a new live packet arrives, the conversion
check only looked for non-historical markers, missing the existing
"current" marker. Broaden the check to also convert markers with
is_most_recent_for_callsign set.
2026-02-19 18:27:26 -06:00
ac51ecaada
Fix multiple TODO bugs: drain handler, callsign dedup, packet pipeline
- Fix drain_connections missing {:noreply, socket} wrapper
- Fix get_callsign_key to match atom keys from Ecto structs
- Add SpatialPubSub.broadcast_packet to PacketDistributor
- Fix pid_alive? treating {:badrpc, _} as truthy
- Fix SpatialPubSub ensure_float crash on integer strings
- Fix inverted memory calculation in ConnectionMonitor
- Remove dead SSL handler from Is module, consolidate into handle_socket_data
- Remove duplicate packet transformation in Is.dispatch
- Remove invalid placeholders option from insert_all calls
- Change PacketPipelineSupervisor to :rest_for_one strategy
- Update TODO.md marking fixed items
2026-02-19 18:27:25 -06:00
CI
c8777f9687 chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771546872-2030a7c [skip ci] 2026-02-20 00:22:33 +00:00
2030a7c8a6
Fix trail hover showing wrong igate RF path
Use mousemove instead of mouseover so the RF path updates as the
cursor moves along the trail. Send the mouse position rather than
the nearest-position's coordinates so the path line anchors where
the user is actually hovering. Debounce and deduplicate events to
avoid flooding the server.
2026-02-19 18:20:49 -06:00
CI
a42dc1d68f chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771546731-753fa50 [skip ci] 2026-02-20 00:20:13 +00:00
753fa50463
Fix ConnectionMonitor deadlock and LeaderElection silent leadership loss
ConnectionMonitor.gather_cluster_stats was calling :rpc.call(Node.self())
which GenServer.call'd back into itself while blocked in handle_info,
causing a 5-second deadlock timeout every 30 seconds. Fixed by computing
local stats directly from state.

LeaderElection never verified that a leader still held the :global
registration. After :global conflict resolution (two isolated nodes
merging), the loser was silently unregistered but stayed is_leader: true
forever, causing both pods to connect to APRS-IS with the same callsign
in a reconnect loop. Fixed by adding verify_leadership/1 to
check_leadership which detects lost registrations and steps down.
2026-02-19 18:18:31 -06:00
CI
6e29bd80be chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771544325-29da64a [skip ci] 2026-02-19 23:38:53 +00:00
29da64a991
Remove TODO.md — all items assessed or completed 2026-02-19 17:38:26 -06:00
CI
cb884606f3 chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771544148-497ad85 [skip ci] 2026-02-19 23:35:56 +00:00
497ad85af2
Assess bounds update loading indicator TODO as not needed
Existing historical loading spinner already covers the slow DB
query path; the only unindicated gap is the 400ms debounce which
is too short for a spinner without causing flicker.
2026-02-19 17:35:28 -06:00
CI
8c1a3c84ab chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771543306-66061a5 [skip ci] 2026-02-19 23:23:10 +00:00
66061a5abe
Fix leader election: prevent self-deregistration on re-election
:global.register_name returns :no even when the name is already
registered to the calling PID. This caused the leader to clear its
own leadership state every 5 seconds after cluster formation triggered
a redundant re-election attempt.

Now checks if we already hold the registration before attempting to
re-register, preserving leadership state across cluster topology changes.
2026-02-19 17:21:26 -06:00
CI
dda4bbce15 chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771538611-976b6f1 [skip ci] 2026-02-19 22:04:59 +00:00
976b6f153f
Fix slideover toggle: always-light button, map resize via LiveView JS
The slideover toggle button now uses a fixed light theme (white bg,
dark arrow) so it's always visible on the map regardless of OS dark
mode. Removed prefers-color-scheme overrides since the button overlays
the always-light map tiles.

Fixed map not resizing when closing the sidebar — the #aprs-map div
has phx-update="ignore" so server-rendered classes never updated.
Now uses LiveView JS commands (JS.toggle_class + JS.dispatch) to
toggle classes client-side and trigger map.invalidateSize(), replacing
the custom push_event/handleEvent JS handler.
2026-02-19 16:02:08 -06:00
75ceb73607
Fix Erlang clustering: use IP-based node names with DNS strategy
DNSSRV strategy resolves to IP-based names but nodes were named by
pod name, causing connection failures. Switch to DNS strategy with
IP-based node naming so libcluster discovery matches actual node names.
2026-02-19 16:01:20 -06:00
CI
bff63bb8a8 chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771538286-d182024 [skip ci] 2026-02-19 21:58:14 +00:00
d182024b8a
Scale to 2 replicas and enable Erlang clustering 2026-02-19 15:57:42 -06:00
CI
ae9d8eb595 chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771537665-ca9ad68 [skip ci] 2026-02-19 21:49:06 +00:00
ca9ad68c25
Drop 31 unused indexes on packets table (~4.6 GB)
Exact duplicates, indexes subsumed by better composites,
dead lower() indexes (codebase uses upper()), and indexes
with no matching query pattern in the codebase.
2026-02-19 15:47:27 -06:00
CI
a9b0fad1de chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771537316-14fa16d [skip ci] 2026-02-19 21:43:18 +00:00
14fa16df6d
Fix RF path visualization for hovering over track points
parse_rf_path was filtering out all paths containing qA* constructs,
which is virtually every APRS-IS packet. The qA* element delimits the
RF path from the APRS-IS metadata — it should be split on, not
discarded. Now properly extracts digipeaters (before qA*) and the
igate (after qA*), while filtering APRS aliases like WIDE/RELAY/TRACE.

Also adds hover handlers to trail polylines so hovering anywhere on a
call's track shows the RF path, not just on the dot markers.
2026-02-19 15:41:32 -06:00
CI
91ced547b1 chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771536841-1d0b1ba [skip ci] 2026-02-19 21:34:09 +00:00
1d0b1ba99c
Mark backpressure item as done in TODO.md 2026-02-19 15:33:22 -06:00
fa89c6d3ca
Add TCP backpressure to packet pipeline
When the PacketProducer buffer crosses 80% capacity, signal Aprsme.Is
to switch its TCP socket to passive mode, leveraging TCP flow control
to slow APRS-IS intake. Resume active mode when buffer drains below 30%.

A 30-second safety valve timer auto-resumes if backpressure gets stuck.
Disconnect/reconnect/terminate paths all clear backpressure state.
2026-02-19 15:31:51 -06:00