Commit graph

1062 commits

Author SHA1 Message Date
ca05b497b9
Consolidate 24 packet columns into JSONB data field
Drop 10 dead parser-compat columns (srccallsign, dstcallsign,
origpacket, body, header, alive, posambiguity, symboltable,
symbolcode, messaging) and move 14 display-only columns into a
single `data` JSONB column (PHG, telemetry, radiorange,
information_field, format, posresolution, position_ambiguity,
luminosity, rain_midnight).

Reduces row width from ~70 to ~48 columns. Table is ephemeral
so migration uses DROP/recreate. Also fixes snow_24h -> snow bug
in weather check query and adds null byte stripping for JSONB
string sanitization.
2026-02-20 13:02:47 -06:00
CI
0e53bc475d chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771612042-13bc58e [skip ci] 2026-02-20 18:31:18 +00:00
13bc58e650
Convert JS hooks to TypeScript and update esbuild to 0.25.4
Convert app.js, info_map.js, error_boundary.js, and time_ago_hook.js
to TypeScript with proper typing for Phoenix hooks, Leaflet, DOM
interactions, and timer management. Update esbuild hex package to
~> 0.10 and binary to 0.25.4. Entry point changed to app.ts.
2026-02-20 12:27:00 -06:00
CI
b562c23589 chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771608339-b08eba2 [skip ci] 2026-02-20 17:29:33 +00:00
b08eba2c6b
Partition packets table by day and simplify cleanup
Convert the packets table to a time-partitioned table with daily
partitions (PARTITION BY RANGE on received_at). This replaces the
expensive CTE-based batch DELETE cleanup with instant DROP TABLE
partition drops — no dead tuples, no WAL pressure, no VACUUM needed.

- Add PartitionManager GenServer to create/drop daily partitions
- Simplify PacketCleanupWorker to delegate to PartitionManager
- Replace gridsquare/geocalc deps with local Maidenhead module
- Remove gettext_pseudolocalize dependency
- Fix Decimal comparison in packet consumer test
2026-02-20 11:25:16 -06:00
CI
d5a1f3fe1e chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771604468-f1ead99 [skip ci] 2026-02-20 16:25:49 +00:00
f1ead99e24
Split Dockerfile build steps for better layer caching 2026-02-20 10:18:48 -06:00
84e883701d
Suppress debconf frontend warnings in Dockerfile 2026-02-20 10:17:12 -06:00
2e3fc1234b
Fix Dockerfile: use libncurses6 for Debian trixie 2026-02-20 10:15:38 -06:00
92351ef1d4
Bump Elixir 1.19.5/OTP 28.3/Debian trixie in Dockerfile and .tool-versions 2026-02-20 10:14:28 -06:00
CI
e11d8fb414 chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771602978-d09e015 [skip ci] 2026-02-20 15:57:54 +00:00
d09e015857
Fix Credo issues and telemetry Decimal coercion
- Extract prepare_markers_for_push/2 to reduce nesting depth in
  handle_info
- Split detect_item_or_object into smaller predicate and application
  functions to reduce cyclomatic complexity
- Coerce Decimal/nil values to native numbers in telemetry metrics
  (PostgreSQL sum() returns numeric type mapped to Decimal by Postgrex)
- Align locate button horizontally with zoom controls
2026-02-20 09:55:56 -06:00
CI
8dbf8d3c20 chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771602427-b8d2095 [skip ci] 2026-02-20 15:48:41 +00:00
b8d2095346
Fix UI issues: popup positioning, trail/cluster sync, CSS cleanup
- Remove CSS `bottom: 50px !important` on popups that pushed them too
  high above markers
- Hide trails when markers are clustered (zoom < 11) so trails don't
  point to invisible markers inside cluster bubbles
- Fix locate button overlap with zoom controls and keep it always
  light mode
- Remove duplicate slideover/toggle CSS from layout files (single
  source of truth in component)
- Remove dead CSS classes and standardize breakpoints to 1024px
- Add isDestroyed guards on async callbacks to prevent race conditions
- Escape callsigns in marker HTML to prevent XSS
- Replace Leaflet _container private API with getContainer()
- Clean up about page placeholder text
2026-02-20 09:46:45 -06:00
CI
e147a19043 chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771600553-4558f69 [skip ci] 2026-02-20 15:17:30 +00:00
4558f694f4
Improve trail color distinctness and road contrast
Replace trail color palette with bold, high-saturation colors that avoid
yellows, oranges, and grays (common road colors on map tiles). Interleave
warm/cool tones so consecutive color assignments are maximally distinct.

Change color assignment to prefer globally unique colors first, falling
back to proximity-based deconfliction only when all 15 colors are in use.
2026-02-20 09:15:31 -06:00
CI
52d182e25e chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771600029-a800910 [skip ci] 2026-02-20 15:07:17 +00:00
a800910dfa
Remove completed items from TODO.md 2026-02-20 09:06:49 -06:00
CI
f6823a70e0 chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771599888-84e6523 [skip ci] 2026-02-20 15:06:20 +00:00
84e65232e5
Batch RF path queries, batch marker removals, and JS callsign index
- Replace N+1 get_latest_packet_for_callsign calls in RF path loading
  with single get_latest_packets_for_callsigns batch query (DISTINCT ON)
- Change batch packet processing to collect marker removal IDs and push
  one remove_markers_batch event instead of individual remove_marker calls
- Add callsignIndex reverse map (callsign_group → Set<markerID>) for O(1)
  lookup in new_packet/new_packets handlers instead of scanning all markers
2026-02-20 09:04:22 -06:00
CI
4c8023f1f1 chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771599420-6c8df98 [skip ci] 2026-02-20 14:57:10 +00:00
6c8df98fdd
Update TODO.md with JS reverse index optimization note 2026-02-20 08:55:28 -06:00
b4bbfda620
Move client-side logic to server-side LiveView
- Server tracks callsign replacements in prepare_packet_state, sends
  convert_to_historical list so JS skips full markerStates scan
- Historical packets get red dot HTML server-side in build_minimal_packet_data
  instead of JS createMarkerIcon generating it
- Add callsign_group field to all packet data output; server pre-sorts
  by group with chronological order so JS iterates directly
- Normalize heat map intensity in Clustering.cluster_packets (min/50, cap 1.0)
  instead of JS doing it per-point
- Remove buildPopupContent JS fallback and unused escapeHtml import;
  server already provides popup in all data paths
2026-02-20 08:54:34 -06:00
CI
9365fc4fad chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771597877-46e18a6 [skip ci] 2026-02-20 14:32:49 +00:00
46e18a62e4
Add TODO.md with remaining performance improvements 2026-02-20 08:30:54 -06:00
dcabb744df
Batch push_events, RF path queries, and ETS tuning for hot paths
- Batch real-time packet push_events: collect marker data across the
  packet batch and send a single "new_packets" WebSocket message instead
  of one per packet. JS handler processes all live→historical conversions
  before adding new markers to avoid intra-batch conflicts.
- Batch RF path station queries: replace N+1 get_latest_packet_for_callsign
  calls with a single DISTINCT ON query selecting only callsign + lat/lng.
- Add read_concurrency: true to streaming_packets_subscribers and :aprsme
  ETS tables (read-heavy, rarely written).
- Remove dead GC code in PacketConsumer (threshold > 1000 unreachable with
  max batch_size of 100).
2026-02-20 08:30:54 -06:00
CI
c377dfaf67 chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771596583-781fdd8 [skip ci] 2026-02-20 14:13:33 +00:00
781fdd88a3
Remove dead modules, unused deps, and commented-out code
Delete 5 unused modules: Presence (untracked), LiveView ShutdownHandler
(unused mixin), RateLimiterWrapper (pointless delegation), Passcode
(only used by own test), MapLive.Utils (pure delegates).

Fix ShutdownHandler to not call deleted Presence. Inline ParamUtils
at call sites. Remove cachex dep and 27 stale entries from mix.lock.
Clean commented-out Exq/Redis config from application.ex.
2026-02-20 08:09:22 -06:00
CI
d58043488c chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771595402-6ff8838 [skip ci] 2026-02-20 13:50:11 +00:00
6ff88380e5
Remove TODO.md — all items assessed or completed 2026-02-20 07:48:55 -06:00
4153f66b86
Remove broken dual packet state system (PacketManager/PacketStore)
System A (PacketManager + PacketStore ETS) had a fatal ID mismatch:
extract_packet_ids returned Ecto UUIDs but PacketStore stored under
MD5 hashes — cleanup could never find stored packets. Unified all
packet tracking on visible_packets map in LiveView assigns.

Deletes packet_manager.ex and packet_store.ex, removes from
supervision tree and PacketDistributor.
2026-02-20 07:47:39 -06:00
CI
b1358a8fad chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771594898-8755bba [skip ci] 2026-02-20 13:43:13 +00:00
8755bba9f0
Optimize historical packet loading by selecting only needed columns
Historical map loading previously fetched all 73 columns (SELECT *) from
the packets table (~2068 bytes/row). Added get_recent_packets_for_map/1
with select_map_fields/1 that fetches only the 22 columns needed for map
rendering (~200 bytes/row), yielding a ~3x query speedup.

Also removes unused all_packets tracking from PacketProcessor and
simplifies visible_packets cleanup in Index.
2026-02-20 07:41:14 -06:00
1832a8ef04
Restore TODO.md with remaining items at top, completed at bottom 2026-02-20 07:41:14 -06:00
CI
9886c9171f chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771594193-bcb3556 [skip ci] 2026-02-20 13:31:28 +00:00
bcb3556bf6
Use PostGIS GiST index for spatial bounds queries 2026-02-20 07:29:27 -06:00
6c2b664333
Fix code review batch 3: auth, performance, dedup, cache TTL 2026-02-20 07:29:27 -06:00
7b488e9fd7
Fix code review findings batch 2: dead code, encoding, correctness
- Delete dead modules: Archiver, PacketPipelineSetup, SystemMonitor
- Encoding: fast-path for clean ASCII, fix latin1 C1 control chars (128-159)
- SpatialPubSub: fix date line grid cell wrapping
- QueryBuilder: use indexed has_weather column instead of 10-way OR
- PacketProcessor/HistoricalLoader: wire up heat map via DisplayManager
- MapLive: add PacketBatcher crash recovery with Process.monitor
- MapLive: fix time display re-render by touching assign
- DeviceCache: remove unreachable outer try/rescue
- Remove dead stubs: RateLimiterWrapper count/reset, DeviceIdentification enqueue_refresh_job
- Simplify Application.pubsub_config/0
- Add tests for SpatialPubSub, PacketBatcher, encoding, weather_only
2026-02-20 07:29:27 -06:00
CI
b5eb59d169 chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771549280-793947a [skip ci] 2026-02-20 01:02:41 +00:00
793947a3e8
Align table headers left to match content on info page 2026-02-19 19:00:58 -06:00
CI
b2bc6e27b7 chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771548932-993af84 [skip ci] 2026-02-20 00:56:54 +00:00
993af8447e
Move APRS icons before callsigns on info page
Reorder icon and callsign in the page header, Other SSIDs table, and
Stations Near Current Position table so the icon appears first. Fix
icon clipping with flex-shrink-0 h-8 and add py-2 padding to SSIDs
table cells for proper vertical alignment.
2026-02-19 18:55:11 -06:00
CI
1caa2fe6bc chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771548585-d9c04f7 [skip ci] 2026-02-20 00:51:10 +00:00
d9c04f7e0c
Fix code review findings: security, correctness, performance, dead code
Security:
- Add SQL identifier validation to DbOptimizer.analyze_table/vacuum_table
- Move health/readiness routes to non-rate-limited pipeline

Correctness:
- Fix monitor reference leak in StreamingPacketsPubSub (track refs, demonitor on unsubscribe)
- Persist circuit breaker half-open state transitions
- Simplify ShutdownHandler.terminate to avoid pointless Process.send_after

Performance:
- Batch marker removal into single WebSocket event (remove_markers_batch)
- Use Task.Supervisor.start_child for fire-and-forget broadcasts

Dead code removal:
- Remove copy_insert/regular_batch_insert/rows_to_csv from DbOptimizer
- Remove dead packet_pipeline_integration tests
- Remove unused telemetry event detach

Other:
- Fix Callsign.valid? docstring to match permissive behavior
- Fix DatabaseMetrics to delegate instead of hardcoding pool stats
- Add circuit breaker and encoding test coverage
2026-02-19 18:49:27 -06:00
CI
0084539bb1 chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771548120-118a55b [skip ci] 2026-02-20 00:43:20 +00:00
118a55b26e
Fix APRS object name extraction for names with special characters
The object name regex only matched [A-Z0-9 ] which missed names with
hyphens (P-K5SGD), hash symbols (#146.760), and other printable ASCII.
Changed to .{9}[*_] to match all valid object names including killed
objects. Also merged the two cond branches so data_extended is tried
first with information_field parsing as fallback.

Removed dead copy_insert/3 tests from db_optimizer_test.
2026-02-19 18:41:37 -06:00
8a8e16aaba
Expand API test response to full container width
Move JSON response display outside the max-w-md form container
so API results use the full available width instead of being
constrained to the narrow form width.
2026-02-19 18:41:37 -06:00
CI
39bc7e3eaa chore: update aprs.me image to git.mcintire.me/graham/aprs.me:main-1771547268-27f9b32 [skip ci] 2026-02-20 00:29:11 +00:00
27f9b32dff
Fix duplicate station icons when live packet replaces historical
Historical loading overrides all markers to historical:true, including
the most-recent marker. When a new live packet arrives, the conversion
check only looked for non-historical markers, missing the existing
"current" marker. Broaden the check to also convert markers with
is_most_recent_for_callsign set.
2026-02-19 18:27:26 -06:00
ac51ecaada
Fix multiple TODO bugs: drain handler, callsign dedup, packet pipeline
- Fix drain_connections missing {:noreply, socket} wrapper
- Fix get_callsign_key to match atom keys from Ecto structs
- Add SpatialPubSub.broadcast_packet to PacketDistributor
- Fix pid_alive? treating {:badrpc, _} as truthy
- Fix SpatialPubSub ensure_float crash on integer strings
- Fix inverted memory calculation in ConnectionMonitor
- Remove dead SSL handler from Is module, consolidate into handle_socket_data
- Remove duplicate packet transformation in Is.dispatch
- Remove invalid placeholders option from insert_all calls
- Change PacketPipelineSupervisor to :rest_for_one strategy
- Update TODO.md marking fixed items
2026-02-19 18:27:25 -06:00