Commit graph

595 commits

Author SHA1 Message Date
f4db38f051
update 2026-02-25 16:29:45 -06:00
d385d1eb75
Default sidebar to open on desktop via CSS
Make the map and panel CSS default to "sidebar open" on desktop
(>= 1024px) without requiring the slideover-open class. Only the
slideover-closed class now hides the sidebar. This prevents a blank
gap when the map and panel states get out of sync after LiveView
reconnects.
2026-02-25 12:25:50 -06:00
f5049a6e2d
updates 2026-02-22 15:33:35 -06:00
d22fc7c661
Fix dialyzer type errors in coordinate validation
- Remove redundant nil check for callsign (spec guarantees String.t())
- Simplify coordinate validation by removing faulty infinity checks
- Remove unreachable catch-all clause in has_weather_packets?

Floats cannot equal atoms :infinity or :neg_infinity in Elixir.
Range checks already ensure finite values, making explicit infinity
checks unnecessary. Reduces dialyzer errors from 40 to 34.
2026-02-20 18:17:59 -06:00
b45993a8be
Fix APRS object display and comment encoding issues
- Strip Mic-E telemetry from comments before database storage
- Show sender callsign on map labels instead of object names for clarity
- Object names still used for grouping to maintain trail organization
- Add comprehensive tests for new encoding utilities

Fixes display issues where weather balloon "X3234025" was shown
instead of station "WA0YMH", and Mic-E telemetry bytes like "!w>`!"
were appearing in comments.
2026-02-20 18:06:32 -06:00
0061c4b169
Apply TailwindCSS Plus design patterns across site
Dark navbar header with radio icon, refined slideover control panel
with branded header and section cards, polished cards/tables/badges
on info, packets, and about pages using ring borders and shadow-xs.
2026-02-20 17:56:06 -06:00
cf97bf236a
Add client IP to request log metadata
Set Logger metadata from conn.remote_ip in the RemoteIp plug so
request logs include the client address.
2026-02-20 17:48:15 -06:00
4481888976
Fix settings page showing validation errors on initial load
The email and callsign changesets add "did not change" errors when
created with empty attrs (on mount). Guard all field error displays
on changeset.action being set so errors only appear after user
interaction.
2026-02-20 17:09:18 -06:00
f803c46b92
Fix KeyError on :data when rendering raw broadcast packets in callsign view
Raw APRS broadcast payloads don't have a :data key (only %Packet{}
structs do). Use Map.get/3 instead of dot-access to handle both cases.
2026-02-20 14:12:21 -06:00
bcb00801ee
Remove noisy leader election log
The "Not elected as leader" info log was being emitted every 5 seconds
by non-leader nodes, filling up logs unnecessarily in clustered deployments.
2026-02-20 13:43:13 -06:00
50544c8354
Remove dead PostGIS code, deduplicate utilities, and simplify module hierarchy
- Delete ~170 lines of commented-out PostGIS spatial query functions
- Consolidate coordinate validation into CoordinateUtils
- Replace private normalize_data_type with EncodingUtils delegation
- Inline to_float/to_decimal wrappers at call sites
- Fix has_weather_data? missing wind_gust field
- Remove BoundsManager passthrough delegation module
- Simplify MapLive.PacketUtils by removing pure-passthrough delegations
2026-02-20 13:39:07 -06:00
673eb55e97
Add test coverage for comment cleaning edge cases
Cover weather data with spaces/dots for missing values, negative
temperatures, negative altitudes, combined PHG+altitude+RNG
stripping, nil comments, comments that become empty after cleaning,
and false-match protection for comments starting with P or h. Also
fix strip_weather_data to require at least one weather field after
the preamble to avoid false-matching course/speed in non-weather
comments.
2026-02-20 13:17:27 -06:00
ddf26a99ce
Strip weather data, RNG, and negative altitudes from packet comments
Weather packets had raw data strings (e.g. 007/000g000t054...)
stored as the comment instead of the trailing human-readable text.
Now iteratively strips fixed-width APRS weather fields in any order,
handling both position-included (XXX/XXX) and positionless (cXXXsXXX)
formats. Also strips RNG data and fixes altitude regex to handle
negative values. Empty comments after cleaning return nil.
2026-02-20 13:15:19 -06:00
ca05b497b9
Consolidate 24 packet columns into JSONB data field
Drop 10 dead parser-compat columns (srccallsign, dstcallsign,
origpacket, body, header, alive, posambiguity, symboltable,
symbolcode, messaging) and move 14 display-only columns into a
single `data` JSONB column (PHG, telemetry, radiorange,
information_field, format, posresolution, position_ambiguity,
luminosity, rain_midnight).

Reduces row width from ~70 to ~48 columns. Table is ephemeral
so migration uses DROP/recreate. Also fixes snow_24h -> snow bug
in weather check query and adds null byte stripping for JSONB
string sanitization.
2026-02-20 13:02:47 -06:00
b08eba2c6b
Partition packets table by day and simplify cleanup
Convert the packets table to a time-partitioned table with daily
partitions (PARTITION BY RANGE on received_at). This replaces the
expensive CTE-based batch DELETE cleanup with instant DROP TABLE
partition drops — no dead tuples, no WAL pressure, no VACUUM needed.

- Add PartitionManager GenServer to create/drop daily partitions
- Simplify PacketCleanupWorker to delegate to PartitionManager
- Replace gridsquare/geocalc deps with local Maidenhead module
- Remove gettext_pseudolocalize dependency
- Fix Decimal comparison in packet consumer test
2026-02-20 11:25:16 -06:00
d09e015857
Fix Credo issues and telemetry Decimal coercion
- Extract prepare_markers_for_push/2 to reduce nesting depth in
  handle_info
- Split detect_item_or_object into smaller predicate and application
  functions to reduce cyclomatic complexity
- Coerce Decimal/nil values to native numbers in telemetry metrics
  (PostgreSQL sum() returns numeric type mapped to Decimal by Postgrex)
- Align locate button horizontally with zoom controls
2026-02-20 09:55:56 -06:00
b8d2095346
Fix UI issues: popup positioning, trail/cluster sync, CSS cleanup
- Remove CSS `bottom: 50px !important` on popups that pushed them too
  high above markers
- Hide trails when markers are clustered (zoom < 11) so trails don't
  point to invisible markers inside cluster bubbles
- Fix locate button overlap with zoom controls and keep it always
  light mode
- Remove duplicate slideover/toggle CSS from layout files (single
  source of truth in component)
- Remove dead CSS classes and standardize breakpoints to 1024px
- Add isDestroyed guards on async callbacks to prevent race conditions
- Escape callsigns in marker HTML to prevent XSS
- Replace Leaflet _container private API with getContainer()
- Clean up about page placeholder text
2026-02-20 09:46:45 -06:00
84e65232e5
Batch RF path queries, batch marker removals, and JS callsign index
- Replace N+1 get_latest_packet_for_callsign calls in RF path loading
  with single get_latest_packets_for_callsigns batch query (DISTINCT ON)
- Change batch packet processing to collect marker removal IDs and push
  one remove_markers_batch event instead of individual remove_marker calls
- Add callsignIndex reverse map (callsign_group → Set<markerID>) for O(1)
  lookup in new_packet/new_packets handlers instead of scanning all markers
2026-02-20 09:04:22 -06:00
b4bbfda620
Move client-side logic to server-side LiveView
- Server tracks callsign replacements in prepare_packet_state, sends
  convert_to_historical list so JS skips full markerStates scan
- Historical packets get red dot HTML server-side in build_minimal_packet_data
  instead of JS createMarkerIcon generating it
- Add callsign_group field to all packet data output; server pre-sorts
  by group with chronological order so JS iterates directly
- Normalize heat map intensity in Clustering.cluster_packets (min/50, cap 1.0)
  instead of JS doing it per-point
- Remove buildPopupContent JS fallback and unused escapeHtml import;
  server already provides popup in all data paths
2026-02-20 08:54:34 -06:00
dcabb744df
Batch push_events, RF path queries, and ETS tuning for hot paths
- Batch real-time packet push_events: collect marker data across the
  packet batch and send a single "new_packets" WebSocket message instead
  of one per packet. JS handler processes all live→historical conversions
  before adding new markers to avoid intra-batch conflicts.
- Batch RF path station queries: replace N+1 get_latest_packet_for_callsign
  calls with a single DISTINCT ON query selecting only callsign + lat/lng.
- Add read_concurrency: true to streaming_packets_subscribers and :aprsme
  ETS tables (read-heavy, rarely written).
- Remove dead GC code in PacketConsumer (threshold > 1000 unreachable with
  max batch_size of 100).
2026-02-20 08:30:54 -06:00
781fdd88a3
Remove dead modules, unused deps, and commented-out code
Delete 5 unused modules: Presence (untracked), LiveView ShutdownHandler
(unused mixin), RateLimiterWrapper (pointless delegation), Passcode
(only used by own test), MapLive.Utils (pure delegates).

Fix ShutdownHandler to not call deleted Presence. Inline ParamUtils
at call sites. Remove cachex dep and 27 stale entries from mix.lock.
Clean commented-out Exq/Redis config from application.ex.
2026-02-20 08:09:22 -06:00
4153f66b86
Remove broken dual packet state system (PacketManager/PacketStore)
System A (PacketManager + PacketStore ETS) had a fatal ID mismatch:
extract_packet_ids returned Ecto UUIDs but PacketStore stored under
MD5 hashes — cleanup could never find stored packets. Unified all
packet tracking on visible_packets map in LiveView assigns.

Deletes packet_manager.ex and packet_store.ex, removes from
supervision tree and PacketDistributor.
2026-02-20 07:47:39 -06:00
8755bba9f0
Optimize historical packet loading by selecting only needed columns
Historical map loading previously fetched all 73 columns (SELECT *) from
the packets table (~2068 bytes/row). Added get_recent_packets_for_map/1
with select_map_fields/1 that fetches only the 22 columns needed for map
rendering (~200 bytes/row), yielding a ~3x query speedup.

Also removes unused all_packets tracking from PacketProcessor and
simplifies visible_packets cleanup in Index.
2026-02-20 07:41:14 -06:00
bcb3556bf6
Use PostGIS GiST index for spatial bounds queries 2026-02-20 07:29:27 -06:00
6c2b664333
Fix code review batch 3: auth, performance, dedup, cache TTL 2026-02-20 07:29:27 -06:00
7b488e9fd7
Fix code review findings batch 2: dead code, encoding, correctness
- Delete dead modules: Archiver, PacketPipelineSetup, SystemMonitor
- Encoding: fast-path for clean ASCII, fix latin1 C1 control chars (128-159)
- SpatialPubSub: fix date line grid cell wrapping
- QueryBuilder: use indexed has_weather column instead of 10-way OR
- PacketProcessor/HistoricalLoader: wire up heat map via DisplayManager
- MapLive: add PacketBatcher crash recovery with Process.monitor
- MapLive: fix time display re-render by touching assign
- DeviceCache: remove unreachable outer try/rescue
- Remove dead stubs: RateLimiterWrapper count/reset, DeviceIdentification enqueue_refresh_job
- Simplify Application.pubsub_config/0
- Add tests for SpatialPubSub, PacketBatcher, encoding, weather_only
2026-02-20 07:29:27 -06:00
793947a3e8
Align table headers left to match content on info page 2026-02-19 19:00:58 -06:00
993af8447e
Move APRS icons before callsigns on info page
Reorder icon and callsign in the page header, Other SSIDs table, and
Stations Near Current Position table so the icon appears first. Fix
icon clipping with flex-shrink-0 h-8 and add py-2 padding to SSIDs
table cells for proper vertical alignment.
2026-02-19 18:55:11 -06:00
d9c04f7e0c
Fix code review findings: security, correctness, performance, dead code
Security:
- Add SQL identifier validation to DbOptimizer.analyze_table/vacuum_table
- Move health/readiness routes to non-rate-limited pipeline

Correctness:
- Fix monitor reference leak in StreamingPacketsPubSub (track refs, demonitor on unsubscribe)
- Persist circuit breaker half-open state transitions
- Simplify ShutdownHandler.terminate to avoid pointless Process.send_after

Performance:
- Batch marker removal into single WebSocket event (remove_markers_batch)
- Use Task.Supervisor.start_child for fire-and-forget broadcasts

Dead code removal:
- Remove copy_insert/regular_batch_insert/rows_to_csv from DbOptimizer
- Remove dead packet_pipeline_integration tests
- Remove unused telemetry event detach

Other:
- Fix Callsign.valid? docstring to match permissive behavior
- Fix DatabaseMetrics to delegate instead of hardcoding pool stats
- Add circuit breaker and encoding test coverage
2026-02-19 18:49:27 -06:00
118a55b26e
Fix APRS object name extraction for names with special characters
The object name regex only matched [A-Z0-9 ] which missed names with
hyphens (P-K5SGD), hash symbols (#146.760), and other printable ASCII.
Changed to .{9}[*_] to match all valid object names including killed
objects. Also merged the two cond branches so data_extended is tried
first with information_field parsing as fallback.

Removed dead copy_insert/3 tests from db_optimizer_test.
2026-02-19 18:41:37 -06:00
8a8e16aaba
Expand API test response to full container width
Move JSON response display outside the max-w-md form container
so API results use the full available width instead of being
constrained to the narrow form width.
2026-02-19 18:41:37 -06:00
ac51ecaada
Fix multiple TODO bugs: drain handler, callsign dedup, packet pipeline
- Fix drain_connections missing {:noreply, socket} wrapper
- Fix get_callsign_key to match atom keys from Ecto structs
- Add SpatialPubSub.broadcast_packet to PacketDistributor
- Fix pid_alive? treating {:badrpc, _} as truthy
- Fix SpatialPubSub ensure_float crash on integer strings
- Fix inverted memory calculation in ConnectionMonitor
- Remove dead SSL handler from Is module, consolidate into handle_socket_data
- Remove duplicate packet transformation in Is.dispatch
- Remove invalid placeholders option from insert_all calls
- Change PacketPipelineSupervisor to :rest_for_one strategy
- Update TODO.md marking fixed items
2026-02-19 18:27:25 -06:00
753fa50463
Fix ConnectionMonitor deadlock and LeaderElection silent leadership loss
ConnectionMonitor.gather_cluster_stats was calling :rpc.call(Node.self())
which GenServer.call'd back into itself while blocked in handle_info,
causing a 5-second deadlock timeout every 30 seconds. Fixed by computing
local stats directly from state.

LeaderElection never verified that a leader still held the :global
registration. After :global conflict resolution (two isolated nodes
merging), the loser was silently unregistered but stayed is_leader: true
forever, causing both pods to connect to APRS-IS with the same callsign
in a reconnect loop. Fixed by adding verify_leadership/1 to
check_leadership which detects lost registrations and steps down.
2026-02-19 18:18:31 -06:00
66061a5abe
Fix leader election: prevent self-deregistration on re-election
:global.register_name returns :no even when the name is already
registered to the calling PID. This caused the leader to clear its
own leadership state every 5 seconds after cluster formation triggered
a redundant re-election attempt.

Now checks if we already hold the registration before attempting to
re-register, preserving leadership state across cluster topology changes.
2026-02-19 17:21:26 -06:00
976b6f153f
Fix slideover toggle: always-light button, map resize via LiveView JS
The slideover toggle button now uses a fixed light theme (white bg,
dark arrow) so it's always visible on the map regardless of OS dark
mode. Removed prefers-color-scheme overrides since the button overlays
the always-light map tiles.

Fixed map not resizing when closing the sidebar — the #aprs-map div
has phx-update="ignore" so server-rendered classes never updated.
Now uses LiveView JS commands (JS.toggle_class + JS.dispatch) to
toggle classes client-side and trigger map.invalidateSize(), replacing
the custom push_event/handleEvent JS handler.
2026-02-19 16:02:08 -06:00
14fa16df6d
Fix RF path visualization for hovering over track points
parse_rf_path was filtering out all paths containing qA* constructs,
which is virtually every APRS-IS packet. The qA* element delimits the
RF path from the APRS-IS metadata — it should be split on, not
discarded. Now properly extracts digipeaters (before qA*) and the
igate (after qA*), while filtering APRS aliases like WIDE/RELAY/TRACE.

Also adds hover handlers to trail polylines so hovering anywhere on a
call's track shows the RF path, not just on the dot markers.
2026-02-19 15:41:32 -06:00
fa89c6d3ca
Add TCP backpressure to packet pipeline
When the PacketProducer buffer crosses 80% capacity, signal Aprsme.Is
to switch its TCP socket to passive mode, leveraging TCP flow control
to slow APRS-IS intake. Resume active mode when buffer drains below 30%.

A 30-second safety valve timer auto-resumes if backpressure gets stuck.
Disconnect/reconnect/terminate paths all clear backpressure state.
2026-02-19 15:31:51 -06:00
15e6fda5ac
Fix broken scrolling on non-map pages
Remove overly aggressive CSS overrides that set height: auto and
overflow: auto on html/body. These forced the html element to expand
to full content height, preventing the browser's native viewport
scrolling. The browser handles non-map page scrolling correctly by
default — only map pages need explicit overflow: hidden.
2026-02-19 15:08:16 -06:00
bb6c24856d
Add RemoteIp plug to show real client IP in K8s logs
Extracts client IP from CF-Connecting-IP or X-Forwarded-For headers
and sets conn.remote_ip before request logging, so K8s logs show the
actual client address instead of the internal cluster/ingress IP.
2026-02-19 15:06:41 -06:00
b5d7cf4a38
Replace ip-api.com geolocation with Cloudflare headers
Use CF-IPLatitude/CF-IPLongitude headers instead of making HTTP calls
to ip-api.com on every page load. Eliminates ~500ms latency, external
API dependency, and flaky tests.

Also disable test server to prevent port conflicts during development.
2026-02-19 15:01:49 -06:00
871620223d
Performance improvements, code cleanup, remove sobelow
- Switch PacketProducer buffer from list to :queue for O(1) overflow handling
- Add cached leadership check via :persistent_term to eliminate GenServer.call
  bottleneck in packet distribution hot path
- Simplify packets_live coordinate extraction from 70 lines of nested
  conditionals to helper functions (extract_coordinate/2, format_coordinate/1)
- Remove debug logging from hot paths (packets.ex queries, PacketDistributor,
  app.js console.logs)
- Remove sobelow dependency (false positives blocking commits)
- Add 25 new tests for PacketProducer, coordinate helpers, and cached leadership
2026-02-19 14:51:01 -06:00
509f271eac
Improve packet pipeline reliability, security, and cleanup efficiency
- Fix XSS vulnerability in buildPopupContent fallback (escape callsign/comment)
- Add batch insert retry with individual fallback in PacketConsumer
- Add APRS-IS login response validation (logresp dispatch handler)
- Fix stale generation check bypass in HistoricalLoader
- Replace two-step SELECT+DELETE cleanup with single CTE-based batch DELETE
- Change default packet retention from 365 to 7 days
- Delete MapHelpers (100% duplicate of CoordinateUtils + BoundsUtils)
- Delete AprsIsConnection (dead code, second unused APRS-IS connection)
- Fix InfoMap hook memory leak (store/cancel setTimeout ref)
- Extract singleton loadMapBundle with callback queue in app.js
- Extract @heat_map_max_zoom constant in DisplayManager
- Add telemetry to cleanup worker and PacketProducer buffer overflow
- Track PacketProducer buffer_size as integer for O(1) length checks
- Remove duplicate bounds functions from index.ex
2026-02-19 14:11:48 -06:00
a7d934cb5b
Fix APRS-IS connection, packet pipeline, and cleanup reliability
- Handle login send failure in AprsIsConnection instead of crashing
- Close leaked sockets on login failure in Aprsme.Is (init + reconnect)
- Keep rescheduling keepalive timer when disconnected to prevent permanent loss
- Use supervised BroadcastTaskSupervisor instead of unsupervised Task.start
  for packet broadcasting
- Remove racy Process.alive? check in StreamingPacketsPubSub; send/2 to
  dead PIDs is a no-op, :DOWN monitors handle cleanup
- Add dateline wrapping to SpatialPubSub.point_in_bounds? matching
  StreamingPacketsPubSub behavior
- Add bad_packets table cleanup to PacketCleanupWorker
- Add TODO.md with remaining improvements
2026-02-19 13:34:10 -06:00
b4f85bf12c
Fix callsign registration to be case-insensitive
Users could register duplicate callsigns with different casing (e.g.,
"w1aw" and "W1AW") because the database unique index was case-sensitive.
Additionally, live validation showed false "already taken" errors on
every keystroke.

Changes:
- Add case-insensitive unique index using lower(callsign)
- Skip callsign uniqueness validation during live form changes
- Add test verifying case-insensitive uniqueness enforcement
2026-02-19 13:20:07 -06:00
f6730b282a
Fix API documentation to match implementation
- REST API: Fix id type from integer to UUID, add device/equipment
  fields, correct rate limiting info (100 req/min, not "none"),
  add 429 status code
- Mobile API: Fix search_callsign response field (lon not lng),
  add path to packet fields, correct rate limiting/timeout docs,
  clarify update_bounds behavior
- Fix flaky pipeline tests (async race on global Application config)
2026-02-19 10:22:10 -06:00
4de48294c6
Remove DaisyUI, migrate to pure Tailwind CSS with TailwindUI patterns
Replace all DaisyUI component classes with pure Tailwind CSS utility
classes following TailwindUI conventions. Use dark: variants for dark
mode instead of DaisyUI's data-theme semantic color system.

- Remove DaisyUI plugin and theme vendor files
- Configure Tailwind v4 custom dark variant for data-theme attribute
- Migrate all components: cards, tables, badges, buttons, forms,
  navigation, dropdowns, modals, alerts, star ratings, spinners
- Update all auth pages (login, register, settings, password reset,
  confirmation) to TailwindUI card layout
- Update content pages (about, API docs, status, packets, bad packets,
  info, weather) to TailwindUI patterns
- Replace opacity-based text styling with semantic gray colors
- Update test assertions to match new class names
2026-02-19 10:07:50 -06:00
aca0571a3e
Replace JS hooks with LiveView patterns for map page styling, slideover, and theme switching
- Remove BodyClassHook: use CSS :has() selectors on data-map-page attribute
  instead of JS toggling body.map-page class
- Remove ResponsiveSlideoverHook: pass viewport_width via LiveSocket connect
  params and determine initial slideover state server-side
- Convert theme selector from data-set-theme attributes with DOMContentLoaded
  listeners to JS.dispatch("phx:set-theme") with a single event handler
- Fix buggy reRenderAllCharts that incorrectly used `new MapAPRSMap()` as a
  Map constructor; charts now re-render via themeChanged event directly
2026-02-19 09:26:53 -06:00
f591e15449
Fix registration form showing validation errors before submission
Only display field errors when the changeset has an action set,
preventing "can't be blank" messages from appearing on initial
page load.
2026-02-19 09:20:31 -06:00
68f64f3699
Display APRS object/item names instead of sender callsign
APRS object packets (e.g. DAPNET transmitters) were showing the
sender's callsign on the map instead of the object name. For example,
DB0SDA (Germany) sending an object for P-K5SGD (Texas) would display
as "DB0SDA" at the Texas coordinates.

- Add display_name/1 to PacketUtils that returns object_name for
  objects, item_name for items, falling back to sender
- Update DataBuilder to use display_name in all callsign display paths
- Fix object/item detection order in packet_consumer to prevent
  objects from being incorrectly flagged as items
- Add DataBuilder tests for object/item display name behavior
- Fix flaky PacketPipelineSupervisor test (ensure module loaded)
2026-02-19 09:13:03 -06:00
d803d070ac
Add other SSIDs list to map sidebar, fix context/web boundary
- Add Packets.get_other_ssids/1 to query other SSIDs for a base callsign
- Show other SSIDs in MapLive sidebar with track and info link buttons
- Clicking an SSID tracks it on the map (zooms, shows marker, updates URL)
- Refactor InfoLive.Show to use shared Packets.get_other_ssids/1
- Remove web-layer dependency from Packets context (no more
  AprsmeWeb.TimeHelpers calls from data layer)
- Return raw received_at DateTime instead of pre-formatted timestamp maps
- Format timestamps in the view layer where they belong
2026-02-18 18:15:02 -06:00