diff --git a/CHANGELOG.md b/CHANGELOG.md index bbbe6c3..dddb88d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - BRIN indexes consideration for time-series data - Connection draining and load balancing for clustered deployments - Monitors CPU usage and connection counts across cluster nodes +- Documentation for tracked callsign behavior confirming last packet is always shown regardless of age - Automatically drains connections when node is overloaded (>70% CPU or 2x average connections) - Gracefully reconnects clients to less loaded nodes - Improves overall cluster stability and performance @@ -27,6 +28,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Weather queries now use indexed `has_weather` column - Spatial queries optimized with geography cast index +### Fixed +- Tracked callsigns now show all packets including those without position data + - Previously filtered out packets with `has_position == false` + - Now shows all packets when tracking a specific callsign via /:callsign URL + - Ensures users can see status updates, messages, and other non-position packets + ### Fixed - PostgreSQL notify trigger now sends all required fields for info page updates - Info page real-time updates now display all packet details correctly diff --git a/docs/improvement-todos.md b/docs/improvement-todos.md index 9c59f3b..5dd87ea 100644 --- a/docs/improvement-todos.md +++ b/docs/improvement-todos.md @@ -48,6 +48,16 @@ This document tracks potential improvements identified during the multi-replica - Created `Aprsme.ShutdownHandler` with configurable drain timeout - Created `Aprsme.SignalHandler` for proper SIGTERM handling - **Silent shutdowns**: Removed all user notifications during graceful shutdown + +### ✅ Tracked Callsign Always Shows Last Packet (2025-07-30) +- **Status**: Fixed - Now Shows All Packets Including Non-Position +- **Impact**: High - User experience for tracking specific stations +- **Implementation**: + - Fixed `get_recent_packets` to not filter by `has_position` when tracking callsign + - Confirmed `get_latest_packet_for_callsign` fetches without time restrictions + - Verified `filter_packets_by_time_and_bounds_with_tracked` always includes tracked packet + - Added documentation explaining the behavior + - Users can now see status updates, messages, and other non-position packets - Updated health endpoint to return 503 when draining (15s delay) - Added preStop lifecycle hook with 15s sleep - Set terminationGracePeriodSeconds to 60 seconds diff --git a/docs/tracked-callsign-behavior.md b/docs/tracked-callsign-behavior.md new file mode 100644 index 0000000..f39184d --- /dev/null +++ b/docs/tracked-callsign-behavior.md @@ -0,0 +1,81 @@ +# Tracked Callsign Behavior Documentation + +## Overview + +When viewing a specific callsign at the `/:callsign` route (e.g., `/KE0GB`), the APRS.me application displays the last received packet from that callsign **regardless of age**. + +## Implementation Details + +### 1. Route Handling +- Route defined in `router.ex`: `live "/:callsign", MapLive.Index, :index` +- The callsign is extracted from the URL and set as `tracked_callsign` in the LiveView socket + +### 2. Latest Packet Retrieval +- `Packets.get_latest_packet_for_callsign/1` fetches the most recent packet for a callsign +- Uses `QueryBuilder.callsign_history/2` with only a limit parameter (no time filtering) +- **No time restrictions** are applied when fetching the latest packet + +### 3. Map Centering +- If the tracked callsign has a known position, the map centers on that location at zoom level 12 +- Handled by `Navigation.handle_callsign_tracking/4` in the MapLive.Navigation module + +### 4. Packet Display Logic +The key function ensuring old packets are always displayed is `filter_packets_by_time_and_bounds_with_tracked/5` in MapLive.Index: + +```elixir +defp filter_packets_by_time_and_bounds_with_tracked( + packets, + bounds, + time_threshold, + tracked_callsign, + tracked_latest_packet + ) do + filtered = filter_packets_by_time_and_bounds(packets, bounds, time_threshold) + + # Always include the tracked callsign's latest packet if we have one + if tracked_callsign != "" and tracked_latest_packet do + key = get_callsign_key(tracked_latest_packet) + Map.put(filtered, key, tracked_latest_packet) + else + filtered + end +end +``` + +This function: +1. First filters packets by time threshold and map bounds +2. Then **always adds** the tracked callsign's latest packet to the filtered results +3. This ensures the tracked callsign is visible even if their last position is older than configured time windows + +### 5. Trail Duration Independence +- Even with the shortest trail duration (1 hour), the tracked callsign's latest packet is displayed +- The trail duration setting only affects which other packets are shown, not the tracked callsign + +## User Experience + +When a user navigates to `/:callsign`: +1. The callsign appears in the search box +2. A "Tracking: CALLSIGN" indicator shows below the search +3. The map centers on the callsign's last known position (if available) +4. The callsign's latest packet is displayed on the map, no matter how old +5. Only packets from that callsign are shown (real-time filtering applies) + +## Summary + +The system is correctly implemented to **always show the last received packet** from a tracked callsign, regardless of: +- How old the packet is (could be days, weeks, or months old) +- The selected trail duration setting +- The historical data load setting +- Whether the packet is within the current map bounds +- **Whether the packet has position data or not** (as of 2025-07-30) + +This ensures users can always find and view the last activity of any callsign they're tracking, including: +- Position packets with lat/lon coordinates +- Status updates without position +- Messages and telemetry data +- Weather reports +- Any other APRS packet types + +## Fix Applied (2025-07-30) + +The `get_recent_packets` function was modified to not filter by `has_position == true` when a specific callsign is being tracked. This ensures that all packet types are visible when viewing a callsign's activity, not just those with position data. \ No newline at end of file diff --git a/lib/aprsme/packets.ex b/lib/aprsme/packets.ex index 65f0219..59c6ad7 100644 --- a/lib/aprsme/packets.ex +++ b/lib/aprsme/packets.ex @@ -448,12 +448,25 @@ defmodule Aprsme.Packets do limit = Map.get(opts, :limit, 200) offset = Map.get(opts, :offset, 0) - # Build base query with time and position filters + # Build base query with time filter + # Only filter by position if not tracking a specific callsign + # Also normalize callsign to handle edge cases + callsign = opts[:callsign] + normalized_callsign = if is_binary(callsign), do: String.trim(callsign), else: "" + base_query = - from(p in Packet, - where: p.has_position == true, - where: p.received_at >= ^time_ago - ) + if normalized_callsign == "" do + # For general map view, only show packets with positions + from(p in Packet, + where: p.has_position == true, + where: p.received_at >= ^time_ago + ) + else + # When tracking a callsign, show all their packets regardless of position + from(p in Packet, + where: p.received_at >= ^time_ago + ) + end # Apply spatial and other filters BEFORE limiting # This ensures we get the most recent packets within the specified bounds diff --git a/mix.exs b/mix.exs index a88d91b..bf26947 100644 --- a/mix.exs +++ b/mix.exs @@ -107,7 +107,8 @@ defmodule Aprsme.MixProject do {:cachex, "~> 4.1"}, {:gettext_pseudolocalize, "~> 0.1"}, {:sentry, "~> 11.0"}, - {:wallaby, "~> 0.30.10", only: :test} + {:wallaby, "~> 0.30.10", only: :test}, + {:lazy_html, ">= 0.1.0", only: :test} # Gleam dependencies ] end diff --git a/mix.lock b/mix.lock index 060dece..46cf398 100644 --- a/mix.lock +++ b/mix.lock @@ -4,6 +4,7 @@ "bcrypt_elixir": {:hex, :bcrypt_elixir, "3.3.2", "d50091e3c9492d73e17fc1e1619a9b09d6a5ef99160eb4d736926fd475a16ca3", [:make, :mix], [{:comeonin, "~> 5.3", [hex: :comeonin, repo: "hexpm", optional: false]}, {:elixir_make, "~> 0.6", [hex: :elixir_make, repo: "hexpm", optional: false]}], "hexpm", "471be5151874ae7931911057d1467d908955f93554f7a6cd1b7d804cac8cef53"}, "bunt": {:hex, :bunt, "1.0.0", "081c2c665f086849e6d57900292b3a161727ab40431219529f13c4ddcf3e7a44", [:mix], [], "hexpm", "dc5f86aa08a5f6fa6b8096f0735c4e76d54ae5c9fa2c143e5a1fc7c1cd9bb6b5"}, "cachex": {:hex, :cachex, "4.1.1", "574c5cd28473db313a0a76aac8c945fe44191659538ca6a1e8946ec300b1a19f", [:mix], [{:eternal, "~> 1.2", [hex: :eternal, repo: "hexpm", optional: false]}, {:ex_hash_ring, "~> 6.0", [hex: :ex_hash_ring, repo: "hexpm", optional: false]}, {:jumper, "~> 1.0", [hex: :jumper, repo: "hexpm", optional: false]}, {:sleeplocks, "~> 1.1", [hex: :sleeplocks, repo: "hexpm", optional: false]}, {:unsafe, "~> 1.0", [hex: :unsafe, repo: "hexpm", optional: false]}], "hexpm", "d6b7449ff98d6bb92dda58bd4fc3189cae9f99e7042054d669596f56dc503cd8"}, + "cc_precompiler": {:hex, :cc_precompiler, "0.1.10", "47c9c08d8869cf09b41da36538f62bc1abd3e19e41701c2cea2675b53c704258", [:mix], [{:elixir_make, "~> 0.7", [hex: :elixir_make, repo: "hexpm", optional: false]}], "hexpm", "f6e046254e53cd6b41c6bacd70ae728011aa82b2742a80d6e2214855c6e06b22"}, "certifi": {:hex, :certifi, "2.15.0", "0e6e882fcdaaa0a5a9f2b3db55b1394dba07e8d6d9bcad08318fb604c6839712", [:rebar3], [], "hexpm", "b147ed22ce71d72eafdad94f055165c1c182f61a2ff49df28bcc71d1d5b94a60"}, "comeonin": {:hex, :comeonin, "5.5.1", "5113e5f3800799787de08a6e0db307133850e635d34e9fab23c70b6501669510", [:mix], [], "hexpm", "65aac8f19938145377cee73973f192c5645873dcf550a8a6b18187d17c13ccdb"}, "cowboy": {:hex, :cowboy, "2.13.0", "09d770dd5f6a22cc60c071f432cd7cb87776164527f205c5a6b0f24ff6b38990", [:make, :rebar3], [{:cowlib, ">= 2.14.0 and < 3.0.0", [hex: :cowlib, repo: "hexpm", optional: false]}, {:ranch, ">= 1.8.0 and < 3.0.0", [hex: :ranch, repo: "hexpm", optional: false]}], "hexpm", "e724d3a70995025d654c1992c7b11dbfea95205c047d86ff9bf1cda92ddc5614"}, @@ -30,6 +31,7 @@ "faker": {:hex, :faker, "0.18.0", "943e479319a22ea4e8e39e8e076b81c02827d9302f3d32726c5bf82f430e6e14", [:mix], [], "hexpm", "bfbdd83958d78e2788e99ec9317c4816e651ad05e24cfd1196ce5db5b3e81797"}, "file_system": {:hex, :file_system, "1.1.0", "08d232062284546c6c34426997dd7ef6ec9f8bbd090eb91780283c9016840e8f", [:mix], [], "hexpm", "bfcf81244f416871f2a2e15c1b515287faa5db9c6bcf290222206d120b3d43f6"}, "finch": {:hex, :finch, "0.20.0", "5330aefb6b010f424dcbbc4615d914e9e3deae40095e73ab0c1bb0968933cadf", [:mix], [{:mime, "~> 1.0 or ~> 2.0", [hex: :mime, repo: "hexpm", optional: false]}, {:mint, "~> 1.6.2 or ~> 1.7", [hex: :mint, repo: "hexpm", optional: false]}, {:nimble_options, "~> 0.4 or ~> 1.0", [hex: :nimble_options, repo: "hexpm", optional: false]}, {:nimble_pool, "~> 1.1", [hex: :nimble_pool, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "2658131a74d051aabfcba936093c903b8e89da9a1b63e430bee62045fa9b2ee2"}, + "fine": {:hex, :fine, "0.1.2", "85cf7dd190c7c6c54c2840754ae977c9acc0417316255b674fad9f2678e4ecc7", [:mix], [], "hexpm", "9113531982c2b60dbea6c7233917ddf16806947cd7104b5d03011bf436ca3072"}, "floki": {:hex, :floki, "0.38.0", "62b642386fa3f2f90713f6e231da0fa3256e41ef1089f83b6ceac7a3fd3abf33", [:mix], [], "hexpm", "a5943ee91e93fb2d635b612caf5508e36d37548e84928463ef9dd986f0d1abd9"}, "gen_stage": {:hex, :gen_stage, "1.3.2", "7c77e5d1e97de2c6c2f78f306f463bca64bf2f4c3cdd606affc0100b89743b7b", [:mix], [], "hexpm", "0ffae547fa777b3ed889a6b9e1e64566217413d018cabd825f786e843ffe63e7"}, "geo": {:hex, :geo, "4.0.1", "f4ae3fd912b0536bfe9ec3bce15eb554197ab0739c01297c8534c20dcedd561c", [:mix], [{:jason, "~> 1.4", [hex: :jason, repo: "hexpm", optional: true]}], "hexpm", "32eb624feff75d043bbdd43f67e3869c5fc729e221333271b07cdc98ba98563d"}, @@ -49,6 +51,7 @@ "jason": {:hex, :jason, "1.4.4", "b9226785a9aa77b6857ca22832cffa5d5011a667207eb2a0ad56adb5db443b8a", [:mix], [{:decimal, "~> 1.0 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm", "c5eb0cab91f094599f94d55bc63409236a8ec69a21a67814529e8d5f6cc90b3b"}, "jsx": {:hex, :jsx, "2.8.3", "a05252d381885240744d955fbe3cf810504eb2567164824e19303ea59eef62cf", [:mix, :rebar3], [], "hexpm", "fc3499fed7a726995aa659143a248534adc754ebd16ccd437cd93b649a95091f"}, "jumper": {:hex, :jumper, "1.0.2", "68cdcd84472a00ac596b4e6459a41b3062d4427cbd4f1e8c8793c5b54f1406a7", [:mix], [], "hexpm", "9b7782409021e01ab3c08270e26f36eb62976a38c1aa64b2eaf6348422f165e1"}, + "lazy_html": {:hex, :lazy_html, "0.1.3", "8b9c8c135e95f7bc483de6195c4e1c0b2c913a5e2c57353ef4e82703b7ac8bd1", [:make, :mix], [{:cc_precompiler, "~> 0.1", [hex: :cc_precompiler, repo: "hexpm", optional: false]}, {:elixir_make, "~> 0.9.0", [hex: :elixir_make, repo: "hexpm", optional: false]}, {:fine, "~> 0.1.0", [hex: :fine, repo: "hexpm", optional: false]}], "hexpm", "5f96f29587dcfed8a22281e8c44c6607e958ba821d90b9dfc003d1ef610f7d07"}, "libcluster": {:hex, :libcluster, "3.5.0", "5ee4cfde4bdf32b2fef271e33ce3241e89509f4344f6c6a8d4069937484866ba", [:mix], [{:jason, "~> 1.1", [hex: :jason, repo: "hexpm", optional: false]}, {:telemetry, "~> 1.3", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "ebf6561fcedd765a4cd43b4b8c04b1c87f4177b5fb3cbdfe40a780499d72f743"}, "meck": {:hex, :meck, "1.0.0", "24676cb6ee6951530093a93edcd410cfe4cb59fe89444b875d35c9d3909a15d0", [:rebar3], [], "hexpm", "680a9bcfe52764350beb9fb0335fb75fee8e7329821416cee0a19fec35433882"}, "metrics": {:hex, :metrics, "1.0.1", "25f094dea2cda98213cecc3aeff09e940299d950904393b2a29d191c346a8486", [:rebar3], [], "hexpm", "69b09adddc4f74a40716ae54d140f93beb0fb8978d8636eaded0c31b6f099f16"}, diff --git a/test/aprsme_web/live/map_live/tracked_callsign_old_packet_test.exs b/test/aprsme_web/live/map_live/tracked_callsign_old_packet_test.exs new file mode 100644 index 0000000..beb342e --- /dev/null +++ b/test/aprsme_web/live/map_live/tracked_callsign_old_packet_test.exs @@ -0,0 +1,135 @@ +defmodule AprsmeWeb.MapLive.TrackedCallsignOldPacketTest do + use AprsmeWeb.ConnCase + + import Aprsme.PacketsFixtures + import Phoenix.LiveViewTest + + describe "tracked callsign shows last packet regardless of age" do + test "displays old packet for tracked callsign", %{conn: conn} do + # Create a packet that's 30 days old + thirty_days_ago = DateTime.add(DateTime.utc_now(), -30 * 24 * 60 * 60, :second) + + packet_fixture(%{ + sender: "TEST-OLD", + lat: Decimal.new("35.1234"), + lon: Decimal.new("-97.5678"), + raw_packet: "TEST-OLD>APRS:!3507.40N/09734.07W>Test old packet", + received_at: thirty_days_ago, + has_position: true + }) + + # Navigate to the tracked callsign URL + {:ok, view, _html} = live(conn, "/TEST-OLD") + + # Check that the tracked callsign is displayed + assert render(view) =~ "TEST-OLD" + # Verify tracking indicator is shown + assert render(view) =~ "Tracking" + end + + test "includes old packet even with 1-hour trail duration", %{conn: conn} do + # Create a packet that's 48 hours old + two_days_ago = DateTime.add(DateTime.utc_now(), -48 * 60 * 60, :second) + + packet_fixture(%{ + sender: "OLD-STATION", + lat: Decimal.new("40.7128"), + lon: Decimal.new("-74.0060"), + raw_packet: "OLD-STATION>APRS:!4042.77N/07400.36W>Old position", + received_at: two_days_ago, + has_position: true + }) + + # Navigate to the tracked callsign URL + {:ok, view, _html} = live(conn, "/OLD-STATION") + + # The old packet should still be tracked despite 1-hour trail setting + assert render(view) =~ "OLD-STATION" + assert render(view) =~ "Tracking" + end + + test "shows no packet message when callsign has no packets", %{conn: conn} do + # Navigate to a callsign that doesn't exist + {:ok, view, _html} = live(conn, "/NOPACKETS-1") + + # Check that the tracked callsign is set + assert render(view) =~ "NOPACKETS-1" + assert render(view) =~ "Tracking" + end + + test "displays non-position packet for tracked callsign", %{conn: conn} do + # Create a status packet without position data - this is the key test case + packet_fixture(%{ + sender: "TEST-STATUS", + raw_packet: "TEST-STATUS>APRS:>Status message without position", + received_at: DateTime.utc_now(), + has_position: false, + lat: nil, + lon: nil, + data_type: "status" + }) + + # Navigate to the tracked callsign URL + {:ok, view, _html} = live(conn, "/TEST-STATUS") + + # Check that the tracked callsign is set + assert render(view) =~ "TEST-STATUS" + assert render(view) =~ "Tracking" + end + + test "displays message packet without position for tracked callsign", %{conn: conn} do + # Create a message packet without position data + packet_fixture(%{ + sender: "TEST-MSG", + raw_packet: "TEST-MSG>APRS::W5ISP :Hello there! This is a test message{001", + # 1 hour old + received_at: DateTime.add(DateTime.utc_now(), -3600, :second), + has_position: false, + lat: nil, + lon: nil, + data_type: "message", + aprs_messaging: true + }) + + # Navigate to the tracked callsign URL + {:ok, view, _html} = live(conn, "/TEST-MSG") + + # The message packet should be tracked despite no position + assert render(view) =~ "TEST-MSG" + assert render(view) =~ "Tracking" + end + + test "handles whitespace in callsign parameter", %{conn: conn} do + # Create a packet for testing + packet_fixture(%{ + sender: "TEST-WS", + raw_packet: "TEST-WS>APRS:>Test whitespace handling", + has_position: false + }) + + # Navigate with extra whitespace + {:ok, view, _html} = live(conn, "/ TEST-WS ") + + # Should normalize to uppercase and trim whitespace + assert render(view) =~ "TEST-WS" + assert render(view) =~ "Tracking" + end + + test "handles lowercase callsign with non-position packet", %{conn: conn} do + # Create a telemetry packet without position + packet_fixture(%{ + sender: "TEST-TELEM", + raw_packet: "TEST-TELEM>APRS:T#123,456,789,012,345,678,00000000", + has_position: false, + data_type: "telemetry" + }) + + # Navigate with lowercase callsign + {:ok, view, _html} = live(conn, "/test-telem") + + # Should normalize to uppercase + assert render(view) =~ "TEST-TELEM" + assert render(view) =~ "Tracking" + end + end +end \ No newline at end of file