updates with parser updates
This commit is contained in:
parent
4e00c3cf5d
commit
9b207e0d67
17 changed files with 317 additions and 324 deletions
|
|
@ -1,84 +0,0 @@
|
|||
# Claude Session Summary - August 2, 2025
|
||||
|
||||
## Session Overview
|
||||
This session focused on updating the APRS.me codebase to handle improved APRS parser features based on updates from the vendor/aprs library.
|
||||
|
||||
## Completed Tasks
|
||||
|
||||
### 1. Enhanced APRS Parser Integration
|
||||
- **Initial Request**: "the parser has been improved again with these updates, please update the code to handle them"
|
||||
- **Parser Improvements Implemented**:
|
||||
- Added 11 new standard parser fields for better compatibility
|
||||
- Implemented comprehensive weather data extraction with dedicated wx field
|
||||
- Fixed PHG parsing to return string format instead of map structure
|
||||
- Added radiorange (RNG) field parsing from comments
|
||||
- Enhanced comment processing with proper data extraction and cleaning
|
||||
- Improved compressed position parsing with APRS messaging capability
|
||||
|
||||
### 2. Database Schema Updates
|
||||
- Created migration `20250801231447_add_enhanced_parser_fields.exs`
|
||||
- Added 20 new fields to packets table:
|
||||
- Standard parser fields: srccallsign, dstcallsign, body, origpacket, header, alive, posambiguity, symboltable, symbolcode, messaging
|
||||
- Radio range field: radiorange
|
||||
- Weather fields: rain_midnight, has_weather (some already existed)
|
||||
|
||||
### 3. Code Updates in lib/aprsme/packet.ex
|
||||
- Added `put_standard_parser_fields/2` function to extract new parser compatibility fields
|
||||
- Added `put_radio_range_field/2` function for radiorange extraction
|
||||
- Enhanced `extract_weather_data/2` to prioritize new wx field from improved parser
|
||||
- Updated `put_phg_fields/2` to handle both string format ("1060") and legacy map structure
|
||||
- Added `parse_phg_string/2` for parsing 4-character PHG strings
|
||||
|
||||
### 4. Comprehensive Testing
|
||||
- Created `test/aprsme/enhanced_parser_test.exs` with 6 tests
|
||||
- Tests cover:
|
||||
- Standard parser field extraction
|
||||
- Radio range field extraction
|
||||
- Weather data from wx field
|
||||
- PHG data in both string and map formats
|
||||
- Full packet storage with enhanced fields
|
||||
- All 467 tests passing with 0 failures
|
||||
|
||||
### 5. Previous Work in Session
|
||||
Before the parser update task, the session included:
|
||||
- Improving test coverage from 41.62% to meet 90% threshold
|
||||
- Writing comprehensive test suites for Aprsme.Packets, Cluster.LeaderElection, and Cluster.ConnectionManager modules
|
||||
- Fixing production issues including:
|
||||
- Buffer full errors (actually PacketConsumer crashes)
|
||||
- Telemetry_vals data type mismatches
|
||||
- Missing position_ambiguity column
|
||||
- Optimizing test suite performance (48% speed improvement)
|
||||
- Multiple git commits and pushes throughout
|
||||
|
||||
## Key Technical Details
|
||||
|
||||
### Parser Field Mappings
|
||||
The enhanced parser now provides these additional fields that are mapped to database columns:
|
||||
- `posambiguity` → position_ambiguity (0-4 level indicator)
|
||||
- `wx` → dedicated weather data field (prioritized over other weather fields)
|
||||
- `radiorange` → RNG field from comments (e.g., "RNG0050")
|
||||
- PHG data now comes as string "1060" instead of map structure
|
||||
|
||||
### Backward Compatibility
|
||||
All changes maintain backward compatibility:
|
||||
- Weather extraction checks multiple field locations (wx, weather, weather_report, raw_weather_data)
|
||||
- PHG parsing handles both string and map formats
|
||||
- Standard fields are extracted from top-level attributes if present
|
||||
|
||||
## Current State
|
||||
- All code changes committed and pushed to main branch
|
||||
- Database migration successfully applied
|
||||
- Test suite fully passing
|
||||
- Code formatted and no compilation warnings
|
||||
- Ready for deployment to production
|
||||
|
||||
## Remaining Tasks
|
||||
From the todo list:
|
||||
1. Write tests for Aprsme.Is module (APRS-IS connection) - pending
|
||||
2. Run final coverage report to verify improvements - pending
|
||||
|
||||
## Notes for Next Session
|
||||
- The enhanced parser integration is complete and tested
|
||||
- Consider deploying to production and monitoring for any issues with the new fields
|
||||
- The Aprsme.Is module still needs test coverage
|
||||
- May want to verify that the parser improvements are working correctly with live APRS data
|
||||
|
|
@ -28,6 +28,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||
- Supports ambiguity levels 0-4 as defined in APRS specification
|
||||
- Enhanced APRS parser integration with latest improvements
|
||||
- Added `posresolution` field showing position accuracy in meters (18.52m for uncompressed, 0.291m for compressed)
|
||||
- Upgraded vendor/aprs parser to version 0.1.5+ with comprehensive compatibility improvements
|
||||
- Added standard APRS parser fields: posambiguity, format, alive, symboltable, symbolcode, messaging
|
||||
- Enhanced weather data extraction with dedicated `wx` field
|
||||
- Added radio range (RNG) parsing from comments
|
||||
- Improved PHG data format handling (now returns string representation)
|
||||
- Added telemetry fields (seq, vals, bits) extraction
|
||||
- Better UTF-8 handling and error messages for invalid position data
|
||||
- Fixed compressed latitude calculation and third-party traffic parsing
|
||||
- Added support for alternate compressed position formats
|
||||
- Added 20 new database fields for enhanced parser compatibility
|
||||
- Implemented weather data extraction with dedicated `wx` field support
|
||||
- Enhanced PHG parsing to handle both string format and legacy map structure
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
console.log("app.js loading...");
|
||||
|
||||
// If you want to use Phoenix channels, run `mix help phx.gen.channel`
|
||||
// to get started and then uncomment the line below.
|
||||
// import "./user_socket.js"
|
||||
|
|
@ -124,23 +126,28 @@ let BodyClassHook = {
|
|||
},
|
||||
};
|
||||
|
||||
// APRS Map Hook
|
||||
// APRS MapAPRSMap Hook
|
||||
let Hooks = {};
|
||||
|
||||
// Map hooks - load map bundle when needed
|
||||
// Store original mounted function before creating wrapper
|
||||
const originalMapMounted = MapAPRSMap.mounted;
|
||||
Hooks.APRSMap = {
|
||||
...MapAPRSMap,
|
||||
mounted() {
|
||||
console.log("APRSMap wrapper mounted() called");
|
||||
const self = this;
|
||||
if (window.VendorLoader && !window.mapBundleLoaded) {
|
||||
console.log("Loading map bundle...");
|
||||
// Load map bundle and wait for it to complete
|
||||
const script = document.createElement('script');
|
||||
script.src = window.VendorLoader.mapBundleUrl;
|
||||
script.onload = () => {
|
||||
console.log("Map bundle loaded, calling original mounted");
|
||||
window.mapBundleLoaded = true;
|
||||
// Now call the original mounted function
|
||||
if (MapAPRSMap.mounted) {
|
||||
MapAPRSMap.mounted.call(self);
|
||||
if (originalMapMounted) {
|
||||
originalMapMounted.call(self);
|
||||
}
|
||||
};
|
||||
script.onerror = () => {
|
||||
|
|
@ -148,9 +155,10 @@ Hooks.APRSMap = {
|
|||
};
|
||||
document.head.appendChild(script);
|
||||
} else {
|
||||
console.log("Map bundle already loaded, calling original mounted");
|
||||
// Map bundle already loaded, proceed immediately
|
||||
if (MapAPRSMap.mounted) {
|
||||
MapAPRSMap.mounted.call(this);
|
||||
if (originalMapMounted) {
|
||||
originalMapMounted.call(this);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -176,7 +184,7 @@ Hooks.InfoMap = {
|
|||
};
|
||||
document.head.appendChild(script);
|
||||
} else {
|
||||
// Map bundle already loaded, proceed immediately
|
||||
// MapAPRSMap bundle already loaded, proceed immediately
|
||||
if (InfoMap.mounted) {
|
||||
InfoMap.mounted.call(this);
|
||||
}
|
||||
|
|
@ -263,7 +271,7 @@ window.localStorage.setItem("theme", theme);
|
|||
window.reRenderAllCharts = () => {
|
||||
// Store all chart instances globally so we can access them
|
||||
if (!window.chartInstances) {
|
||||
window.chartInstances = new Map();
|
||||
window.chartInstances = new MapAPRSMap();
|
||||
}
|
||||
|
||||
// Re-render all stored chart instances
|
||||
|
|
@ -310,6 +318,7 @@ document.addEventListener("DOMContentLoaded", () => {
|
|||
});
|
||||
});
|
||||
|
||||
console.log("Creating LiveSocket with hooks:", Object.keys(Hooks));
|
||||
let liveSocket = new LiveSocket("/live", Socket, {
|
||||
longPollFallbackMs: 2500,
|
||||
params: { _csrf_token: csrfToken },
|
||||
|
|
|
|||
|
|
@ -64,6 +64,7 @@ import {
|
|||
let MapAPRSMap = {
|
||||
mounted() {
|
||||
const self = this as unknown as LiveViewHookContext;
|
||||
console.log("APRSMap hook mounted() called on element:", this.el);
|
||||
// Initialize error tracking
|
||||
self.errors = [];
|
||||
self.initializationAttempts = 0;
|
||||
|
|
@ -438,6 +439,7 @@ let MapAPRSMap = {
|
|||
if (self.map) {
|
||||
saveMapState(self.map, self.pushEvent.bind(self));
|
||||
// Also send bounds to server to trigger historical loading
|
||||
console.log("Calling sendBoundsToServer after map_ready (retry)");
|
||||
self.sendBoundsToServer();
|
||||
}
|
||||
// Also trigger map state update after a delay
|
||||
|
|
@ -1059,6 +1061,11 @@ let MapAPRSMap = {
|
|||
self.handleEvent(
|
||||
"add_historical_packets_batch",
|
||||
(data: { packets: MarkerData[]; batch: number; is_final: boolean }) => {
|
||||
console.log("Received add_historical_packets_batch event:", {
|
||||
packetCount: data.packets?.length || 0,
|
||||
batch: data.batch,
|
||||
is_final: data.is_final
|
||||
});
|
||||
try {
|
||||
if (data.packets && Array.isArray(data.packets)) {
|
||||
// Process all packets immediately for maximum speed
|
||||
|
|
@ -1325,6 +1332,7 @@ let MapAPRSMap = {
|
|||
|
||||
sendBoundsToServer() {
|
||||
const self = this as unknown as LiveViewHookContext;
|
||||
console.log("sendBoundsToServer called, map:", !!self.map, "isDestroyed:", self.isDestroyed);
|
||||
if (!self.map || self.isDestroyed) return;
|
||||
|
||||
try {
|
||||
|
|
@ -1351,6 +1359,7 @@ let MapAPRSMap = {
|
|||
},
|
||||
zoom: zoom,
|
||||
};
|
||||
console.log("Sending bounds_changed event:", boundsData);
|
||||
self.pushEvent("bounds_changed", boundsData);
|
||||
}
|
||||
} catch (error) {
|
||||
|
|
|
|||
|
|
@ -435,6 +435,10 @@ defmodule Aprsme.Packets do
|
|||
@impl true
|
||||
@spec get_recent_packets(map()) :: [struct()]
|
||||
def get_recent_packets(opts \\ %{}) do
|
||||
require Logger
|
||||
|
||||
Logger.debug("Packets.get_recent_packets called with opts: #{inspect(opts)}")
|
||||
|
||||
# Use hours_back from opts if provided, otherwise default to 24 hours
|
||||
hours_back = Map.get(opts, :hours_back, 24)
|
||||
time_ago = DateTime.add(DateTime.utc_now(), -hours_back * 3600, :second)
|
||||
|
|
@ -475,7 +479,9 @@ defmodule Aprsme.Packets do
|
|||
|> offset(^offset)
|
||||
|> QueryBuilder.with_coordinates()
|
||||
|
||||
Repo.all(query)
|
||||
result = Repo.all(query)
|
||||
Logger.debug("Packets.get_recent_packets returning #{length(result)} packets")
|
||||
result
|
||||
end
|
||||
|
||||
@doc """
|
||||
|
|
|
|||
|
|
@ -76,7 +76,7 @@
|
|||
</script>
|
||||
|
||||
<!-- App scripts -->
|
||||
<script defer phx-track-static type="text/javascript" src={~p"/assets/app.js"}>
|
||||
<script phx-track-static type="text/javascript" src={~p"/assets/app.js"}>
|
||||
</script>
|
||||
<script>
|
||||
(function(c,u,v,n,p,e,z,A,w){function k(a){if(!x){x=!0;var l=u.getElementsByTagName(v)[0],d=u.createElement(v);d.src=A;d.crossOrigin="anonymous";d.addEventListener("load",function(){try{c[n]=r;c[p]=t;var b=c[e],d=b.init;b.init=function(a){for(var b in a)Object.prototype.hasOwnProperty.call(a,b)&&(w[b]=a[b]);d(w)};B(a,b)}catch(g){console.error(g)}});l.parentNode.insertBefore(d,l)}}function B(a,l){try{for(var d=m.data,b=0;b<a.length;b++)if("function"===typeof a[b])a[b]();var e=!1,g=c.__SENTRY__;"undefined"!==typeof g&&g.hub&&g.hub.getClient()&&(e=!0);g=!1;for(b=0;b<d.length;b++)if(d[b].f){g=!0;var f=d[b];!1===e&&"init"!==f.f&&l.init();e=!0;l[f.f].apply(l,f.a)}!1===e&&!1===g&&l.init();var h=c[n],k=c[p];for(b=0;b<d.length;b++)d[b].e&&h?h.apply(c,d[b].e):d[b].p&&k&&k.apply(c,[d[b].p])}catch(C){console.error(C)}}for(var f=!0,y=!1,q=0;q<document.scripts.length;q++)if(-1<document.scripts[q].src.indexOf(z)){f="no"!==document.scripts[q].getAttribute("data-lazy");break}var x=!1,h=[],m=function(a){(a.e||a.p||a.f&&-1<a.f.indexOf("capture")||a.f&&-1<a.f.indexOf("showReportDialog"))&&f&&k(h);m.data.push(a)};m.data=[];c[e]=c[e]||{};c[e].onLoad=function(a){h.push(a);f&&!y||k(h)};c[e].forceLoad=function(){y=!0;f&&setTimeout(function(){k(h)})};"init addBreadcrumb captureMessage captureException captureEvent configureScope withScope showReportDialog".split(" ").forEach(function(a){c[e][a]=function(){m({f:a,a:arguments})}});var r=c[n];c[n]=function(a,e,d,b,f){m({e:[].slice.call(arguments)});r&&r.apply(c,arguments)};var t=c[p];c[p]=function(a){m({p:a.reason});t&&t.apply(c,arguments)};f||setTimeout(function(){k(h)})})(window,document,"script","onerror","onunhandledrejection","Sentry","be4b53768e7c243cc72fa78ee7b7ec8c","https://js.sentry-cdn.com/be4b53768e7c243cc72fa78ee7b7ec8c.min.js",{"dsn":"https://337ece4c07ff53c6719d900adfddd6e4@o4509627566063616.ingest.us.sentry.io/4509691336785920"});
|
||||
|
|
|
|||
|
|
@ -34,6 +34,8 @@ defmodule AprsmeWeb.MapLive.HistoricalLoader do
|
|||
def start_progressive_historical_loading(socket) do
|
||||
require Logger
|
||||
|
||||
Logger.debug("HistoricalLoader: Starting progressive historical loading")
|
||||
|
||||
Logger.debug(
|
||||
"start_progressive_historical_loading called with zoom: #{socket.assigns.map_zoom}, bounds: #{inspect(socket.assigns.map_bounds)}"
|
||||
)
|
||||
|
|
@ -128,6 +130,8 @@ defmodule AprsmeWeb.MapLive.HistoricalLoader do
|
|||
defp do_load_historical_batch(%{assigns: %{map_bounds: nil}} = socket, _batch_offset), do: socket
|
||||
|
||||
defp do_load_historical_batch(%{assigns: %{map_bounds: bounds}} = socket, batch_offset) do
|
||||
require Logger
|
||||
|
||||
bounds_list = [
|
||||
bounds.west,
|
||||
bounds.south,
|
||||
|
|
@ -172,7 +176,9 @@ defmodule AprsmeWeb.MapLive.HistoricalLoader do
|
|||
params = Map.put(params, :hours_back, historical_hours)
|
||||
|
||||
# Get recent packets within time filter
|
||||
Logger.debug("HistoricalLoader: Querying packets with params: #{inspect(params)}")
|
||||
recent_packets = Packets.get_recent_packets(params)
|
||||
Logger.debug("HistoricalLoader: Got #{length(recent_packets)} packets")
|
||||
|
||||
# If tracking a callsign and this is the first batch, ensure we always include
|
||||
# the most recent packet for that callsign, even if it's older than the time filter
|
||||
|
|
@ -300,6 +306,12 @@ defmodule AprsmeWeb.MapLive.HistoricalLoader do
|
|||
|
||||
# Handle high zoom (markers)
|
||||
defp handle_zoom_based_display(socket, _historical_packets, packet_data_list, is_final_batch, batch_offset) do
|
||||
require Logger
|
||||
|
||||
Logger.debug(
|
||||
"HistoricalLoader: Pushing #{length(packet_data_list)} packets to client, batch #{batch_offset}, is_final: #{is_final_batch}"
|
||||
)
|
||||
|
||||
# Use LiveView's efficient push_event for incremental updates
|
||||
LiveView.push_event(socket, "add_historical_packets_batch", %{
|
||||
packets: packet_data_list,
|
||||
|
|
|
|||
|
|
@ -4,7 +4,6 @@ defmodule AprsmeWeb.MapLive.Index do
|
|||
"""
|
||||
use AprsmeWeb, :live_view
|
||||
|
||||
import AprsmeWeb.Components.ErrorBoundary
|
||||
import AprsmeWeb.Live.Shared.PacketUtils, only: [get_callsign_key: 1]
|
||||
import AprsmeWeb.TimeHelpers, only: [time_ago_in_words: 1]
|
||||
import Phoenix.LiveView, only: [connected?: 1, push_event: 3, push_patch: 2, put_flash: 3]
|
||||
|
|
@ -266,6 +265,9 @@ defmodule AprsmeWeb.MapLive.Index do
|
|||
# Handle both bounds_changed and update_bounds events
|
||||
@impl true
|
||||
def handle_event(event, %{"bounds" => bounds}, socket) when event in ["bounds_changed", "update_bounds"] do
|
||||
require Logger
|
||||
|
||||
Logger.debug("Received #{event} event with bounds: #{inspect(bounds)}")
|
||||
handle_bounds_update(bounds, socket)
|
||||
end
|
||||
|
||||
|
|
@ -362,6 +364,17 @@ defmodule AprsmeWeb.MapLive.Index do
|
|||
# The calculated bounds might be too small/inaccurate
|
||||
Logger.debug("Map ready - waiting for JavaScript to send actual bounds before loading historical packets")
|
||||
|
||||
# If we need initial historical load and have bounds, trigger it
|
||||
socket =
|
||||
if socket.assigns.needs_initial_historical_load and socket.assigns.map_bounds do
|
||||
Logger.debug("Map ready with needs_initial_historical_load=true, triggering historical loading")
|
||||
# Send a message to trigger bounds processing
|
||||
send(self(), {:process_bounds_update, socket.assigns.map_bounds})
|
||||
socket
|
||||
else
|
||||
socket
|
||||
end
|
||||
|
||||
{:noreply, socket}
|
||||
end
|
||||
|
||||
|
|
@ -1257,19 +1270,17 @@ defmodule AprsmeWeb.MapLive.Index do
|
|||
}
|
||||
</style>
|
||||
|
||||
<.error_boundary id="map-error-boundary">
|
||||
<div
|
||||
id="aprs-map"
|
||||
class={if @slideover_open, do: "slideover-open", else: "slideover-closed"}
|
||||
phx-hook="APRSMap"
|
||||
phx-update="ignore"
|
||||
data-center={Jason.encode!(@map_center)}
|
||||
data-zoom={@map_zoom}
|
||||
role="application"
|
||||
aria-label={gettext("APRS packet map showing real-time amateur radio stations")}
|
||||
>
|
||||
</div>
|
||||
</.error_boundary>
|
||||
<div
|
||||
id="aprs-map"
|
||||
class={if @slideover_open, do: "slideover-open", else: "slideover-closed"}
|
||||
phx-hook="APRSMap"
|
||||
phx-update="ignore"
|
||||
data-center={Jason.encode!(@map_center)}
|
||||
data-zoom={@map_zoom}
|
||||
role="application"
|
||||
aria-label={gettext("APRS packet map showing real-time amateur radio stations")}
|
||||
>
|
||||
</div>
|
||||
|
||||
<button class="locate-button" phx-click="locate_me" title={Gettext.gettext(AprsmeWeb.Gettext, "Find my location")}>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="#374151" stroke="none">
|
||||
|
|
@ -1947,6 +1958,11 @@ defmodule AprsmeWeb.MapLive.Index do
|
|||
|
||||
# Load historical packets for the new bounds (now socket.assigns.map_bounds is correct)
|
||||
Logger.debug("Starting progressive historical loading for new bounds")
|
||||
|
||||
Logger.debug(
|
||||
"Current assigns: historical_loading=#{socket.assigns.historical_loading}, map_bounds=#{inspect(socket.assigns.map_bounds)}"
|
||||
)
|
||||
|
||||
socket = HistoricalLoader.start_progressive_historical_loading(socket)
|
||||
|
||||
# Mark initial historical as completed if this was the initial load
|
||||
|
|
|
|||
|
|
@ -111,7 +111,5 @@ defmodule AprsmeWeb.MapLive.PacketBatcher do
|
|||
|
||||
# Send batch to parent LiveView
|
||||
send(parent_pid, {:packet_batch, packets})
|
||||
|
||||
Logger.debug("Processing batch of #{length(packets)} packets")
|
||||
end
|
||||
end
|
||||
|
|
|
|||
|
|
@ -0,0 +1,21 @@
|
|||
defmodule Aprsme.Repo.Migrations.AddParserCompatibilityFields do
|
||||
use Ecto.Migration
|
||||
|
||||
def change do
|
||||
alter table(:packets) do
|
||||
# Position resolution and format fields from enhanced parser
|
||||
add_if_not_exists :posresolution, :float
|
||||
add_if_not_exists :format, :string
|
||||
|
||||
# Additional message field
|
||||
add_if_not_exists :addressee, :string
|
||||
add_if_not_exists :message_text, :string
|
||||
add_if_not_exists :message_number, :string
|
||||
|
||||
# Telemetry fields
|
||||
add_if_not_exists :telemetry_seq, :integer
|
||||
add_if_not_exists :telemetry_vals, {:array, :integer}
|
||||
add_if_not_exists :telemetry_bits, :string
|
||||
end
|
||||
end
|
||||
end
|
||||
|
|
@ -1,16 +0,0 @@
|
|||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
echo "Testing Docker build..."
|
||||
|
||||
# Build the Docker image
|
||||
echo "Building Docker image..."
|
||||
docker build -t aprsme-test:latest .
|
||||
|
||||
echo "Docker build completed successfully!"
|
||||
|
||||
# Test if the image runs
|
||||
echo "Testing if the image can run..."
|
||||
docker run --rm aprsme-test:latest bin/aprsme eval "IO.puts(:ok)"
|
||||
|
||||
echo "All tests passed!"
|
||||
|
|
@ -31,11 +31,17 @@ defmodule Aprsme.Cluster.LeaderElectionTest do
|
|||
assert {:ok, pid} = LeaderElection.start_link([])
|
||||
assert Process.alive?(pid)
|
||||
assert Process.whereis(LeaderElection) == pid
|
||||
|
||||
# Clean up after this specific test
|
||||
GenServer.stop(pid)
|
||||
end
|
||||
|
||||
test "registers with the given name" do
|
||||
{:ok, _pid} = LeaderElection.start_link([])
|
||||
{:ok, pid} = LeaderElection.start_link([])
|
||||
assert Process.whereis(LeaderElection)
|
||||
|
||||
# Clean up after this specific test
|
||||
GenServer.stop(pid)
|
||||
end
|
||||
end
|
||||
|
||||
|
|
|
|||
|
|
@ -12,9 +12,12 @@ defmodule Aprsme.PacketsOldestTest do
|
|||
test "returns the timestamp of the oldest packet" do
|
||||
# Create packets with different timestamps relative to now
|
||||
now = DateTime.utc_now()
|
||||
oldest_time = DateTime.add(now, -365 * 24 * 60 * 60, :second) # 1 year ago
|
||||
middle_time = DateTime.add(now, -180 * 24 * 60 * 60, :second) # 6 months ago
|
||||
newest_time = DateTime.add(now, -30 * 24 * 60 * 60, :second) # 1 month ago
|
||||
# 1 year ago
|
||||
oldest_time = DateTime.add(now, -365 * 24 * 60 * 60, :second)
|
||||
# 6 months ago
|
||||
middle_time = DateTime.add(now, -180 * 24 * 60 * 60, :second)
|
||||
# 1 month ago
|
||||
newest_time = DateTime.add(now, -30 * 24 * 60 * 60, :second)
|
||||
|
||||
# Insert packets with different timestamps
|
||||
{:ok, _} = create_test_packet("OLD-1", oldest_time)
|
||||
|
|
@ -23,9 +26,9 @@ defmodule Aprsme.PacketsOldestTest do
|
|||
|
||||
# Should return the oldest timestamp
|
||||
result = Packets.get_oldest_packet_timestamp()
|
||||
|
||||
# Allow for small time differences due to database precision
|
||||
assert DateTime.diff(result, oldest_time, :second) == 0
|
||||
|
||||
# Allow for small time differences due to database precision (±1 second)
|
||||
assert abs(DateTime.diff(result, oldest_time, :second)) <= 1
|
||||
end
|
||||
|
||||
test "handles packets with microsecond precision" do
|
||||
|
|
@ -33,7 +36,7 @@ defmodule Aprsme.PacketsOldestTest do
|
|||
now = DateTime.utc_now()
|
||||
# Truncate to microseconds to match database precision
|
||||
timestamp_with_microseconds = DateTime.truncate(now, :microsecond)
|
||||
|
||||
|
||||
{:ok, _} = create_test_packet("TEST-1", timestamp_with_microseconds)
|
||||
|
||||
result = Packets.get_oldest_packet_timestamp()
|
||||
|
|
|
|||
|
|
@ -81,7 +81,7 @@ defmodule AprsmeWeb.MapLive.HistoricalLoadingTest do
|
|||
packet_ids = Enum.map(recent_packets, & &1.id)
|
||||
assert packet1.id in packet_ids
|
||||
assert packet2.id in packet_ids
|
||||
refute Enum.any?(packet_ids, fn id -> id == _old_packet.id end)
|
||||
refute _old_packet.id in packet_ids
|
||||
end
|
||||
|
||||
test "loads historical packets with custom historical hours setting", %{conn: conn} do
|
||||
|
|
@ -239,26 +239,27 @@ defmodule AprsmeWeb.MapLive.HistoricalLoadingTest do
|
|||
describe "historical packet push events" do
|
||||
test "sends add_historical_packets_batch events to client", %{conn: conn} do
|
||||
now = DateTime.utc_now()
|
||||
|
||||
|
||||
# Create test packets
|
||||
packets = for i <- 1..5 do
|
||||
packet_fixture(%{
|
||||
sender: "PUSH#{i}",
|
||||
base_callsign: "PUSH#{i}",
|
||||
ssid: "0",
|
||||
lat: 40.7 + (i * 0.01),
|
||||
lon: -74.0 + (i * 0.01),
|
||||
received_at: DateTime.add(now, -(i * 5 * 60), :second),
|
||||
comment: "Push test station #{i}"
|
||||
})
|
||||
end
|
||||
|
||||
packets =
|
||||
for i <- 1..5 do
|
||||
packet_fixture(%{
|
||||
sender: "PUSH#{i}",
|
||||
base_callsign: "PUSH#{i}",
|
||||
ssid: "0",
|
||||
lat: 40.7 + i * 0.01,
|
||||
lon: -74.0 + i * 0.01,
|
||||
received_at: DateTime.add(now, -(i * 5 * 60), :second),
|
||||
comment: "Push test station #{i}"
|
||||
})
|
||||
end
|
||||
|
||||
# Connect to the map at high zoom (should use marker mode, not heat map)
|
||||
{:ok, view, _html} = live(conn, "/?z=12")
|
||||
|
||||
|
||||
# Send map_ready event
|
||||
assert render_hook(view, "map_ready", %{})
|
||||
|
||||
|
||||
# Send bounds that include all test packets
|
||||
bounds = %{
|
||||
"north" => 40.8,
|
||||
|
|
@ -266,29 +267,30 @@ defmodule AprsmeWeb.MapLive.HistoricalLoadingTest do
|
|||
"east" => -73.9,
|
||||
"west" => -74.1
|
||||
}
|
||||
|
||||
|
||||
# Send bounds_changed event
|
||||
assert render_hook(view, "bounds_changed", %{"bounds" => bounds})
|
||||
|
||||
|
||||
# Wait for historical loading
|
||||
Process.sleep(200)
|
||||
|
||||
|
||||
# Verify all packets would be included in the query
|
||||
recent_packets = Packets.get_recent_packets(%{
|
||||
bounds: [bounds["west"], bounds["south"], bounds["east"], bounds["north"]],
|
||||
hours_back: 1,
|
||||
limit: 500
|
||||
})
|
||||
|
||||
recent_packets =
|
||||
Packets.get_recent_packets(%{
|
||||
bounds: [bounds["west"], bounds["south"], bounds["east"], bounds["north"]],
|
||||
hours_back: 1,
|
||||
limit: 500
|
||||
})
|
||||
|
||||
packet_ids = Enum.map(recent_packets, & &1.id)
|
||||
assert length(packet_ids) >= 5
|
||||
|
||||
|
||||
# Verify all our test packets are included
|
||||
for packet <- packets do
|
||||
assert packet.id in packet_ids
|
||||
end
|
||||
end
|
||||
|
||||
|
||||
test "does not send historical packets when bounds are invalid", %{conn: conn} do
|
||||
# Create a test packet
|
||||
packet_fixture(%{
|
||||
|
|
@ -300,13 +302,13 @@ defmodule AprsmeWeb.MapLive.HistoricalLoadingTest do
|
|||
received_at: DateTime.add(DateTime.utc_now(), -30 * 60, :second),
|
||||
comment: "Test for invalid bounds"
|
||||
})
|
||||
|
||||
|
||||
# Connect to the map
|
||||
{:ok, view, _html} = live(conn, "/")
|
||||
|
||||
|
||||
# Send map_ready event
|
||||
assert render_hook(view, "map_ready", %{})
|
||||
|
||||
|
||||
# Send invalid bounds (north < south)
|
||||
invalid_bounds = %{
|
||||
"north" => 40.5,
|
||||
|
|
@ -314,13 +316,13 @@ defmodule AprsmeWeb.MapLive.HistoricalLoadingTest do
|
|||
"east" => -73.7,
|
||||
"west" => -74.3
|
||||
}
|
||||
|
||||
|
||||
# This should not cause an error, but bounds won't be processed
|
||||
assert render_hook(view, "bounds_changed", %{"bounds" => invalid_bounds})
|
||||
|
||||
|
||||
# Wait briefly
|
||||
Process.sleep(100)
|
||||
|
||||
|
||||
# Since we can't check internal state, we verify the view is still functional
|
||||
# by sending valid bounds and checking it still works
|
||||
valid_bounds = %{
|
||||
|
|
@ -329,11 +331,11 @@ defmodule AprsmeWeb.MapLive.HistoricalLoadingTest do
|
|||
"east" => -73.7,
|
||||
"west" => -74.3
|
||||
}
|
||||
|
||||
|
||||
assert render_hook(view, "bounds_changed", %{"bounds" => valid_bounds})
|
||||
end
|
||||
end
|
||||
|
||||
|
||||
describe "progressive historical loading" do
|
||||
test "loads packets in batches for low zoom levels", %{conn: conn} do
|
||||
now = DateTime.utc_now()
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ defmodule AprsmeWeb.TimeHelpersTest do
|
|||
# Use a fixed reference time for all tests to ensure consistency
|
||||
# This prevents tests from failing due to timing issues
|
||||
reference_time = ~U[2024-01-15 12:00:00Z]
|
||||
|
||||
|
||||
# Mock DateTime.utc_now() to return our reference time
|
||||
# Note: Since time_ago_in_words likely uses DateTime.utc_now() internally,
|
||||
# we need to test with actual time differences
|
||||
|
|
|
|||
|
|
@ -6,246 +6,248 @@ defmodule AprsmeWeb.HistoricalLoadingIntegrationTest do
|
|||
use ExUnit.Case, async: false
|
||||
use Wallaby.Feature
|
||||
|
||||
import Aprsme.PacketsFixtures
|
||||
import Wallaby.Browser
|
||||
import Wallaby.Query
|
||||
import Aprsme.PacketsFixtures
|
||||
|
||||
@tag :integration
|
||||
describe "historical packet loading on page load" do
|
||||
test "displays historical packets immediately after map loads", %{session: session} do
|
||||
@describetag :integration
|
||||
feature "displays historical packets immediately after map loads", %{session: session} do
|
||||
# Create test packets with known positions
|
||||
now = DateTime.utc_now()
|
||||
|
||||
packet1 = packet_fixture(%{
|
||||
sender: "INTTEST1",
|
||||
base_callsign: "INTTEST1",
|
||||
ssid: "0",
|
||||
lat: 40.7128,
|
||||
lon: -74.0060,
|
||||
received_at: DateTime.add(now, -20 * 60, :second),
|
||||
comment: "Integration test station 1",
|
||||
symbol_code: "k",
|
||||
symbol_table_id: "/"
|
||||
})
|
||||
|
||||
packet2 = packet_fixture(%{
|
||||
sender: "INTTEST2",
|
||||
base_callsign: "INTTEST2",
|
||||
ssid: "0",
|
||||
lat: 40.7580,
|
||||
lon: -73.9855,
|
||||
received_at: DateTime.add(now, -10 * 60, :second),
|
||||
comment: "Integration test station 2",
|
||||
symbol_code: "-",
|
||||
symbol_table_id: "/"
|
||||
})
|
||||
|
||||
|
||||
_packet1 =
|
||||
packet_fixture(%{
|
||||
sender: "INTTEST1",
|
||||
base_callsign: "INTTEST1",
|
||||
ssid: "0",
|
||||
lat: 40.7128,
|
||||
lon: -74.0060,
|
||||
received_at: DateTime.add(now, -20 * 60, :second),
|
||||
comment: "Integration test station 1",
|
||||
symbol_code: "k",
|
||||
symbol_table_id: "/"
|
||||
})
|
||||
|
||||
_packet2 =
|
||||
packet_fixture(%{
|
||||
sender: "INTTEST2",
|
||||
base_callsign: "INTTEST2",
|
||||
ssid: "0",
|
||||
lat: 40.7580,
|
||||
lon: -73.9855,
|
||||
received_at: DateTime.add(now, -10 * 60, :second),
|
||||
comment: "Integration test station 2",
|
||||
symbol_code: "-",
|
||||
symbol_table_id: "/"
|
||||
})
|
||||
|
||||
# Navigate to the map
|
||||
session
|
||||
|> visit("/")
|
||||
|> assert_has(css("#map", text: ""))
|
||||
|
||||
|
||||
# Wait for map to initialize and markers to appear
|
||||
# The map should automatically load historical packets
|
||||
Process.sleep(3000)
|
||||
|
||||
|
||||
# Check that markers are present on the map
|
||||
# Look for Leaflet marker elements
|
||||
assert_has(session, css(".leaflet-marker-icon"))
|
||||
|
||||
|
||||
# Verify at least 2 markers are present (our test packets)
|
||||
marker_count =
|
||||
marker_count =
|
||||
session
|
||||
|> all(css(".leaflet-marker-icon"))
|
||||
|> length()
|
||||
|
||||
|
||||
assert marker_count >= 2, "Expected at least 2 markers, found #{marker_count}"
|
||||
|
||||
|
||||
# Click on a marker to verify it's one of our test packets
|
||||
session
|
||||
|> click(css(".leaflet-marker-icon", at: 0))
|
||||
|
||||
click(session, css(".leaflet-marker-icon", at: 0))
|
||||
|
||||
# Wait for popup to appear
|
||||
Process.sleep(1000)
|
||||
|
||||
|
||||
# Verify popup contains one of our test callsigns
|
||||
assert_has(session, css(".leaflet-popup-content", text: ~r/INTTEST[12]/))
|
||||
end
|
||||
|
||||
test "loads packets within specified historical time range", %{session: session} do
|
||||
|
||||
feature "loads packets within specified historical time range", %{session: session} do
|
||||
now = DateTime.utc_now()
|
||||
|
||||
|
||||
# Create packets at different times
|
||||
_old_packet = packet_fixture(%{
|
||||
sender: "OLDTEST",
|
||||
base_callsign: "OLDTEST",
|
||||
ssid: "0",
|
||||
lat: 40.7128,
|
||||
lon: -74.0060,
|
||||
received_at: DateTime.add(now, -3 * 60 * 60, :second), # 3 hours ago
|
||||
comment: "Old packet - should not appear with hist=1"
|
||||
})
|
||||
|
||||
recent_packet = packet_fixture(%{
|
||||
sender: "RECENTTEST",
|
||||
base_callsign: "RECENTTEST",
|
||||
ssid: "0",
|
||||
lat: 40.7128,
|
||||
lon: -74.0060,
|
||||
received_at: DateTime.add(now, -30 * 60, :second), # 30 minutes ago
|
||||
comment: "Recent packet - should appear"
|
||||
})
|
||||
|
||||
_old_packet =
|
||||
packet_fixture(%{
|
||||
sender: "OLDTEST",
|
||||
base_callsign: "OLDTEST",
|
||||
ssid: "0",
|
||||
lat: 40.7128,
|
||||
lon: -74.0060,
|
||||
# 3 hours ago
|
||||
received_at: DateTime.add(now, -3 * 60 * 60, :second),
|
||||
comment: "Old packet - should not appear with hist=1"
|
||||
})
|
||||
|
||||
_recent_packet =
|
||||
packet_fixture(%{
|
||||
sender: "RECENTTEST",
|
||||
base_callsign: "RECENTTEST",
|
||||
ssid: "0",
|
||||
lat: 40.7128,
|
||||
lon: -74.0060,
|
||||
# 30 minutes ago
|
||||
received_at: DateTime.add(now, -30 * 60, :second),
|
||||
comment: "Recent packet - should appear"
|
||||
})
|
||||
|
||||
# Navigate to map with 1 hour historical range (default)
|
||||
session
|
||||
|> visit("/?hist=1")
|
||||
|> assert_has(css("#map"))
|
||||
|
||||
|
||||
# Wait for historical loading
|
||||
Process.sleep(3000)
|
||||
|
||||
|
||||
# Click on the marker (should be the recent one)
|
||||
session
|
||||
|> click(css(".leaflet-marker-icon", at: 0))
|
||||
|
||||
click(session, css(".leaflet-marker-icon", at: 0))
|
||||
Process.sleep(1000)
|
||||
|
||||
|
||||
# Verify it's the recent packet, not the old one
|
||||
assert_has(session, css(".leaflet-popup-content", text: "RECENTTEST"))
|
||||
refute_has(session, css(".leaflet-popup-content", text: "OLDTEST"))
|
||||
|
||||
|
||||
# Now test with extended historical range
|
||||
session
|
||||
|> visit("/?hist=6") # 6 hours
|
||||
# 6 hours
|
||||
|> visit("/?hist=6")
|
||||
|> assert_has(css("#map"))
|
||||
|
||||
|
||||
Process.sleep(3000)
|
||||
|
||||
|
||||
# Now both packets should be visible
|
||||
marker_count =
|
||||
marker_count =
|
||||
session
|
||||
|> all(css(".leaflet-marker-icon"))
|
||||
|> length()
|
||||
|
||||
|
||||
assert marker_count >= 2, "Expected at least 2 markers with 6-hour range"
|
||||
end
|
||||
|
||||
test "updates historical packets when bounds change", %{session: session} do
|
||||
|
||||
feature "updates historical packets when bounds change", %{session: session} do
|
||||
now = DateTime.utc_now()
|
||||
|
||||
|
||||
# Create packets in different locations
|
||||
nyc_packet = packet_fixture(%{
|
||||
sender: "NYC1",
|
||||
base_callsign: "NYC1",
|
||||
ssid: "0",
|
||||
lat: 40.7128,
|
||||
lon: -74.0060,
|
||||
received_at: DateTime.add(now, -30 * 60, :second),
|
||||
comment: "NYC packet"
|
||||
})
|
||||
|
||||
la_packet = packet_fixture(%{
|
||||
sender: "LA1",
|
||||
base_callsign: "LA1",
|
||||
ssid: "0",
|
||||
lat: 34.0522,
|
||||
lon: -118.2437,
|
||||
received_at: DateTime.add(now, -30 * 60, :second),
|
||||
comment: "LA packet"
|
||||
})
|
||||
|
||||
_nyc_packet =
|
||||
packet_fixture(%{
|
||||
sender: "NYC1",
|
||||
base_callsign: "NYC1",
|
||||
ssid: "0",
|
||||
lat: 40.7128,
|
||||
lon: -74.0060,
|
||||
received_at: DateTime.add(now, -30 * 60, :second),
|
||||
comment: "NYC packet"
|
||||
})
|
||||
|
||||
_la_packet =
|
||||
packet_fixture(%{
|
||||
sender: "LA1",
|
||||
base_callsign: "LA1",
|
||||
ssid: "0",
|
||||
lat: 34.0522,
|
||||
lon: -118.2437,
|
||||
received_at: DateTime.add(now, -30 * 60, :second),
|
||||
comment: "LA packet"
|
||||
})
|
||||
|
||||
# Start focused on NYC
|
||||
session
|
||||
|> visit("/?lat=40.7128&lng=-74.0060&z=10")
|
||||
|> assert_has(css("#map"))
|
||||
|
||||
|
||||
Process.sleep(3000)
|
||||
|
||||
|
||||
# Should see NYC packet
|
||||
session
|
||||
|> click(css(".leaflet-marker-icon", at: 0))
|
||||
|
||||
click(session, css(".leaflet-marker-icon", at: 0))
|
||||
Process.sleep(1000)
|
||||
|
||||
|
||||
assert_has(session, css(".leaflet-popup-content", text: "NYC1"))
|
||||
|
||||
|
||||
# Close popup
|
||||
session
|
||||
|> send_keys([:escape])
|
||||
|
||||
send_keys(session, [:escape])
|
||||
|
||||
# Pan to LA (this would be done via map interaction in real usage)
|
||||
# For testing, we'll navigate to new URL
|
||||
session
|
||||
|> visit("/?lat=34.0522&lng=-118.2437&z=10")
|
||||
|
||||
visit(session, "/?lat=34.0522&lng=-118.2437&z=10")
|
||||
Process.sleep(3000)
|
||||
|
||||
|
||||
# Should now see LA packet instead
|
||||
session
|
||||
|> click(css(".leaflet-marker-icon", at: 0))
|
||||
|
||||
click(session, css(".leaflet-marker-icon", at: 0))
|
||||
Process.sleep(1000)
|
||||
|
||||
|
||||
assert_has(session, css(".leaflet-popup-content", text: "LA1"))
|
||||
end
|
||||
end
|
||||
|
||||
@tag :integration
|
||||
|
||||
describe "historical loading with tracked callsigns" do
|
||||
test "loads all packets for tracked callsign regardless of bounds", %{session: session} do
|
||||
@describetag :integration
|
||||
feature "loads all packets for tracked callsign regardless of bounds", %{session: session} do
|
||||
now = DateTime.utc_now()
|
||||
|
||||
|
||||
# Create packets for tracked station at different locations
|
||||
packet1 = packet_fixture(%{
|
||||
sender: "TRACK1-9",
|
||||
base_callsign: "TRACK1",
|
||||
ssid: "9",
|
||||
lat: 40.7128,
|
||||
lon: -74.0060,
|
||||
received_at: DateTime.add(now, -45 * 60, :second),
|
||||
comment: "NYC position"
|
||||
})
|
||||
|
||||
packet2 = packet_fixture(%{
|
||||
sender: "TRACK1-9",
|
||||
base_callsign: "TRACK1",
|
||||
ssid: "9",
|
||||
lat: 41.8781,
|
||||
lon: -87.6298,
|
||||
received_at: DateTime.add(now, -30 * 60, :second),
|
||||
comment: "Chicago position"
|
||||
})
|
||||
|
||||
packet3 = packet_fixture(%{
|
||||
sender: "TRACK1-9",
|
||||
base_callsign: "TRACK1",
|
||||
ssid: "9",
|
||||
lat: 34.0522,
|
||||
lon: -118.2437,
|
||||
received_at: DateTime.add(now, -15 * 60, :second),
|
||||
comment: "LA position"
|
||||
})
|
||||
|
||||
_packet1 =
|
||||
packet_fixture(%{
|
||||
sender: "TRACK1-9",
|
||||
base_callsign: "TRACK1",
|
||||
ssid: "9",
|
||||
lat: 40.7128,
|
||||
lon: -74.0060,
|
||||
received_at: DateTime.add(now, -45 * 60, :second),
|
||||
comment: "NYC position"
|
||||
})
|
||||
|
||||
_packet2 =
|
||||
packet_fixture(%{
|
||||
sender: "TRACK1-9",
|
||||
base_callsign: "TRACK1",
|
||||
ssid: "9",
|
||||
lat: 41.8781,
|
||||
lon: -87.6298,
|
||||
received_at: DateTime.add(now, -30 * 60, :second),
|
||||
comment: "Chicago position"
|
||||
})
|
||||
|
||||
_packet3 =
|
||||
packet_fixture(%{
|
||||
sender: "TRACK1-9",
|
||||
base_callsign: "TRACK1",
|
||||
ssid: "9",
|
||||
lat: 34.0522,
|
||||
lon: -118.2437,
|
||||
received_at: DateTime.add(now, -15 * 60, :second),
|
||||
comment: "LA position"
|
||||
})
|
||||
|
||||
# Navigate to tracked callsign URL
|
||||
session
|
||||
|> visit("/TRACK1-9")
|
||||
|> assert_has(css("#map"))
|
||||
|
||||
|
||||
# Wait for map to load and center on latest position
|
||||
Process.sleep(3000)
|
||||
|
||||
|
||||
# Should see trail connecting all positions
|
||||
# Verify we have markers (latest position + trail points)
|
||||
marker_count =
|
||||
marker_count =
|
||||
session
|
||||
|> all(css(".leaflet-marker-icon"))
|
||||
|> length()
|
||||
|
||||
|
||||
# Should have at least 1 marker for current position
|
||||
assert marker_count >= 1, "Expected markers for tracked station"
|
||||
|
||||
|
||||
# Verify polyline trail exists
|
||||
assert_has(session, css(".leaflet-pane .leaflet-overlay-pane polyline"))
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
|
|
|
|||
2
vendor/aprs
vendored
2
vendor/aprs
vendored
|
|
@ -1 +1 @@
|
|||
Subproject commit 7e120409975f8e76af0f1c7b5446978685151a52
|
||||
Subproject commit b072eb7b693e5bc48a24867aeebbc928df877e59
|
||||
Loading…
Add table
Reference in a new issue