Commit graph

1571 commits

Author SHA1 Message Date
7e1498bf4e
perf: batch high-frequency LiveView updates via socket.private
Some checks failed
Build and Push / Build and Push Docker Image (push) Failing after 50s
Buffer incoming packets in socket.private (no rerender) and flush to
assigns/stream on a 150ms timer, preventing client-side morphdom from
being overwhelmed during traffic bursts. Applies to the global packet
feed (stream) and per-callsign detail view (assigns).
2026-07-22 11:23:09 -05:00
Graham McInitre
0e322a2403 docs: add structured documentation covering architecture, contexts, web, infra, and dev setup 2026-07-21 11:01:27 -05:00
Graham McInitre
1006fdaefc refactor: remove dead code, duplication, and over-abstractions
- Delete dead config keys and test file (typo'd aprsme_is_*, vacuous disable test)
- Remove duplicate function clauses and identical function pairs
- Rename misleading one_hour_ago variable to one_day_ago
- Strip stale Oban comment
- Remove TestHelpers time function duplicates
- Inline Aprsme.Schema module (only 1 caller)
- Deduplicate prod esbuild config
- Thin SymbolRenderer delegations, inline TimeUtils wrappers
- Remove redundant DeploymentNotifier GenServer polling
- Replace random fallback in get_callsign_key with sentinel
2026-07-21 10:36:49 -05:00
Graham McInitre
243a98df77 fix: security hardening and query performance improvements
- Session cookie: add encryption_salt and secure flag to endpoint
- CSP: remove unsafe-eval from script-src directive
- /metrics: wrap PromEx endpoint behind rate-limited pipeline
- LiveView signing_salt: move to env var in production config
- LiveView performance: defer heavy aggregation queries past initial mount
- SQL: replace correlated subquery with DISTINCT ON in get_heard_by_stations
2026-07-21 10:13:30 -05:00
Graham McInitre
b88eb9e584 chore: update mix deps (circular_buffer, elixir_make, lazy_html) 2026-07-21 08:55:42 -05:00
Graham McInitre
bb73c17ce6 chore: commit shared .envrc, gitignore .envrc.local for secrets 2026-07-21 08:53:35 -05:00
77d2a6ca30
updat edeps 2026-07-17 16:42:53 -05:00
281aef095c
fix: resolve all credo warnings and pre-existing dialyzer errors
- Move require Logger to module top level in SpatialPubSub
- Replace weak is_list assertions with stronger checks
- Remove redundant 'is a list' test in PromEx tests
- Replace apply/3 with direct function call in packet_replay test
- Fix unmatched_return in weather_cache.ex and callsign_view.ex
2026-07-17 16:41:15 -05:00
7bdf21fb53
deps: ecto_sql 3.13→3.14, prom_ex 1.11→1.12, decimal 2.4→3.1, ecto 3.13→3.14; bump test pool for sandbox 2026-07-08 10:46:05 -05:00
62bbeaadb3
deps: update phoenix 1.8.8→1.8.9, swoosh 1.26.2→1.26.3; cleanup test assertions 2026-07-08 10:28:18 -05:00
088fd20711
refactor: FP patterns in packets_live/callsign_view
- mount: extract subscribe_if_connected/2, split assign_valid/assign_invalid
  to eliminate inline if/else for validation + subscription logic.

- enrich_packet_with_device_identifier: extract resolve_canonical_identifier/1
  as multi-clause function (binary guard + catch-all) instead of nested if.

- update_packet_lists: cond → multi-clause do_update/4 with guards.
  Three explicit paths: at-cap no-stored, at-cap with-stored, under-cap.

- get_timestamp_microseconds: case → multi-clause (%DateTime{}, binary,
  catch-all) for cleaner dispatch.
2026-07-01 18:08:56 -05:00
80983e6223
fix: repair all 60 failing tests after refactor and dep updates
- IS GenServer: add missing :failure_started_at to test build_state/1 map
- IS GenServer: fix stale status server assertion and reconnection test
- IS GenServer: set Logger level to :debug for dispatch parse-error tests
- PacketReplay: remove conflicting Registry start_supervised! from setup
- PacketReplay: update assertion from {:continue_replay} to :start_replay
- Doctests: fix float precision, stale sprite coords, quoting escapes
- API controllers: use string keys for JSON error/postion details
- PacketUtils: fix operator precedence (not is_nil(result).field)
- ThemeManager: update expected dark theme text color
- StatusLive: remove/update stale :loading assign assertions
- Movement: remove {:ok, _v} wrapper from render_hook/3 (returns HTML)
- AprsIsMock: update packet_stats assertion for populated default shape
2026-07-01 09:41:10 -05:00
01ecf3d6bc
refactor: apply functional programming patterns across codebase
- accounts.ex: replace validate_user_password (if-based) with with-chain;
  drop unnecessary try/rescue around Repo.get_by. Extract transaction_unwrap
  helper to eliminate 3 copies of the Multi result-unwrap case block.

- packets.ex: deduplicate store_bad_packet by extracting raw_packet_string,
  extract_error_type, and extract_error_message into composable helpers.
  Normalize find_coordinate_value — collapse 12 clauses into a generic
  deep_get/direct_get that handles atom/string keys and nested maps uniformly.

- data_builder.ex: split build_simple_popup into separate data-construction
  (build_simple_popup_data) and rendering (render_popup) phases, eliminating
  duplicated PopupComponent/Safe/IO pipelines.

- param_utils.ex: unify parse_float_in_range and parse_int_in_range via a
  shared parse_in_range that accepts Float.parse/Integer.parse as a function
  parameter — functions-as-values pattern.
2026-07-01 08:45:19 -05:00
bc60fec98c
deps: update all dependencies and fix doctests for Elixir 1.20
- Update 9 Hex packages (bandit, credo, finch, phoenix, phoenix_live_view,
  plug, req, swoosh, tidewave)
- Load bad_packets data unconditionally in mount/3 so crawlers and link
  previews see content instead of an empty page
- Fix Accounts doctests: use concrete values instead of unbound variables,
  disable illustrative doctests broken by Elixir 1.20.2 strictness
2026-07-01 08:40:17 -05:00
07cb40f0c2
elixir bump 2026-06-23 09:39:57 -05:00
6eda1c0290
fix: resolve 19 bugs across Elixir, JS/TS, and config
Critical:
- Remove :protected from WeatherCache ETS (conflicted with :public)
- Register Aprsme.ReplayRegistry in supervision tree
- Route :continue_replay dead message to :start_replay handler
- Add 5000ms timeout to unbounded :rpc.call calls
- Fix falsy lat/lng check rejecting valid 0 coordinates

Medium:
- Fix live_reload pattern (temp_web -> aprsme_web)
- Remove duplicate ecto_repos config
- Make DB SSL configurable via DB_SSL env var
- Track sizeCheckTimeout on self for cleanup in destroyed()
- Wrap pushEvent calls in try/catch
- Remove unused size variable in marker cluster icon
- Capture touch coords at touchstart instead of stale TouchEvent
- Demote console.log to console.debug in production code
- Add catch-all to normalize_bounds preventing GenServer crash

Style:
- Add missing @impl true annotations in is.ex
- Improve migration failure error message
- Use textContent instead of innerHTML for error messages
- Use proper type for longPressTimer instead of any
2026-06-21 11:47:07 -05:00
b86153cd27
Fix all mix credo --strict warnings (188 → 0)
- Replace apply/2 with direct fully-qualified calls in movement_test
- Fix assert_receive timeouts < 1000ms across 8 test files
- Move nested import statements to module-level scope
- Fix tests with no assertions and add missing doctest
- Replace weak type assertions with specific value checks
- Fix conditional assertions and length/1 expensive patterns
- Disable inappropriate Jump.CredoChecks.AvoidSocketAssignsInTest
- Fix tests not calling application code with credo:disable
- Add various credo:disable comments for legitimate patterns
2026-06-12 16:27:20 -05:00
17838f05d3
Add Plausible Analytics server-side tracking and API Prometheus metrics 2026-06-04 19:37:54 -05:00
0c90a47f9a
update stuff 2026-06-04 18:00:03 -05:00
274af1ce93
Add W5ISP attribution and QRZ link to about page 2026-06-04 17:30:56 -05:00
32fca73d3a
Fix JSON punctuation highlighting inside string values 2026-06-04 17:01:36 -05:00
0a4f921fd9
Format JSON examples with proper indentation and syntax highlighting 2026-06-04 16:53:22 -05:00
2dc42a6cb7
update robots.txt 2026-06-04 16:34:49 -05:00
1ef9e00a05
update robots.txt 2026-06-04 16:33:12 -05:00
f2c8f7020c
update robots.txt 2026-06-04 16:29:53 -05:00
2e0529ee75
update robots.txt 2026-06-04 16:26:13 -05:00
5569d8e2a0
Add user-agent to request logging metadata 2026-06-04 16:19:47 -05:00
6d190bc98e
cleanup todo 2026-06-04 16:14:40 -05:00
c3921e9277
Add Plausible analytics snippet to root layout 2026-06-04 16:05:24 -05:00
1eceb3664d
fix: install libsctp1 to silence Erlang 29 SCTP warning 2026-06-04 15:39:40 -05:00
a4b140beaf
chore: update elixir/erlang versions, remove renovate forgejo task, update deps 2026-06-04 15:33:50 -05:00
e8d48f8058
speed up slow tests: remove Process.sleep, add batch fixture helper
- Replace Process.sleep(N) with explicit synchronization:
  render(view) to force LV mailbox drain, PubSub assert_receive for
  leader election, and render_hook sync guarantees
- Add batch_packet_fixtures/2 using Repo.insert_all for bulk test
  fixtures (200 inserts -> 1 round trip)
- Remove sleeps from historical_loading_test — tests query DB
  directly, don't need LV async processing
2026-06-02 15:02:07 -05:00
73c6703102
perf: debounce BadPacketsLive refresh, remove stale test cases
Coalesce rapid PubSub notifications into single 2-second delayed DB query. Remove tests for removed :update_time_display handler.

Ultraworked with Sisyphus

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-06-01 16:50:28 -05:00
3169abf0f0
perf: convert packets_live table to streams for efficient diffs
Replace full-list assign with stream_insert + phx-update=stream. Add DOM IDs to all packet rows. Skip deprecated phx-update=prepend.

Ultraworked with Sisyphus

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-06-01 16:50:09 -05:00
ff3a478bbd
perf: add LIMIT to unbounded DISTINCT ON, guard device table scan, add DB indexes
Bound batch DISTINCT ON queries by input count. Guard Repo.all(Devices) with exists? check. Add trigram GIN index on base_callsign and (has_weather, received_at DESC) partial index.

Ultraworked with Sisyphus

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-06-01 16:49:48 -05:00
7f814462ed
perf: reduce traversals, optimize packet field lookups, use has_weather boolean
Remove redundant in-memory sort (DB already orders), shortcut get_packet_field when field found at top level, use pre-computed has_weather boolean instead of field-by-field scan

Ultraworked with Sisyphus

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-06-01 16:49:28 -05:00
e02fafc8da
perf: optimize map LiveView hot paths
Remove unused 30s timer, skip heatmap recluster on every single packet, trail dedup O(n^2)->O(n)

Ultraworked with Sisyphus

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-06-01 16:49:08 -05:00
ae1194ebb0
perf: fix PacketBatcher O(n) length/1 to O(1) counter
Ultraworked with Sisyphus

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-06-01 16:48:48 -05:00
21ab9fb20c
perf: add persistent ETS weather cache for N+1 prevention
Ultraworked with Sisyphus

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-06-01 16:48:28 -05:00
3948ae5894
Remove dead packets.html.heex template and create findings.md
The unstyled <h1>Packets</h1> template was dead code - no route in the
router renders it (/packets uses PacketsLive.Index LiveView instead).
Removed the template and its test. Added findings.md with all audit
results from comprehensive security and code quality review.
2026-05-29 15:50:23 -05:00
d0b9513b41
fix elixir type warnings 2026-05-29 14:00:29 -05:00
24796f98d4
Optimize prod: scale deployment, drop expensive metric queries, fix slow callsign and nearby-stations queries
- k8s: 2 replicas, POOL_SIZE 25, cpu 2000m, mem 1Gi (was 1 replica, 5, 500m, 512Mi)
- telemetry poller 10s -> 60s; drop pg_database_size, pg_stat_statements,
  pg_table_size and pg_indexes_size (these were ~7100s of cumulative DB time
  on their own)
- get_latest_packet_for_callsign: bound by :packet_retention_days so partition
  pruning kicks in
- get_nearby_stations_knn: ST_DWithin spatial pre-filter (default 500km) so
  the GiST geography index cuts candidates before DISTINCT ON sort
  (EXPLAIN: 5793ms -> 400ms)
- New migration: partial functional indexes on upper(object_name) and
  upper(item_name), built per-partition CONCURRENTLY then attached to parent.
  Lets get_latest_packet_for_callsign use BitmapOr across all 3 upper()
  indexes instead of falling back to a scan
2026-05-12 17:48:58 -05:00
249e3fa675
Update dependencies
Upgrades: circular_buffer 1.0.1, db_connection 2.10.1, decimal 2.4.1,
mint 1.8.0, phoenix 1.8.7, swoosh 1.25.2, telemetry 1.4.2
2026-05-11 09:58:18 -05:00
6f1d4eee4e
Fix Analyzer.debug_info_line to return nil on with-fallthrough
When :beam_lib.chunks fails (e.g. a project module hasn't been loaded
into :code.which yet), the existing with returned the raw error tuple
as the 'line' value, which then crashed format/1 via String.Chars.

Add an explicit else clause so the function always returns nil or an
integer.
2026-05-09 11:26:41 -05:00
5184d2ee5e
Cache MixUnused analyzer state to skip repeated BEAM scans in tests
- Split Analyzer.analyze/1 into compute_state/1 (slow, scans all BEAMs)
  and a fast filter step. analyze/1 now accepts :precomputed_state opt.
- Mix.Tasks.Compile.Unused.run/1 picks up precomputed state from process
  dict :mix_unused_extra_analyze_opts when present (test escape hatch).
- AnalyzerTest setup_all computes state once; exclude tests pass it
  through analyze/1, dropping per-test cost from ~700ms to <1ms.
- UnusedTest setup_all does the same — the consolidated severity-args
  test drops from 3.4s to ~5ms.
- Trim further sleeps in HistoricalLoadingTest (100->30ms) and
  MovementTest (50->20ms, receive timeouts 100->30ms).

Total suite: 40.6s -> 37.7s; 2488 tests, 0 failures.
2026-05-09 11:17:08 -05:00
4c5c730ee7
Speed up the slowest tests
- packet_replay: stop_replay test reduces fake-task receive timeout from 5s to 200ms
- broadcast_task_supervisor: scheduler_usage takes configurable sample seconds;
  test uses sample-pair API (instant) instead of blocking 1-second sample
- health_check: ask_if_alive timeout is now configurable; unresponsive-pid test
  uses 50ms timeout instead of waiting full 5-second default
- mix_unused/analyzer_test: cache analyze() result in setup_all so 4 tests share
  one analyze pass instead of running 4 separate scans
- mix/tasks/compile/unused_test: consolidate three slow severity tests into one
- log_sanitizer + packet_field_whitelist: cap property-test runs at 25 (was 100)
- historical_loading: trim 200ms post-event sleeps to 50ms
- movement: refute_push_event timeout from 200ms to 50ms

Total suite time: ~45s -> 40.6s; 2488 tests, 0 failures.
2026-05-09 10:56:03 -05:00
1ddc817f50
Remove :slow test tag; make slow tests run fast via configurable timeouts
- Make LeaderElection check_interval, max_cluster_wait, and cluster_check_interval
  configurable via Application env so tests can override with short values
- Slow LeaderElection tests now use 50-150ms timeouts instead of 3-6s
- Remove :slow tag from unused_test (compile cache makes reruns cheap)
- Drop :slow from test_helper exclude list

All 2490 tests now run in ~45s (up from 2485 with 5 excluded).
2026-05-09 09:05:16 -05:00
001af22286
Remove dead code: round_coordinate non-float heads, unreachable get_nearby_stations branch, redundant pattern-match guards 2026-05-08 17:49:23 -05:00
0caf7e4fda
Remove flaky LeaderElection dead-pid cleanup test 2026-05-08 17:27:48 -05:00
e95e2bfe32
Add LogSanitizer tests for sanitize_error_context non-list/non-map branches 2026-05-08 17:22:07 -05:00