Remove unused 30s timer, skip heatmap recluster on every single packet, trail dedup O(n^2)->O(n)
Ultraworked with Sisyphus
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
The unstyled <h1>Packets</h1> template was dead code - no route in the
router renders it (/packets uses PacketsLive.Index LiveView instead).
Removed the template and its test. Added findings.md with all audit
results from comprehensive security and code quality review.
- k8s: 2 replicas, POOL_SIZE 25, cpu 2000m, mem 1Gi (was 1 replica, 5, 500m, 512Mi)
- telemetry poller 10s -> 60s; drop pg_database_size, pg_stat_statements,
pg_table_size and pg_indexes_size (these were ~7100s of cumulative DB time
on their own)
- get_latest_packet_for_callsign: bound by :packet_retention_days so partition
pruning kicks in
- get_nearby_stations_knn: ST_DWithin spatial pre-filter (default 500km) so
the GiST geography index cuts candidates before DISTINCT ON sort
(EXPLAIN: 5793ms -> 400ms)
- New migration: partial functional indexes on upper(object_name) and
upper(item_name), built per-partition CONCURRENTLY then attached to parent.
Lets get_latest_packet_for_callsign use BitmapOr across all 3 upper()
indexes instead of falling back to a scan
When :beam_lib.chunks fails (e.g. a project module hasn't been loaded
into :code.which yet), the existing with returned the raw error tuple
as the 'line' value, which then crashed format/1 via String.Chars.
Add an explicit else clause so the function always returns nil or an
integer.
- Split Analyzer.analyze/1 into compute_state/1 (slow, scans all BEAMs)
and a fast filter step. analyze/1 now accepts :precomputed_state opt.
- Mix.Tasks.Compile.Unused.run/1 picks up precomputed state from process
dict :mix_unused_extra_analyze_opts when present (test escape hatch).
- AnalyzerTest setup_all computes state once; exclude tests pass it
through analyze/1, dropping per-test cost from ~700ms to <1ms.
- UnusedTest setup_all does the same — the consolidated severity-args
test drops from 3.4s to ~5ms.
- Trim further sleeps in HistoricalLoadingTest (100->30ms) and
MovementTest (50->20ms, receive timeouts 100->30ms).
Total suite: 40.6s -> 37.7s; 2488 tests, 0 failures.
- packet_replay: stop_replay test reduces fake-task receive timeout from 5s to 200ms
- broadcast_task_supervisor: scheduler_usage takes configurable sample seconds;
test uses sample-pair API (instant) instead of blocking 1-second sample
- health_check: ask_if_alive timeout is now configurable; unresponsive-pid test
uses 50ms timeout instead of waiting full 5-second default
- mix_unused/analyzer_test: cache analyze() result in setup_all so 4 tests share
one analyze pass instead of running 4 separate scans
- mix/tasks/compile/unused_test: consolidate three slow severity tests into one
- log_sanitizer + packet_field_whitelist: cap property-test runs at 25 (was 100)
- historical_loading: trim 200ms post-event sleeps to 50ms
- movement: refute_push_event timeout from 200ms to 50ms
Total suite time: ~45s -> 40.6s; 2488 tests, 0 failures.
- Make LeaderElection check_interval, max_cluster_wait, and cluster_check_interval
configurable via Application env so tests can override with short values
- Slow LeaderElection tests now use 50-150ms timeouts instead of 3-6s
- Remove :slow tag from unused_test (compile cache makes reruns cheap)
- Drop :slow from test_helper exclude list
All 2490 tests now run in ~45s (up from 2485 with 5 excluded).
Adds a self-contained 'unused' Mix compiler task that finds public
functions in this project no caller in the project ever invokes:
- MixUnused.Analyzer: enumerates public functions across the project's
compiled BEAM files (filtered by app), walks the abstract code in
each BEAM via :beam_lib to collect every remote {Module, fun, arity}
call observed at compile time, and subtracts the call set from the
def set. Exposes :exclude (modules or MFA triples), :compile_path,
and :app options. Behaviour callbacks (@impl true and any listed
in behaviour_info(:callbacks)) and well-known entry points
(start_link, child_spec, mount, render, GenServer callbacks,
__info__, module_info, etc.) are excluded automatically.
- Mix.Tasks.Compile.Unused: a Mix.Task.Compiler that compiles the
project then asks the analyzer for unused defs. Severity is
configurable via the :unused project key; per-file exclusion of
test/ paths is on by default.
Inspired by https://github.com/hauleth/mix_unused — uses BEAM
introspection rather than a compilation tracer to avoid the
chicken-and-egg problem of recompiling the tracer module itself
during analysis.
Adds tests for the PromEx supervision module and custom plugin (both
were 0–25% covered), expands the HealthCheck plug to exercise the
readiness success path through the live ShutdownHandler, adds public-API
roundtrip tests for PacketReplay's via_tuple wrappers, and covers
several private-helper fallback branches in Aprsme.Packet
(ParseError data_extended, MicE map symbol defaults, struct
data_extended, weather binary, has_position via legacy lat/lon, PHG /
altitude error parsing).
Wire up prom_ex with built-in Application, Beam, Phoenix, PhoenixLiveView,
and Ecto plugins, plus a custom plugin that exposes the existing
app-specific telemetry (packet pipeline, spatial PubSub, repo pool,
postgres stats, insert optimizer) as Prometheus metrics.
Mount PromEx.Plug at /metrics — internal-only, intended to be scraped
by the cluster's external Prometheus through the kube-apiserver pod
proxy. K8s pod template now carries prometheus.io/{scrape,port,path}
annotations so the existing kubernetes-pods scrape job picks it up
automatically.
PromEx is disabled in the test environment.
Also harden the packet cleanup and partition manager tests with
Repo.delete_all setup hooks so they aren't poisoned by residual rows
committed outside a sandbox transaction in earlier runs (which were
inflating delete counts and triggering ATTACH PARTITION check_violation
errors against packets_default).
Coverage:
- Adds focused tests across 13 files covering previously-uncovered
branches: Packet changeset normalisation (course, wind_direction,
struct data_extended); Packets store_packet edge cases (nested
position shapes, ssid handling, string-keyed raw_packet);
DeviceIdentification HTTP success/404 via Req plug stub; ResendAdapter
catch-all body, deliver_many, nil/binary formatters; ApiDocsLive
packets with missing fields; AprsSymbol normalize helpers;
BadPacketsLive refresh and postgres_notify; MobileChannel handle_info
catch-all and postgres_packet path; PacketProcessor existing-marker
movement detection; PacketReplay sanitize_value type clauses;
InsertOptimizer GenServer lifecycle and negative-duration metric;
PageController shutdown-handler integration; UrlParams delegated
helpers.
- mix.exs: configure test_coverage with summary threshold of 87 and
ignore_modules for test fixtures and macro-only template modules.
Bug fix:
- MapLive.Index handle_info: add a clause for the raw {:new_deployment,
_} tuple. Phoenix.PubSub.broadcast delivers the raw payload to plain
subscribers, but the existing handler only matched the %Broadcast{}
fast-lane wrapper. DeploymentNotifier broadcasts via the raw path,
which crashed the LiveView under test concurrency.
Submodule:
- Update .gitmodules vendor/aprs URL from
https://github.com/aprsme/aprs.git to
ssh://git@codeberg.org/gmcintire/aprs.git.
- Delete config/appsignal.exs (was never wired into mix.exs or any
other config file)
- Migration: fix decrement_packet_sequence trigger to handle the case
where currval() hasn't been called in the current DB session, matching
the same exception handling already present in get_packet_count()
- Fix get_historical_packet_count test: invalid non-list bounds are
silently ignored (no error), so assert is_integer not count == 0
- Add Repo.delete_all(Packet) setup blocks in PacketsOldestTest and
PacketConsumerTest to clear any committed DB state before tests that
depend on an empty packets table
- maxSurge: 1 (explicit integer, avoid percentage rounding)
- minReadySeconds: 30 (was 10) — new pod must be stable longer before old is killed
- progressDeadlineSeconds: 600 — detect stuck rollouts
- readinessProbe failureThreshold: 3 (was 2) — more tolerance after old pod is gone
- Add PodDisruptionBudget minAvailable: 1 — prevents voluntary disruptions from dropping below 1 pod
