Bound batch DISTINCT ON queries by input count. Guard Repo.all(Devices) with exists? check. Add trigram GIN index on base_callsign and (has_weather, received_at DESC) partial index.
Ultraworked with Sisyphus
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
- k8s: 2 replicas, POOL_SIZE 25, cpu 2000m, mem 1Gi (was 1 replica, 5, 500m, 512Mi)
- telemetry poller 10s -> 60s; drop pg_database_size, pg_stat_statements,
pg_table_size and pg_indexes_size (these were ~7100s of cumulative DB time
on their own)
- get_latest_packet_for_callsign: bound by :packet_retention_days so partition
pruning kicks in
- get_nearby_stations_knn: ST_DWithin spatial pre-filter (default 500km) so
the GiST geography index cuts candidates before DISTINCT ON sort
(EXPLAIN: 5793ms -> 400ms)
- New migration: partial functional indexes on upper(object_name) and
upper(item_name), built per-partition CONCURRENTLY then attached to parent.
Lets get_latest_packet_for_callsign use BitmapOr across all 3 upper()
indexes instead of falling back to a scan
- packet_replay: stop_replay test reduces fake-task receive timeout from 5s to 200ms
- broadcast_task_supervisor: scheduler_usage takes configurable sample seconds;
test uses sample-pair API (instant) instead of blocking 1-second sample
- health_check: ask_if_alive timeout is now configurable; unresponsive-pid test
uses 50ms timeout instead of waiting full 5-second default
- mix_unused/analyzer_test: cache analyze() result in setup_all so 4 tests share
one analyze pass instead of running 4 separate scans
- mix/tasks/compile/unused_test: consolidate three slow severity tests into one
- log_sanitizer + packet_field_whitelist: cap property-test runs at 25 (was 100)
- historical_loading: trim 200ms post-event sleeps to 50ms
- movement: refute_push_event timeout from 200ms to 50ms
Total suite time: ~45s -> 40.6s; 2488 tests, 0 failures.
- Make LeaderElection check_interval, max_cluster_wait, and cluster_check_interval
configurable via Application env so tests can override with short values
- Slow LeaderElection tests now use 50-150ms timeouts instead of 3-6s
- Remove :slow tag from unused_test (compile cache makes reruns cheap)
- Drop :slow from test_helper exclude list
All 2490 tests now run in ~45s (up from 2485 with 5 excluded).
Wire up prom_ex with built-in Application, Beam, Phoenix, PhoenixLiveView,
and Ecto plugins, plus a custom plugin that exposes the existing
app-specific telemetry (packet pipeline, spatial PubSub, repo pool,
postgres stats, insert optimizer) as Prometheus metrics.
Mount PromEx.Plug at /metrics — internal-only, intended to be scraped
by the cluster's external Prometheus through the kube-apiserver pod
proxy. K8s pod template now carries prometheus.io/{scrape,port,path}
annotations so the existing kubernetes-pods scrape job picks it up
automatically.
PromEx is disabled in the test environment.
Also harden the packet cleanup and partition manager tests with
Repo.delete_all setup hooks so they aren't poisoned by residual rows
committed outside a sandbox transaction in earlier runs (which were
inflating delete counts and triggering ATTACH PARTITION check_violation
errors against packets_default).
- consolidate Haversine: CoordinateUtils delegates to GeoUtils;
InfoLive.haversine/4 reuses it (m -> km)
- AprsSymbol: collapse 3 sprite-info builders onto build_sprite_info/2
and add a sprite_info @type
- DataBuilder: 3 weather unit converters share convert_unit/4 with
explicit clauses for "N/A", numeric, and binary inputs
- mobile_channel: do_callsign_match/2 uses pattern-match clauses for
wildcard handling instead of nested if
- info_live: replace nested case in position_changed? with multi-clause
coords_changed?/1; format_timestamp_for_display pipelines into a
guarded build_timestamp_display/1
- api_docs_live: collapse chained single-key assigns into keyword-list
assign/2 calls
- add strict @spec coverage on all touched helpers; tighten new specs to
match dialyzer success typing (no contract_supertype warnings)
Remove faker, floki, igniter, ecto_psql_extras (zero usages), exvcr
(zero usages), and resend. Replace resend with Aprsme.Swoosh.ResendAdapter
using the already-present req dependency. Frees 23 packages total from
the lockfile including hackney, certifi, and the tesla dependency chain.
Fix two pre-existing test isolation bugs:
- signal_handler_test called the real ShutdownHandler.shutdown() via SIGTERM
simulation without restoring state, causing page_controller_test to see
shutting_down: true and return 503
- page_controller_test setup now resets ShutdownHandler state defensively
The original get_status/0 had a nested case/try that duplicated the
"disconnected" status map across two branches. Split into:
- status_from/1 with nil + is_pid heads for the "no GenServer" and
"GenServer alive" paths. The alive path isolates the catch :exit, _
fallback for a non-responding process.
- disconnected_status/1 builds the shared status map once, parameterised
on the default server string to preserve the original branch-specific
fallback behavior.
Removes ~20 duplicated lines; all 61 Is tests still pass.
- get_cluster_aprs_status: fetch_cluster_status/1 dispatches on the
clustering flag — true fans out cluster-wide, false goes straight to
the local Aprsme.Is status.
- init/1: extract schedule_initial_election/1 with true/false heads.
The clustered branch sets up the check-and-backstop timers; the
non-clustered branch queues an immediate election. Makes the two
startup paths obvious from the function signatures.
No behavior change; all 17 LeaderElection tests still pass.
- encoding_info/1 (binary clause): extract build_encoding_info/2 with
true/false heads on String.valid?/1, so the valid-UTF-8 and invalid
branches each own their map literal.
- try_utf8_conversion/1: convert_to_utf8/2 dispatches on String.valid?/1
so the latin1 → UTF-8 fallback is a function head, not an inline if.
No behavior change; all 34 Encoding tests still pass.
Cluster.ConnectionManager.handle_info({:leadership_change, ...}) used
a cond on the (became_leader, node, connection_started) triple. Split
into handle_leadership_change/3 with three clauses:
- became_leader=true, us=true, not already started → start APRS-IS
- became_leader=false, us=true, already started → stop APRS-IS
- everything else → no-op
Makes the three state transitions visible in the function heads and
eliminates one more cond from the codebase.
The original handle_cast had nested `if`s: outer branch on demand > 0,
inner branch on buffer-over-capacity. Split into:
- handle_cast with a `when demand > 0` guard for the hot path that
skips the buffer entirely.
- A fallback clause that computes whether the new size would overflow
and delegates to buffer_packet/3.
- buffer_packet/3 has two clauses — one for the drop-oldest path
(with telemetry + warning) and one for the simple append path. Both
share the final `maybe_activate_backpressure |> reply_noemit` step.
All 19 existing PacketProducer tests pass; dialyzer stays clean.
- do_hash_password/3: move the changeset.valid? branch out of the
`if` into apply_hash/3, dispatched on the boolean. The bcrypt 72-byte
cap and plaintext-scrubbing steps now live in one clear function head.
- validate_current_password/2: dispatch on valid_password?/2 via
check_current_password/2 so the add_error branch is its own clause.
No behavior change; all 24 Accounts tests still pass and dialyzer stays
clean.
- handle_call(:shutdown, _, state): split into a shutting_down=true
fast-path clause and a normal initiate_shutdown clause. No more
`if state.shutting_down` inside the callback body.
- terminate/2: extract graceful_shutdown_needed?/1 (four function
heads on the reason tag) and maybe_graceful_drain/2 (guarded on
state.shutting_down) so the shutdown side-effects are isolated to
a single, pattern-matched entry point.
Adds 4 unit tests for the pure callback paths
(shutting_down?, shutdown-already-in-progress, EXIT pass-through,
rescue fallback on unavailable GenServer).
- Aprsme.Packet.put_phg_fields: dispatch on the shape of the phg value
(map / 4-char binary / anything else) via put_phg_fields_for/2.
- Aprsme.Packet.get_field_value: remove the catch-all nil clause. After
the put_phg_fields refactor, dialyzer can prove every call site passes
a map and flagged the fallback as unreachable — so it's gone, and
a future mistyped caller now fails loudly instead of silently.
- LogSanitizer.sanitize_map: extract redact_field/2 for readability.
- DeviceIdentification.pattern_matches?: match_pattern/3 dispatched on
String.contains?/2 keeps the rescue tight around the regex path.
- InfoLive.Show: get_received_at pattern-matches on key shape;
format_distance splits locale × range via four format_*_distance
helpers; calculate_course pipes through normalize_bearing/1.
Adds 15 tests exercising the InfoLive.Show helpers (imperial vs. metric
formatting, cardinal bearings, haversine symmetry, Decimal inputs).
Coverage 67.43 → 67.72%.
- HealthCheck.check_health(:readiness): collapse the 3-branch cond into
readiness_status/2 dispatched on (health_status, shutting_down?).
- HealthCheck.shutting_down?: the whereis + alive? + try/catch nest
becomes a small pipeline of function heads (ask_shutting_down?/1 +
ask_if_alive/2), with the catch isolated to the single place it fires.
- EncodingUtils.sanitize_string_fields and sanitize_nested_map shared a
three-branch atom-or-string key lookup. Extract update_existing_key/3
+ first_present_key/3 + nil_aware/1 helpers; the original functions
become two-liners.
- StatusLive.Index.format_uptime: splits the cond into
format_uptime_parts/4 clauses with numeric guards.
- StatusLive.Index.calculate_health_score: struct-shape pattern match
on %{connected: false} / %{uptime_seconds: s} guards replaces the cond.
- StatusLive.Index.format_time_ago: pipe the diff through
format_seconds_ago/1 with four clauses.
Adds tests for the four public StatusLive.Index helpers
(format_uptime, format_time_ago, get_health_description, format_number).
Coverage 67.29 → 67.43%.
- SharedPacketHandler.handle_packet_update: dispatch on filter-result,
and use maybe_enrich/2 dispatched on the enrich? boolean instead of
an inner `if`. lookup_device_info/1 is now a trio of function heads.
- Cluster.Topology.child_spec: build_spec/3 dispatches on
(cluster_enabled?, topologies) tuples — no more nested `if`.
- Cluster.PacketDistributor.distribute_packet: maybe_broadcast/2
dispatches on the leader-and-enabled check; the non-leader / disabled
paths return :ok explicitly instead of relying on an `if` expression's
implicit nil.
Adds tests for SharedPacketHandler (filter/enrich/match paths,
callsign_filter + callsign_and_weather_filter factories) and extends
Cluster.Topology tests to cover the nil-topology and opts-merge cases.
Coverage 66.41 → 66.81%.
- DeviceCache.pattern_matches?: replace a 3-branch `cond` with a
two-clause `case` on `String.contains?(pattern, "?")`. The old `*`
branch was dead code (it did the same exact-match compare as the
fallback), so it's removed.
- Telemetry.DatabaseMetrics.collect_postgres_metrics: dispatch on
environment atom via multi-clause private functions instead of
`if Application.get_env(...) == :test`.
New tests:
- DeviceCache wildcard/exact lookup paths exercised via a seeded cache.
- MobileUserSocket connect/3 for x_headers/peer_data/empty info and
rate-limit deny.
- DatabaseMetrics pool-metrics emission captured via :telemetry handler
and the :test-env short-circuit.
Swaps `if`/`cond` branches for multi-clause function heads or head-pattern
matching, making the control flow visible in the function signatures:
- Navigation.determine_map_location: resolve_location/4 now dispatches on
the shape of the geolocation input and whether URL params are explicit.
- Navigation.handle_callsign_tracking: per-case clauses for empty callsign
and explicit-URL-params short-circuits, plus a final clause that does the
DB lookup and rescues once.
- Navigation.handle_callsign_search: dispatch_callsign_search/3 separates
the valid/invalid callsign branches.
- InsertOptimizer.optimize_based_on_performance: pattern-match on
[_, _, _ | _] to require ≥3 samples without calling length/1.
- InsertOptimizer.calculate_insert_options and throughput helpers split
into guarded clauses instead of inline `if`.
- maybe_emit_optimization uses same-var pattern matching to skip the
telemetry hit when nothing changed.
Adds unit tests covering every branch of both modules plus SignalHandler,
LocaleHook, and the InsertOptimizer's :noproc fallback path. Coverage
64.69% → 65.72%.
Three related bugs prevented upsert_devices/1 from ever succeeding in
production:
- The JSON map keys ("identifier", "vendor", etc.) were passed through
to Repo.insert_all, which only accepts atom field names.
- updated_at was a %DateTime{} but the Devices schema uses :naive_datetime.
- inserted_at was never set, violating the NOT NULL constraint.
Fix by whitelisting known string keys and converting them to atoms via
String.to_existing_atom/1 (safe against atom exhaustion), using
NaiveDateTime.utc_now/1, and setting both timestamps explicitly.
Converts the inline state maps in Is, PacketConsumer, LeaderElection, and
CircuitBreaker into structs with @type t specs. Enforced keys catch typos
on state updates and give dialyzer a concrete type to check against
instead of map() in every handle_* clause.
Is' nested login_params and packet_stats are extracted to their own
modules (Aprsme.Is.LoginParams, Aprsme.Is.PacketStats) rather than
defined inline, per the project's no-nested-modules rule.
Every other defstruct in the app already has a @type t declaration;
these two GenServer states were the last holdouts. Declaring t here
means the whole codebase is ready for the typed-struct milestone of
Elixir's set-theoretic type system (the next phase after the inference
pass already shipping in 1.19).
get_recent_packets/1 and get_recent_packets_for_map/1 now accept a
:cursor option (%{received_at: DateTime, id: uuid}) instead of :offset
for deep paging. On the 100M-row partitioned packets table, offset
scanning was linear in page depth; keyset is constant.
Ordering tightened to (received_at DESC, id DESC) so the cursor is
stable across ties. idx_packets_received_desc still covers the lookup;
a dedicated (received_at DESC, id DESC) index isn't worth its size
given microsecond timestamp granularity.
historical_loader migrated from batch_offset*batch_size scheduling to
a sequential cursor chain. The old loader fanned out N Process.send_after
batches up-front which could arrive out-of-order; the cursor version
waits for batch N before scheduling N+1, and also short-circuits the
remaining schedule when an under-full batch signals exhaustion.
mobile_channel load_historical_packets dropped its meaningless
`offset: 0`; callsign history already lacked offset.
No LiveView or WebSocket payload shapes changed. offset: 0 (or omitted)
still behaves as first page; non-zero offsets are silently ignored.
Turned on :error_handling, :underspecs, and :unmatched_returns in
mix.exs dialyzer config. The 97 warnings this surfaced were fixed in
place rather than suppressed:
- unmatched_return (79): explicit discard with `_ = ...` for
fire-and-forget side effects (Process.cancel_timer, :ets.new,
send/2), and pattern-matched `:ok = ...` for control-plane
Phoenix.PubSub subscribe/unsubscribe/broadcast calls so a future
return-shape change fails loud.
- contract_supertype (18): tightened @spec arg and return types on
data_builder, historical_loader, url_params, packet_utils,
encoding_utils, aprs_symbol, weather_controller, packet_replay to
match each function's actual success typing.
No behavioural change. mix compile clean, 1008 tests pass, dialyzer
count is now 0.
Stored entries now carry a monotonic access counter refreshed on every
hit. When the cache fills, the 10% least-recently-used entries are
dropped instead of an arbitrary half of the table, which previously
caused thrashing under diverse input by evicting hot entries.
- packet_consumer: when a batch exceeds max_batch_size, carry the
excess over to the next cycle instead of dropping it. Previously
any overage was split off into a drop list that was only logged,
losing packets the producer had already acknowledged.
- map_live: cancel the hover_end_timer in terminate/2 so a user
disconnecting while a marker is still highlighted doesn't leave
an orphaned timer behind.
- packet_consumer: rescue/log exceptions inside the async broadcast
task so PubSub serialization bugs or outages are observable
instead of silently killing the Task.
- prepared_queries: swap ST_Y/ST_X BETWEEN predicates for the && /
ST_MakeEnvelope pattern so bounds queries hit the GIST index on
packets.location instead of sequentially scanning.
Drop the pre-insert aprs_messages broadcast in Is.dispatch (subscribers
already get a richer payload via postgres:aprsme_packets after insert).
Switch PacketsLive.CallsignView to the per-callsign packets:<CS> topic
so it only receives relevant packets instead of filtering every one.
In PacketConsumer: reuse the received_at stamped in Is.dispatch instead
of calling DateTime.utc_now/0 per packet; drop the duplicate struct_to_map
pass (Is.dispatch already handled it); fold coordinate validation and
Geo.Point construction into set_lat_lon + create_location_geometry so
coords are normalized once; extract device_identifier from the already-
normalized attrs; pre-build the broadcast payload once (identifier pick,
lat/lon aliases, routing callsign) so the async broadcast task no longer
does Map.drop/Map.merge per packet.
Rewrite PacketSanitizer.sanitize_packet / sanitize_data_map with Map.new/2
instead of Enum.reduce + Map.put — one allocation per packet instead of N.
Also compute has_weather in Packet.changeset/2 so direct-changeset inserts
(tests, backfills) populate it the same way the GenStage pipeline does.
Dep upgrades (mix deps.update --all):
- phoenix_live_view 1.1.27 → 1.1.28
- swoosh 1.23.1 → 1.25.0
- bandit 1.10.3 → 1.10.4
- credo 1.7.17 → 1.7.18
- hammer 7.2.0 → 7.3.0
- igniter 0.7.6 → 0.7.9
- and minor: fine, lazy_html, meck, mimerl
- removed stale lock entries: geocalc, gettext_pseudolocalize, gridsquare
Bug fixes surfaced during update:
- ETS cache tables were :protected (owned by Application master), preventing
the Cache GenServer from writing; change to :public + write_concurrency
so device/symbol/query caches actually work
- historical_dot_html returned Phoenix.HTML.safe tuple instead of plain string;
symbol_html is JSON-encoded for JS so it must be binary
- String.slice always returns binary so the || "Unknown error" fallback was
unreachable dead code (dialyzer guard_fail)
- Supervisor.which_children always returns a list so the _ -> branch in
database_metrics was unreachable dead code (dialyzer pattern_match_cov)
- Add @type t :: %__MODULE__{} to User schema to resolve unknown_type in user_auth spec
- Extract nested if in leader_election to fix Credo nesting depth violation
- Update .dialyzer_ignore.exs: remove 2 stale entries, add false positives for
Ecto.Multi/Gettext opaque types and Mix.Task PLT limitations
Elixir 1.19 deprecates storing regexes in module attributes. Inline
@wx_preamble_pattern and @wx_field_patterns directly into their
respective functions to resolve the --warnings-as-errors CI failure.
https://claude.ai/code/session_01Ps7Zq3wiBur1RtRKN7JM6X
- RegexCache: change ETS table from :public to :protected for consistency
with other tables, preventing uncontrolled writes bypassing GenServer
- MobileChannel: ensure_float now returns nil instead of the original
unparsed string on parse failure, so validate_bounds properly rejects it
- map.ts: remove dead commented-out localStorage code
- map.ts: use instanceof HTMLAnchorElement instead of unsafe type cast
in popup navigation handler
- TrailManager: add destroyed flag to prevent stale RAF and debounce
callbacks from firing after destroy, and clean up hover timer on destroy
https://claude.ai/code/session_01Ps7Zq3wiBur1RtRKN7JM6X
