- PathCompute: compute_loss_budget/5 and compute_power_budget/2 made
public with @doc false; property tests covering fspl, o2, h2o, rain,
diffraction across all 13 known bands + varying distances
- Viewshed: property tests for destination_point back-bearing round-trip
and east-west bearing at equator (2 new properties)
Coverage: 79.07% → 79.06% (PathCompute 60→63%)
- Create Microwaveprop.Valkey.Adapter behaviour with command/3 + pipeline/3
- Create Microwaveprop.Valkey.RedixAdapter as the production impl
- Inject adapter via Application config; mock with Mox in tests
- Cover all Valkey operations: get, mget, set, mset_with_ttl, zadd,
zrevrange, zrangebyscore, zrem, del, scan_match (multi-cursor)
- Full round-trip tests for encode/decode, error paths, edge cases
Coverage: 78.93% → 79.07%
The producer-only path left older hours stuck at 0% HRRR coverage
forever once their fetch tasks drained: the rolling-window cron only
re-passes the prior hour, so manual lookback runs (or any
out-of-window hour) wrote NULL samples + enqueued tasks, then never
re-ran to UPSERT the now-landed HRRR data.
Fix: split the responsibility cleanly.
* CalibrationSampler.build_for_hour/1 now returns
%{upserted: int, missing_hrrr_cells: int} so callers know whether
the producer enqueued anything (i.e. whether a follow-up makes
sense). Spec-tightened with a new @type result.
* PskrCalibrationWorker, after each hour's sampler pass, schedules a
follow-up of itself for that exact hour 10 min later when missing
> 0. The follow-up carries args %{"hour_utc" => ..., "_follow_up"
=> true}; the sentinel prevents follow-ups from chaining further
follow-ups (rolling-window cron is the safety net).
Tests:
* Updated 4 existing sampler tests to the new map return shape.
* Moved the follow-up assertions from the sampler suite into the
worker suite where they belong (sampler is now scheduling-free).
* Added 3 worker tests: schedules-when-missing, no-schedule-when-
covered, follow-up-doesn't-chain.
3313 tests + 228 properties, 0 failures.
Two performance fixes the property tests pinned down:
* CalibrationSampler.build_for_hour/1 walked every cell through
nearest_hrrr/3 twice — once in the producer and again in
build_sample. Now computed once into a (band, lat, lon) → hrrr_or_nil
map, halving the O(cells × profiles) scan (~10M comparisons saved
per fire at typical sizes).
* PartitionManager.ensure_quarterly_partitions/2 issued one
pg_inherits scan per (parent × lookahead). Refactored to one scan
per parent with in-memory coverage check across all quarters.
Property tests for both modules:
* PartitionManager: tile-coverage invariant (no gaps, no overlaps),
exact 3-month bounds, name format, multi-call idempotence,
multi-parent independence — caught a real NaiveDateTime sort bug
in the original test helpers. Default Enum.sort_by/2 falls back to
term comparison on NaiveDateTime; now uses NaiveDateTime as the
comparator module.
* CalibrationSampler: enqueued points = unique missing midpoints,
empty enqueue when fully covered, sample row count invariant under
HRRR availability.
Saved operational gotchas to CLAUDE.md (Oban leader election spans
all app=prop pods, kubectl exec deploy/prop ambiguity, half-year
partition coexistence, HRRR f000 publish lag).
3310 tests + 228 properties, 0 failures.
Adds the test that verifies the full producer→drain→re-pass loop:
sample written with NULL HRRR fields + fetch task enqueued, then once
an HRRR profile lands, the next sampler pass UPSERTs the same
calibration_samples row (id preserved) with non-NULL weather data.
Also tightens dialyzer surface:
* PartitionManager defines a `result()` typedoc and uses it on both
public functions instead of inlined tuple types.
* PartitionMaintenanceWorker.perform/1 + the lookahead/1 helper get
explicit specs.
* CalibrationSampler.enqueue_missing_hrrr/3 gets a spec covering its
group_by-shape input map and HRRR profile list.
`mix precommit` clean (3310 tests, 0 failures). Local `mix dialyzer`
hits an OTP 28.4 vs 28.5 toolchain mismatch unrelated to this change.
Partition list for hrrr_profiles + hrdps_profiles was hand-edited in
priv/repo/structure.sql; if no human extended it, writes for the next
quarter would silently fail at the worker level (Rust hrrr-point-worker
marks tasks failed; the calibration pipeline silently joins NULLs).
Adds:
* Microwaveprop.PartitionManager — runtime DDL helper. Quarter math via
a single integer index (year*4 + quarter) for rollover safety.
Coverage check via pg_inherits before CREATE so a quarterly target
inside an existing wider partition (some 2027 ones are half-year)
is treated as :exists rather than colliding.
* Microwaveprop.Workers.PartitionMaintenanceWorker — daily cron at
03:15 UTC, 4-quarter lookahead. Idempotent; cheap when nothing's
missing. lookback_quarters arg lets ops widen for catch-up.
Tests cover the create + idempotent re-run + name format + year
rollover paths via a synthetic parent created inside the sandbox
transaction (no DDL leaks into the shared test DB).
The CalibrationSampler joined PSKR midpoints against hrrr_profiles via
nearest-within-window, but post-Phase-3 Stream C nothing bulk-populates
the grid — hrrr_profiles only sees per-QSO point fetches from the Rust
hrrr-point-worker. Result: 0 / 16,042 prod samples had HRRR data.
Producer pattern: when a cell has no profile in the lookahead window,
enqueue a row into hrrr_fetch_tasks at the cell's HRRR-grid-rounded
midpoint. The Rust worker drains it; the next 5-min rolling-window
pass UPSERTs the same sample with non-NULL weather fields. Idempotent
with the existing on-conflict design — no extra worker, no churn on
cells already covered, multiple bands sharing a midpoint dedupe to a
single fetch via Enum.uniq + jsonb point-union in HrrrPointEnqueuer.
A 4-char locator covers ~70 km × 100 km, which dwarfs HRRR's 3 km
native cell and any midpoint we'd derive for the calibration corpus.
Storing those samples gives the recalibrator geometry that's
indistinguishable from noise.
fetch_grid/2 now returns {:error, {:short_grid, key, grid}} for any
locator under 6 chars after Maidenhead validation. The spot is
dropped before it reaches the aggregator, so neither pskr_spots_hourly
nor pskr_calibration_samples ever see it.
Old design fired hourly at HH:25 and built one whole hour of cells
in a single batch. During PSKR peaks (50k+ spots/hour) that batch
held the DB pool long enough to look like a backlog, and a pod
restart between fires lost a full hour because the next cron only
caught the most-recent-prev hour.
- Default window is now current hour + 1 prior. Every 5-min fire
spreads work across the in-flight hour and always covers the
boundary without waiting for the next cron tick.
- lookback_hours arg widens the window for catch-up
(e.g. {"lookback_hours": 24} sweeps the last day after an
outage).
- Unique period 3600 → 240 so consecutive 5-min cron fires actually
slot in.
UPSERT is already idempotent so the ~12 redundant passes per hour
cost only the read+merge. Latency from spot ingestion to sample
creation drops from up-to-60min to ≤5min.
Replaces the "tropo composite + post-hoc aurora boost" model with a
max-over-mechanisms dispatcher so each band's score reflects the
dominant propagation channel rather than summing alternative paths.
- BandConfig.mechanisms/1 derives the stack from frequency:
≤432 MHz gets [:tropo, :aurora]; >432 MHz stays [:tropo]. Bands
may override with an explicit :mechanisms field.
- Scorer.mechanism_score/3 returns a standalone integer score for
one mechanism (:tropo delegates to composite_score; :aurora is a
clean Kp/freq function, not a boost on top of tropo).
- Scorer.cell_score/2 takes max across the band's mechanisms so a
Kp-6 storm scores 6m at 85 even when the tropo picture is poor,
instead of conflating the two channels.
Es and F2 stay in PathCompute since they're path-distance-dependent.
The legacy aurora_boost path is unchanged so no callers shift today.
`Pskr.Recalibrator.run/0` reads `pskr_calibration_samples`, bins
each sample per (band × feature), and writes spot-density stats to
`pskr_feature_bins` so an operator can read whether a feature
actually discriminates propagation at the threshold granularity
the scorer uses.
Bins, not regression: `BandConfig` already encodes scoring as
discrete thresholds, so the bin output matches that shape and an
operator can copy adjusted thresholds directly without translating
from regression coefficients.
Self-healing: corpus too thin ⇒ run row written with status
`skipped_insufficient_data` and the analysis is a no-op until next
fire. `min_total_samples = 1000` (≈ 4-5 days of CONUS PSKR
activity); per-band threshold is 100. Both surface in the run row's
`notes`.
Auto-applies nothing. Weight changes still go through human review
of `BandConfig.@band_configs` and a code commit. The recalibrator
is a read-only analyst that stays out of the production scoring
path.
Features binned (matching the scorer's discriminating fields):
* pwat_mm — humidity U-shape candidate
* hpbl_m — boundary layer (mechanism vs scoring re-eval)
* min_refractivity_gradient — refractivity threshold validation
* surface_pressure_mb — pressure-front proxy
* kp_index — aurora boost magnitude tuning
Schema: two tables.
* `pskr_recalibration_runs` — one row per fire with corpus
stats, status, notes
* `pskr_feature_bins` — one row per (run, band, feature, bin)
with sample_count, spot_count_total/avg/p50/p90
Cron: `0 4 * * 0` (Sundays 04:00 UTC, off-peak, post-climatology).
Manual reruns enqueue with no args.
Tests cover the empty-corpus skip path, sub-threshold totals,
per-band threshold gating, the actual bin emission, nil-feature
handling, spot-count averaging, and the always-records-a-run
audit invariant. 8 new tests, 3282 total passing.
Backfill pipeline untouched.
PSK Reporter is now an always-on data feed, so the calibration
corpus that recalibration will eventually train against can grow
hourly from when the firehose started. One row per (hour, band,
0.125° midpoint cell) joins three sources:
* `pskr_spots_hourly` — observed spot density (truth signal)
* `hrrr_profiles` nearest match at the cell midpoint at hour
boundary (atmospheric features: T, Td, PWAT, P, dN/dh, HPBL,
ducting flag)
* `geomagnetic_observations` latest Kp at the hour (space weather)
Predicted scores are intentionally NOT stored — they're a function
of the algorithm version under evaluation. Storing only features
keeps the corpus stable across every weight refit; the recalibrator
computes predictions on demand.
Components:
* Migration `20260504210756_create_pskr_calibration_samples` —
new table + unique index on (hour, band, midpoint_lat, lon),
plus a midpoint spatial index on `pskr_spots_hourly` to keep
the join cheap.
* `Pskr.CalibrationSample` — schema mirror of the table with
the same `(hour, band, midpoint_lat, lon)` unique constraint.
* `Pskr.CalibrationSampler.build_for_hour/1` — pulls all spots
for the hour, snaps midpoints to the propagation grid (0.125°),
builds an in-memory HRRR index over a ±0.07°/±60 min window,
and bulk-upserts samples. Idempotent — re-runs upsert.
* `Workers.PskrCalibrationWorker` — Oban cron entry on the
`:backfill_enqueue` queue. Default args target the previous
full hour; explicit `hour_utc` arg reruns any hour.
* Cron `25 * * * *` — fires past HRRR analysis publish window
(~HH:50→HH:05) and PSKR aggregator's 60s flush.
Forward-only: PSKR has no historical archive, so the corpus only
grows from when the feed started. Recalibration weight refits
should wait for ~30 days / ~10k samples to cover diurnal and
synoptic variability.
Tests cover cell-snapping, HRRR feature joining, Kp stamping,
median-distance aggregation, mode dedup, idempotent reruns, and
the worker's default-hour and explicit-hour args (12 new tests,
all passing).
Backfill pipeline untouched — none of these changes feed into
contact enrichment.
`Scorer.aurora_boost(score, kp, band_config)` is a terminal boost
applied after `composite_score` in the same shape as the existing
`commercial_link_boost`. It tracks the NOAA G-scale (Kp 5+ opens
auroral E-region paths up to ~2500 km) and is freq-gated to
≤ 432 MHz — auroral propagation is observed almost exclusively on
50/144/222, occasionally on 432, and never at microwave.
Boost magnitudes:
* Kp < 4 (quiet) — pass-through
* Kp 4 — +5
* Kp 5 (G1) — +15
* Kp 6 (G2) — +25
* Kp ≥ 7 (G3+) — +35, clamped to 100
Wired into `Propagation.score_grid_point/4` via the existing
`hrrr_profile` map: callers populate `:kp_index` once per cycle so
the per-grid-point band loop never queries the DB. The UI fallback
path (`factors_from_profile/5`) sources Kp from
`SpaceWeather.latest_kp/0` directly. Other call sites
(GEFS forecast, path-compute, contact-show) can opt in by adding
`:kp_index` to their profile map.
Diagnostic factor `:kp_index` is stashed on VHF/low-UHF results so
the operator-facing factor breakdown can show why a band scored
above its quiet-conditions baseline. Microwave bands don't carry
the diagnostic — they're never affected by the boost.
Refactored `score_with_algorithm/7` to delegate band-level
finalization to `finalize_band_result/5`, keeping the parent
function under credo's cyclomatic-complexity limit.
NOTE: this only affects the Elixir scoring path — the live grid
served by `prop_grid_rs` (Rust) does its own scoring and does not
yet honor Kp. Adding aurora to the Rust side is follow-up work.
Both RTMA and METAR5 schemas + clients + workers were defined but
never had a consumer. `find_nearest_rtma/3` and `recent_surface_obs/3`
existed in `Microwaveprop.Weather` with zero callers cluster-wide;
RtmaFetchWorker had test coverage but was never enqueued anywhere;
no IEM 5-min ASOS fetcher was ever written. The tables sat empty in
prod and the recalibrate audit was permanently flagging them BROKEN
even though the empty state was the steady state.
Drop the lot — schemas, clients, workers, queue config, Req.Test
stubs, audit checks, and the per-table unique tests in the
observation-changesets suite. New migration drops the now-unreferenced
`rtma_observations` and `metar_5min_observations` tables (both 0
rows in prod). Trim the `:rtma` slot out of `backfill_only_queues`
in runtime.exs so Oban Pro's Smart engine stops trying to manage a
queue with no producer.
Audit thresholds reset on the surviving NARR check: drop the
"WARN < 5000 absolute rows" rule that didn't account for
NarrFetchWorker's 0.13° spatial dedup, and point remediation at
the existing BackfillEnqueueWorker cron instead of the dev-only
`mix narr.backfill` task.
scripts/recalibrate_algo.py: ROW_COUNT_SOURCES + DATA_GAP_SQL +
DATA_GAP_RULES all stop referencing the dropped tables.
Pin both the sender and receiver DXCC slots to 291 so the broker
only forwards CONUS↔CONUS spots. The previous two-topic-per-band
filter (sender_us OR receiver_us) was leaking US↔CA paths through
the receiver-side wildcard, and the project doesn't have NWP
coverage for the non-US end yet.
When HRDPS lights up we'll add a Canada-anchored topic alongside.
Spots that include subsquare (EM12kl), extended-square (EM12kl37),
or extended-extended-subsquare (EM12kl37ab) precision now keep
that resolution end-to-end. Previously we truncated everything to
4 chars at parse time, which collapsed distinct paths into the same
row whenever two spotters shared a field.
The aggregation key uses the full grid as reported, so two
spotters in different subsquares of `EM12` produce two rows —
correct, since they are physically different paths. Any spot that
fails Maidenhead validation (odd length, illegal char at position
N) is still dropped via the same error path.
Storage normalizes to uppercase so the unique index doesn't split
on case. Maidenhead.valid?/1 already accepts both cases on input.
`:gen_tcp.recv/3` requires a passive socket and returns `:einval` if
the socket is in any active mode. We were opening with
`active: :once`, sending CONNECT, then calling `:gen_tcp.recv` to
read CONNACK — which kernel-rejected every time.
The visible symptom was "Pskr.Client connect failed: :einval" every
30s on the elected leader, while a manual `:gen_tcp.connect` from
`bin/microwaveprop rpc` worked because it never tried to recv. Each
failed attempt also leaked a port (connect succeeded; recv failed
later), and we'd accumulated ~27 dangling sockets on the leader
process.
Open the socket in `active: false`, finish the synchronous
CONNECT/CONNACK/SUBSCRIBE handshake with `:gen_tcp.recv`, then
`:inet.setopts(active: :once)` so PUBLISHes arrive as
`{:tcp, socket, data}` messages going forward. Also close the
socket on any handshake failure so retries stop leaking ports.
Two related changes to the contact detail page:
1. The "Flagged invalid" badge was wrapping to two lines on
narrow widths because the daisyUI badge has no explicit
no-wrap class. Add `whitespace-nowrap`, drop the redundant
capitalization on "Invalid", and append the flagger's
callsign so the badge reads "Flagged invalid by W5XYZ".
Hover tooltip carries the timestamp.
2. New `flagged_by_user_id` + `flagged_at` columns on contacts.
`Radio.toggle_flagged_invalid!/2` now takes the admin User
doing the flagging; on unflag it clears both fields so a
subsequent re-flag gets a fresh attribution and the badge
never shows a stale flagger.
3. `Radio.delete_contact!/1` + a Delete button next to Flag.
Admin-only, gated in both the event handler and the template.
Uses LV's `data-confirm` for the destructive-action prompt.
All FK references to `contacts` already declare
`on_delete: :delete_all` (terrain_profiles, contact_edits,
contact_common_volume_radar) so a single Repo.delete!
cascades cleanly.
K8s pods have IPv4-only egress; without :inet in the gen_tcp opts,
BEAM was picking mqtt.pskreporter.info's AAAA record and the
kernel's connect() returned EINVAL, surfacing as :einval and a
30-second retry loop.
emqtt's transitive `quicer` dep needs CMake + msquic submodule
clones to build, which the slim Debian builder image doesn't ship —
the prod CI build was failing inside `mix deps.compile`. Tortoise311
is the obvious pure-Elixir alternative but it bundles a
gen_state_machine + retry/inflight stack we don't use because we're
subscribe-only at QoS 0.
Implementing the wire protocol inline is ~150 lines in
`Microwaveprop.Pskr.Mqtt` (CONNECT/SUBSCRIBE/PINGREQ/DISCONNECT
builders, CONNACK/PUBLISH/SUBACK/PINGRESP parser, varint codec).
Pskr.Client now uses :gen_tcp with `active: :once` framing,
buffers partial reads, schedules its own keepalive, and parses
inbound packets in a tight loop.
Drops `BUILD_WITHOUT_QUIC=1` from the Dockerfile — no longer
relevant since there's no native dep at all in the build chain
now.
Subscribes to mqtt.pskreporter.info:1883 over plain TCP and folds
every reception report into hourly aggregates per
(band, sender_grid_4, receiver_grid_4). Each spot is the empirical
"propagation actually occurred on this path right now" signal we'll
correlate against HRRR atmospheric state to recalibrate the scoring
weights. Aggregating at the path-hour bucket collapses 50-reporter
QSOs to one row instead of 50.
Topic filter anchors on USA (DXCC 291) on either sender or receiver
side so the broker pre-filters out everything outside our HRRR
domain. Bands subscribed: VHF and up (6m, 2m, 70cm, 23cm, +
microwave) — HF is dominated by ionospheric propagation, which this
project doesn't model.
Pskr.Client cluster-elects a singleton via :global.register_name —
all replicas start the GenServer but only one connects to MQTT;
the rest stay in :standby and watch for the leader's nodedown to
re-run election. Off in dev/test, on by default in prod
(PSKR_MQTT_ENABLED=false as kill-switch).
Pskr.Aggregator buffers in memory keyed by Pskr.path_key/1 and
flushes every 60s via Repo.insert_all/3 with an additive
ON CONFLICT (count summed, SNR envelope by GREATEST/LEAST,
modes unioned via unnest+DISTINCT). Idempotent across overlapping
flushes and across leader handover.
Dockerfile sets BUILD_WITHOUT_QUIC=1 — emqtt's transitive quicer
dep wants CMake + OpenSSL headers we'd otherwise have to add to
the builder image just to never use QUIC over plain MQTT. Base
image is unchanged; the new dep compiles cleanly into the existing
prop-base runtime.
phoenix_pubsub_redis 3.1's deprecation message ('Move :url inside the
:redis_opts option') is misleading — wrapping the URL as
`redis_opts: [url: url]` forwards the keyword list to Redix, which
rejects `:url` with `unknown options [:url], valid options are:
[:host, :port, :password, ...]`. New pods on commit b4d6b04 onward
crashed at startup with that NimbleOptions.ValidationError; old pods
running e53a34d (top-level :url) coasted on already-established
connections so the breakage only surfaced when fresh pods rolled.
The library accepts `redis_opts` as either a URL binary OR a
keyword list. Pass the URL as a bare string so it takes the
binary-clause path that hands it directly to Redix.PubSub.start_link/1.
The library deprecated top-level Redis connection keys; passing :url
directly logged a warning on every boot. Functionally identical, just
silences the warning.
Switches the PubSub adapter to Phoenix.PubSub.Redis backed by the
shared Valkey instance (same backend GridCache uses), via the new
phoenix_pubsub_redis dep. Falls back to the default PG2 / Erlang-
distribution adapter when VALKEY_URL is unset, so dev/test boot
unchanged.
Decouples PubSub fan-out from libcluster's BEAM connection. A
flapping Erlang distribution (the OOM-cascade scenario from
2026-05-03 surfaced this — libcluster warnings dominated the logs
while pods CrashLoopBackOff'd) no longer takes broadcasts with it,
so the rest of the cluster keeps publishing while the bad pod
recovers.
node_name uses POD_IP for cluster-wide uniqueness, falling back to
Erlang's node() name in non-K8s environments. Required by the Redis
adapter so loopback filtering doesn't drop a pod's own broadcasts as
if they came from a peer.
Each cached HRRR valid_time is ~32 MiB compressed × cap of 8 = up to
256 MiB per pod. With 4 hot pods + 1 backfill that's ~1.25 GiB of
cluster RAM holding identical content. Replacing the per-pod ETS
replicas with a single shared Valkey copy collapses that to ~256 MiB
total (off-pod) and makes the BEAM heap on each replica meaningfully
smaller — directly addresses the headroom the 2026-05-03 OOM cascade
ate into.
* New `Microwaveprop.Valkey`: single Redix connection (named conn,
sync_connect: false, exit_on_disconnection: false). `start_link/0`
returns `:ignore` when VALKEY_URL is unset so dev/test boot without
Valkey. Helpers wrap GET/MGET/SET-with-TTL/ZADD/ZREVRANGE/SCAN with
`:erlang.term_to_binary` round-tripping under the safe atom flag.
* `Microwaveprop.Weather.GridCache` now branches on
`Valkey.configured?/0`. Public API is identical so callers (Weather,
GridCachePruneWorker, MapLive) don't change. ETS path is preserved
unchanged for dev/test.
* Storage layout when on Valkey:
prop:wg:vts ZSET of valid_time iso8601 strings
prop:wg:<vt>:<lat>:<lon> one chunk per 5°×5° spatial bucket
fetch_bounds MGETs only the chunks intersecting the viewport, so a
regional view is 1-4 round-trips instead of pulling the full grid.
* Per-key TTL = 8 h (matches the ETS valid_time cap). Eviction is
automatic; prune_keep_latest is a no-op on Valkey, prune_older_than
uses ZRANGEBYSCORE + DEL.
* All Valkey errors fall back to disk (ScalarFile.read_bounds), logged
as warnings — page works through Valkey outages, no user-visible
break beyond a one-shot latency bump.
ScoreCache, Microwaveprop.Cache, NexradCache, telemetry/Postgres
protocol/libcluster registry tables remain ETS — they're sub-ms
hot-path lookups where a network hop would be felt.
VALKEY_URL must be present in prop-secrets for production rollout
(applied out-of-band, secret.yaml is git-ignored).
Found while auditing pod memory: :weather_grid_cache is by far the
largest ETS table (~32 MiB compressed per entry, vs ~2 MiB/entry for
ScoreCache) because each cell is a 22-field weather row instead of a
scalar score. Cap of 24 meant the cache could hoard up to ~768 MiB on
a single pod, and hot pods sit at 3-5 GiB against a 6 GiB limit — most
of that headroom belongs to GridCache.
The cap was sized for 'analysis + 18 forecasts + stragglers' but the
real access pattern is current-hour + a short forward scrub. Beyond
that, fall back to ScalarFile.read_bounds (sub-100 ms per file) instead
of permanently parking ~500 MiB on data nobody looks at.
Worst-case GridCache spend drops 768 MiB → 256 MiB. Steady-state win
is smaller (most pods don't fill all 24 slots), but bounds the tail.
The :terrain queue at 3 slots/pod was sized for the lightweight per-QSO
TerrainProfileWorker, but RoverPathProfileWorker now runs the full
PathCompute pipeline (terrain + 9 HRRR profiles + sounding + ionosphere
+ scoring + loss + forecast) — same heap profile as :propagation, which
sits at 1 slot precisely to avoid OOM.
A backfill_paths flood today queued ~200 rover-path jobs. With 4 hot
pods × 3 slots, the cluster stacked 15 concurrent path-computes and
two pods OOM-killed in a loop, taking the libcluster ring with them
(visible as the "unable to connect" warnings on the surviving pods).
New :rover_path queue at 1 slot/pod caps cluster-wide concurrency at
(hot_replicas + 1 backfill), drains the existing 200-job backlog
inside ~30 min, and keeps :terrain free for the cheap workers it was
sized for.
Existing Path rows persisted before the PathCompute extraction shipped
have no "term" key in their result map, so the live /path page hits
the :stale_term branch and falls back to live recompute when the
operator clicks a row. The hourly backfill cron now flips those
:complete rows back to :pending so the path-profile worker rerolls
them with the new full-output term, after which row clicks render
from cache instantly. The flip uses Postgres's jsonb_exists() so it's
a single targeted UPDATE instead of pulling rows into Elixir.
Two stacked wins for the form-submit hot path:
1) create_mission now hands path-matrix population to the same async
reconcile worker update_mission uses, so the request returns
right after the Mission row insert. For a mission with hundreds
of (rover x station x band) tuples this used to block the LiveView
for many seconds.
2) Inside reconcile/enqueue_paths_for, the per-pair Multi.insert
loop is replaced with one Repo.insert_all for the Path rows and
one Oban.insert_all for the worker jobs. N round-trips collapse
to two regardless of pair count.
Three connected changes:
1) Extract PathLive.compute_path/4 + every helper it owned (resolve,
profile grid lookup, sounding/ionosphere readouts, scoring, loss /
power budgets) into Microwaveprop.Propagation.PathCompute. PathLive
now delegates; ~410 lines of dead helpers deleted from PathLive.
2) RoverPathProfileWorker calls PathCompute.compute/4 with the
mission's heights and PathLive's default station params (10 W TX,
30 dBi gains). Stores the full atom-keyed compute output as a
Base64-encoded :erlang.term_to_binary/1 blob alongside the flat
summary fields the rover-planning show table reads. The blob
roundtrips structs and DateTime exactly (Jason.encode would lose
them).
3) PathLive accepts ?rover_path_id=UUID. When set, loads the cached
Path, decodes the term (binary_to_term :safe), assigns it as
@result, and renders normally — no compute_path call. The
rover-planning show table now links rows directly to
/path?rover_path_id=UUID, so a click opens the full Path
Calculator UI from cached data without re-running terrain / HRRR /
sounding lookups.
Bonus prod fixes folded in:
- PathShow's elevation chart attribute (data-* → data-profile JSON)
was crashing the JS hook with 'unexpected character at line 1'.
- Station.changeset now wipes previously-resolved callsign/grid/
lat/lon when the user types into :input — typing 'AA5' early
resolved to a wrong location and locked it; subsequent keystrokes
never re-resolved.
- phx-debounce=600 on the station input so QRZ doesn't get hit on
every keystroke.
Worker now stores the full per-point analysis (beam height,
Fresnel-zone radius, clearance) on Path.result.path_points, and
PathShow renders a real cached page: terrain elevation chart fed by
the same ElevationProfile JS hook /path uses (feet on Y, miles on X,
LOS + Fresnel overlay), distance / bearing / clearance / antenna
heights all in feet, baseline link-budget breakdown, and the cached
midpoint propagation score with computed_at timestamp.
A Recompute live button hands off to /path for the time-varying bits
(current HRRR conditions, scoring breakdown, sounding, ionosphere)
that aren't worth caching.
Two fixes:
1) Form save / rover-site add/remove no longer runs the path-matrix
reconcile inline. update_mission and the show LiveView now
enqueue a single RoverMissionReconcileWorker job and return —
for missions with many rover sites x bands the inline path was
firing hundreds of Multi inserts + Oban.inserts on the request.
2) RoverPathProfileWorker.load_path/1 now strictly matches a
4-tuple (mission_id, rover_location_id, station_id, band_mhz)
with band_mhz as an integer, plus updates the Oban unique key
set to include band_mhz so multi-band missions enqueue per-band
jobs (not just one). Legacy/malformed args are logged and
dropped instead of crashing the queue with MultipleResultsError.
Mission now carries bands_mhz ({:array, :integer}) — operator picks
one or more bands as multi-checkboxes. enqueue_paths_for builds the
cross product (rover x station x band) and persists each tuple as its
own Path row keyed by (mission_id, rover_location_id, station_id,
band_mhz). The path-profile worker reads band_mhz from the path
itself (legacy single-band jobs without band_mhz in args still resolve
to their unique row).
replace_mission_paths/1 is now a thin alias for reconcile_mission_paths/1
which diffs desired vs actual: stale tuples (old band that the user
unchecked, station they removed, rover-site they deleted) get dropped,
new tuples become :pending and enqueue, surviving :complete rows are
left in place — no more wholesale destruction of already-computed
paths on every edit.
The show table gains a Band column, and band_label() in the mission
summary becomes bands_label() (joins the list with commas).
The path-profile worker now also computes the baseline link-loss
budget — free-space path loss, oxygen absorption, baseline H2O loss
(at 7.5 g/m^3 default humidity), plus the already-cached terrain
diffraction — and stores all four numbers in Path.result. The show
page renders the per-row total in a new Loss column. /path still
recomputes against live HRRR weather; this is the offline-readable
cached snapshot.
Adds RoverPlanning.backfill_paths/0 that walks every mission and
re-enqueues RoverPathProfileWorker jobs for any (rover × station) pair
whose Path is missing or not :complete. The path-profile worker
short-circuits on :complete, so this is idempotent.
Wires a new RoverMissionBackfillWorker into both dev (config.exs) and
prod (runtime.exs) crons at HH:30 — heals missions stuck in :pending
or :failed from transient elevation-API errors or interrupted
create_mission enqueues without operator action.
The path-profile worker now also looks up the propagation grid score for
the path midpoint at the mission's band before flipping status to
:complete, so the row never appears 'done' until both terrain and
propagation prediction have run. The show table renders that score as a
red/yellow/green badge in a new Propagation column.
Each rover-location group heading is now a link to /rover-locations/:id
so the user can jump straight to the spot's detail page from the
mission view.
A new "Rover sites" section between Stationary stations and Path
profiles lists the candidate rover locations the mission scores
against, with:
- An add form that takes the same flexible input as station inputs
(callsign, Maidenhead grid, or `lat,lon`); on submit it creates a
global :good rover-location and re-runs the path matrix.
- A per-row trash button visible to the location's owner / admins
that deletes the location and re-runs the matrix.
`RoverPlanning.candidate_rover_locations/1` is now public so the show
page can list exactly what the worker enqueues against. Add/remove
both call `replace_mission_paths/1` so the matrix stays consistent
with the rover-site set.
Anonymous visitors see a sign-in prompt instead of the form.
Renames the rover-location status enum and every label that referenced
it. Existing rows are migrated in place. Also touches the consumers:
- Rover.Location enum + default
- RoverLocationsLive index, status filter, form, show, badges
- Rover.Compute and rover_live (only_ideal_locations → only_good_locations)
- RoverPlanning context candidate filter (status == :good)
- RoverPlanning.Show / Form copy
- All rover-related tests
Adds a new globally-scoped, owner-mutable rover-mission tracker:
- /rover-planning paginated table (LiveTable) with View/Edit/Delete
- /rover-planning/new + /:id/edit form: name, band, antenna heights,
notes, "only check against known good locations" toggle (default on),
and a dynamic list of stationary stations entered as callsigns,
Maidenhead grids, or lat,lon pairs (Station changeset geocodes via
LocationResolver, lat/lon stays editable after add)
- /rover-planning/:id show page renders the station list, scope, and a
matrix of computed path profiles (distance, min clearance, diffraction,
verdict) populated as the worker completes each pairing
After save, RoverPlanning enqueues one RoverPathProfileWorker job per
(rover-location × station) pairing. The worker mirrors PathLive's
synchronous compute (ElevationClient + TerrainAnalysis at the mission's
band + heights) and stores the result on the matching path row.
PubSub broadcast on completion lets the show page live-refresh.
Admins can edit/delete any mission; owners can edit their own.
Rover.update_location/3 and delete_location/2 now resolve any record
when the actor is_admin. The index + show LiveViews surface Edit/Delete
on every row for admin users, matching the existing owner UX.
- Extract PathParser.valid_callsign?/1 and reuse from the calibration
Mix task instead of duplicating the regex.
- Document raise behavior on Aprs.recent_packets_with_paths/1 and
Aprs.station_positions/1 (already noted in moduledoc; @doc reinforces
for callers reading the function head).
- Note in the BandConfig guard comment why it sits before the Oban pause
(so a misconfig exits without leaving queues paused).
- Reword the band-mismatched-negatives note as a v1 limitation (no TODO
tag — codebase convention is zero TODOs).
- decode_packet_row no longer guards Decimal.to_float / DateTime.from_naive
with passthrough clauses — Postgrex returns Decimal for numeric and
NaiveDateTime for timestamp without time zone, so the WHERE clause
filters out the only path that could have hit the alternate branches.
- recent_packets_with_paths shifts :since to UTC before to_naive so a
caller passing a non-UTC %DateTime{} doesn't silently match the wrong
window.
- moduledoc now documents that the module raises on connection/SQL
errors and is unsafe in async contexts without a catch.
Q-construct now matches the explicit APRS-IS set (qAC/qAX/qAU/qAo/qAO/
qAS/qAr/qAR/qAZ/qAI) instead of any qA[A-Za-z]. WIDE/TRACE n-N now allows
n and N up to 7 per the APRS New-N paradigm (was [12]).
Three spec-compliance fixes to PathParser:
1. Q-construct regex now accepts lowercase letters (qAo, qAr, etc.).
The moduledoc lists qAo as a Q-construct to skip but the previous
~r/^qA[A-Z]$/ rejected it, leaving it to be misclassified.
2. self_loop?/2 now compares callsigns only; the spurious
src_pos == dst_pos clause would falsely suppress two distinct
stations that happen to share coarse coordinates (e.g. same building).
3. Strengthen the "lookup returned nil → drop hop, do NOT advance source"
test: extend the path to three tokens so the assertion actually proves
the third hop chains from the previous good source rather than from
the dropped one. The two-token version passed regardless of behavior.
Self-loop fixture updated to trigger via callsign equality (the only
remaining mechanism) instead of position equality.
Pure-Elixir parser that walks an APRS digipeater path left-to-right and
emits verified hops for callsigns carrying the used flag (`*`). Filters
Q-constructs, TCPIP/TCPXX, and routing aliases (WIDE/TRACE/RELAY/etc.);
handles anonymous-digi cases, missing station-position lookups, and
self-loops without advancing source past unattributable hops.
Used by upcoming 144 MHz calibration to attribute aprs.me receives to
real digipeater geometry without persisting any APRS data on our side.
Wire up Microwaveprop.AprsRepo as an optional secondary Ecto repo that
points at the sister aprs.me Postgres for read-only access. This is the
foundation for upcoming 144 MHz calibration work that uses verified APRS
RF hops as ground truth.
* lib/microwaveprop/aprs_repo.ex declares a read_only repo; no schemas,
no migrations, no writes — Microwaveprop never persists APRS data.
* Application supervisor starts the repo only when configured (dev/test
via config blocks, prod via APRS_DATABASE_URL). A missing config logs
a warning and skips startup so the main app still boots.
* config/dev.exs + config/test.exs add local aprsme_dev / aprsme_test
endpoints. Test pool uses Ecto SQL Sandbox; tests that don't touch
APRS data don't need to check the connection out.
* config/runtime.exs guards the prod block on APRS_DATABASE_URL —
unset env logs a warning and leaves the repo unconfigured.
