Adds the test that verifies the full producer→drain→re-pass loop:
sample written with NULL HRRR fields + fetch task enqueued, then once
an HRRR profile lands, the next sampler pass UPSERTs the same
calibration_samples row (id preserved) with non-NULL weather data.
Also tightens dialyzer surface:
* PartitionManager defines a `result()` typedoc and uses it on both
public functions instead of inlined tuple types.
* PartitionMaintenanceWorker.perform/1 + the lookahead/1 helper get
explicit specs.
* CalibrationSampler.enqueue_missing_hrrr/3 gets a spec covering its
group_by-shape input map and HRRR profile list.
`mix precommit` clean (3310 tests, 0 failures). Local `mix dialyzer`
hits an OTP 28.4 vs 28.5 toolchain mismatch unrelated to this change.
Partition list for hrrr_profiles + hrdps_profiles was hand-edited in
priv/repo/structure.sql; if no human extended it, writes for the next
quarter would silently fail at the worker level (Rust hrrr-point-worker
marks tasks failed; the calibration pipeline silently joins NULLs).
Adds:
* Microwaveprop.PartitionManager — runtime DDL helper. Quarter math via
a single integer index (year*4 + quarter) for rollover safety.
Coverage check via pg_inherits before CREATE so a quarterly target
inside an existing wider partition (some 2027 ones are half-year)
is treated as :exists rather than colliding.
* Microwaveprop.Workers.PartitionMaintenanceWorker — daily cron at
03:15 UTC, 4-quarter lookahead. Idempotent; cheap when nothing's
missing. lookback_quarters arg lets ops widen for catch-up.
Tests cover the create + idempotent re-run + name format + year
rollover paths via a synthetic parent created inside the sandbox
transaction (no DDL leaks into the shared test DB).
The CalibrationSampler joined PSKR midpoints against hrrr_profiles via
nearest-within-window, but post-Phase-3 Stream C nothing bulk-populates
the grid — hrrr_profiles only sees per-QSO point fetches from the Rust
hrrr-point-worker. Result: 0 / 16,042 prod samples had HRRR data.
Producer pattern: when a cell has no profile in the lookahead window,
enqueue a row into hrrr_fetch_tasks at the cell's HRRR-grid-rounded
midpoint. The Rust worker drains it; the next 5-min rolling-window
pass UPSERTs the same sample with non-NULL weather fields. Idempotent
with the existing on-conflict design — no extra worker, no churn on
cells already covered, multiple bands sharing a midpoint dedupe to a
single fetch via Enum.uniq + jsonb point-union in HrrrPointEnqueuer.
The polylines were sourcing from --band-* CSS vars (a separate, theme-aware
design palette) while the legend swatches rendered from the Elixir @band_colors
map, so the two diverged for every band — e.g. 10 GHz showed lime on the map
and emerald in the legend. Emit `data-band-color` on each band checkbox and
have the JS hook read from there, making the legend the single source of truth.
Drops the now-redundant theme observer / restyleLines plumbing.
The chart is a public-facing seasonal summary, not a personal
view, so private contacts should roll into the totals like any
other. Drop the visible_query scope from monthly_bars and query
Contact directly.
Aggregates contact counts by calendar month summed across every
year, rendered as an inline SVG bar chart above the table. Months
with no contacts render as a 0-height tick so the x-axis stays
continuous at 12 bars.
Respects the same scope visibility the table uses, so private
contacts a viewer can't see don't leak into the chart counts.
Each <rect> carries data-month / data-month-count for testability
and a <title> for hover.
A 4-char locator covers ~70 km × 100 km, which dwarfs HRRR's 3 km
native cell and any midpoint we'd derive for the calibration corpus.
Storing those samples gives the recalibrator geometry that's
indistinguishable from noise.
fetch_grid/2 now returns {:error, {:short_grid, key, grid}} for any
locator under 6 chars after Maidenhead validation. The spot is
dropped before it reaches the aggregator, so neither pskr_spots_hourly
nor pskr_calibration_samples ever see it.
Old design fired hourly at HH:25 and built one whole hour of cells
in a single batch. During PSKR peaks (50k+ spots/hour) that batch
held the DB pool long enough to look like a backlog, and a pod
restart between fires lost a full hour because the next cron only
caught the most-recent-prev hour.
- Default window is now current hour + 1 prior. Every 5-min fire
spreads work across the in-flight hour and always covers the
boundary without waiting for the next cron tick.
- lookback_hours arg widens the window for catch-up
(e.g. {"lookback_hours": 24} sweeps the last day after an
outage).
- Unique period 3600 → 240 so consecutive 5-min cron fires actually
slot in.
UPSERT is already idempotent so the ~12 redundant passes per hour
cost only the read+merge. Latency from spot ingestion to sample
creation drops from up-to-60min to ≤5min.
Replaces the "tropo composite + post-hoc aurora boost" model with a
max-over-mechanisms dispatcher so each band's score reflects the
dominant propagation channel rather than summing alternative paths.
- BandConfig.mechanisms/1 derives the stack from frequency:
≤432 MHz gets [:tropo, :aurora]; >432 MHz stays [:tropo]. Bands
may override with an explicit :mechanisms field.
- Scorer.mechanism_score/3 returns a standalone integer score for
one mechanism (:tropo delegates to composite_score; :aurora is a
clean Kp/freq function, not a boost on top of tropo).
- Scorer.cell_score/2 takes max across the band's mechanisms so a
Kp-6 storm scores 6m at 85 even when the tropo picture is poor,
instead of conflating the two channels.
Es and F2 stay in PathCompute since they're path-distance-dependent.
The legacy aurora_boost path is unchanged so no callers shift today.
`Pskr.Recalibrator.run/0` reads `pskr_calibration_samples`, bins
each sample per (band × feature), and writes spot-density stats to
`pskr_feature_bins` so an operator can read whether a feature
actually discriminates propagation at the threshold granularity
the scorer uses.
Bins, not regression: `BandConfig` already encodes scoring as
discrete thresholds, so the bin output matches that shape and an
operator can copy adjusted thresholds directly without translating
from regression coefficients.
Self-healing: corpus too thin ⇒ run row written with status
`skipped_insufficient_data` and the analysis is a no-op until next
fire. `min_total_samples = 1000` (≈ 4-5 days of CONUS PSKR
activity); per-band threshold is 100. Both surface in the run row's
`notes`.
Auto-applies nothing. Weight changes still go through human review
of `BandConfig.@band_configs` and a code commit. The recalibrator
is a read-only analyst that stays out of the production scoring
path.
Features binned (matching the scorer's discriminating fields):
* pwat_mm — humidity U-shape candidate
* hpbl_m — boundary layer (mechanism vs scoring re-eval)
* min_refractivity_gradient — refractivity threshold validation
* surface_pressure_mb — pressure-front proxy
* kp_index — aurora boost magnitude tuning
Schema: two tables.
* `pskr_recalibration_runs` — one row per fire with corpus
stats, status, notes
* `pskr_feature_bins` — one row per (run, band, feature, bin)
with sample_count, spot_count_total/avg/p50/p90
Cron: `0 4 * * 0` (Sundays 04:00 UTC, off-peak, post-climatology).
Manual reruns enqueue with no args.
Tests cover the empty-corpus skip path, sub-threshold totals,
per-band threshold gating, the actual bin emission, nil-feature
handling, spot-count averaging, and the always-records-a-run
audit invariant. 8 new tests, 3282 total passing.
Backfill pipeline untouched.
PSK Reporter is now an always-on data feed, so the calibration
corpus that recalibration will eventually train against can grow
hourly from when the firehose started. One row per (hour, band,
0.125° midpoint cell) joins three sources:
* `pskr_spots_hourly` — observed spot density (truth signal)
* `hrrr_profiles` nearest match at the cell midpoint at hour
boundary (atmospheric features: T, Td, PWAT, P, dN/dh, HPBL,
ducting flag)
* `geomagnetic_observations` latest Kp at the hour (space weather)
Predicted scores are intentionally NOT stored — they're a function
of the algorithm version under evaluation. Storing only features
keeps the corpus stable across every weight refit; the recalibrator
computes predictions on demand.
Components:
* Migration `20260504210756_create_pskr_calibration_samples` —
new table + unique index on (hour, band, midpoint_lat, lon),
plus a midpoint spatial index on `pskr_spots_hourly` to keep
the join cheap.
* `Pskr.CalibrationSample` — schema mirror of the table with
the same `(hour, band, midpoint_lat, lon)` unique constraint.
* `Pskr.CalibrationSampler.build_for_hour/1` — pulls all spots
for the hour, snaps midpoints to the propagation grid (0.125°),
builds an in-memory HRRR index over a ±0.07°/±60 min window,
and bulk-upserts samples. Idempotent — re-runs upsert.
* `Workers.PskrCalibrationWorker` — Oban cron entry on the
`:backfill_enqueue` queue. Default args target the previous
full hour; explicit `hour_utc` arg reruns any hour.
* Cron `25 * * * *` — fires past HRRR analysis publish window
(~HH:50→HH:05) and PSKR aggregator's 60s flush.
Forward-only: PSKR has no historical archive, so the corpus only
grows from when the feed started. Recalibration weight refits
should wait for ~30 days / ~10k samples to cover diurnal and
synoptic variability.
Tests cover cell-snapping, HRRR feature joining, Kp stamping,
median-distance aggregation, mode dedup, idempotent reruns, and
the worker's default-hour and explicit-hour args (12 new tests,
all passing).
Backfill pipeline untouched — none of these changes feed into
contact enrichment.
`Scorer.aurora_boost(score, kp, band_config)` is a terminal boost
applied after `composite_score` in the same shape as the existing
`commercial_link_boost`. It tracks the NOAA G-scale (Kp 5+ opens
auroral E-region paths up to ~2500 km) and is freq-gated to
≤ 432 MHz — auroral propagation is observed almost exclusively on
50/144/222, occasionally on 432, and never at microwave.
Boost magnitudes:
* Kp < 4 (quiet) — pass-through
* Kp 4 — +5
* Kp 5 (G1) — +15
* Kp 6 (G2) — +25
* Kp ≥ 7 (G3+) — +35, clamped to 100
Wired into `Propagation.score_grid_point/4` via the existing
`hrrr_profile` map: callers populate `:kp_index` once per cycle so
the per-grid-point band loop never queries the DB. The UI fallback
path (`factors_from_profile/5`) sources Kp from
`SpaceWeather.latest_kp/0` directly. Other call sites
(GEFS forecast, path-compute, contact-show) can opt in by adding
`:kp_index` to their profile map.
Diagnostic factor `:kp_index` is stashed on VHF/low-UHF results so
the operator-facing factor breakdown can show why a band scored
above its quiet-conditions baseline. Microwave bands don't carry
the diagnostic — they're never affected by the boost.
Refactored `score_with_algorithm/7` to delegate band-level
finalization to `finalize_band_result/5`, keeping the parent
function under credo's cyclomatic-complexity limit.
NOTE: this only affects the Elixir scoring path — the live grid
served by `prop_grid_rs` (Rust) does its own scoring and does not
yet honor Kp. Adding aurora to the Rust side is follow-up work.
Both RTMA and METAR5 schemas + clients + workers were defined but
never had a consumer. `find_nearest_rtma/3` and `recent_surface_obs/3`
existed in `Microwaveprop.Weather` with zero callers cluster-wide;
RtmaFetchWorker had test coverage but was never enqueued anywhere;
no IEM 5-min ASOS fetcher was ever written. The tables sat empty in
prod and the recalibrate audit was permanently flagging them BROKEN
even though the empty state was the steady state.
Drop the lot — schemas, clients, workers, queue config, Req.Test
stubs, audit checks, and the per-table unique tests in the
observation-changesets suite. New migration drops the now-unreferenced
`rtma_observations` and `metar_5min_observations` tables (both 0
rows in prod). Trim the `:rtma` slot out of `backfill_only_queues`
in runtime.exs so Oban Pro's Smart engine stops trying to manage a
queue with no producer.
Audit thresholds reset on the surviving NARR check: drop the
"WARN < 5000 absolute rows" rule that didn't account for
NarrFetchWorker's 0.13° spatial dedup, and point remediation at
the existing BackfillEnqueueWorker cron instead of the dev-only
`mix narr.backfill` task.
scripts/recalibrate_algo.py: ROW_COUNT_SOURCES + DATA_GAP_SQL +
DATA_GAP_RULES all stop referencing the dropped tables.
Pin both the sender and receiver DXCC slots to 291 so the broker
only forwards CONUS↔CONUS spots. The previous two-topic-per-band
filter (sender_us OR receiver_us) was leaking US↔CA paths through
the receiver-side wildcard, and the project doesn't have NWP
coverage for the non-US end yet.
When HRDPS lights up we'll add a Canada-anchored topic alongside.
Spots that include subsquare (EM12kl), extended-square (EM12kl37),
or extended-extended-subsquare (EM12kl37ab) precision now keep
that resolution end-to-end. Previously we truncated everything to
4 chars at parse time, which collapsed distinct paths into the same
row whenever two spotters shared a field.
The aggregation key uses the full grid as reported, so two
spotters in different subsquares of `EM12` produce two rows —
correct, since they are physically different paths. Any spot that
fails Maidenhead validation (odd length, illegal char at position
N) is still dropped via the same error path.
Storage normalizes to uppercase so the unique index doesn't split
on case. Maidenhead.valid?/1 already accepts both cases on input.
`:gen_tcp.recv/3` requires a passive socket and returns `:einval` if
the socket is in any active mode. We were opening with
`active: :once`, sending CONNECT, then calling `:gen_tcp.recv` to
read CONNACK — which kernel-rejected every time.
The visible symptom was "Pskr.Client connect failed: :einval" every
30s on the elected leader, while a manual `:gen_tcp.connect` from
`bin/microwaveprop rpc` worked because it never tried to recv. Each
failed attempt also leaked a port (connect succeeded; recv failed
later), and we'd accumulated ~27 dangling sockets on the leader
process.
Open the socket in `active: false`, finish the synchronous
CONNECT/CONNACK/SUBSCRIBE handshake with `:gen_tcp.recv`, then
`:inet.setopts(active: :once)` so PUBLISHes arrive as
`{:tcp, socket, data}` messages going forward. Also close the
socket on any handshake failure so retries stop leaking ports.
Two related changes to the contact detail page:
1. The "Flagged invalid" badge was wrapping to two lines on
narrow widths because the daisyUI badge has no explicit
no-wrap class. Add `whitespace-nowrap`, drop the redundant
capitalization on "Invalid", and append the flagger's
callsign so the badge reads "Flagged invalid by W5XYZ".
Hover tooltip carries the timestamp.
2. New `flagged_by_user_id` + `flagged_at` columns on contacts.
`Radio.toggle_flagged_invalid!/2` now takes the admin User
doing the flagging; on unflag it clears both fields so a
subsequent re-flag gets a fresh attribution and the badge
never shows a stale flagger.
3. `Radio.delete_contact!/1` + a Delete button next to Flag.
Admin-only, gated in both the event handler and the template.
Uses LV's `data-confirm` for the destructive-action prompt.
All FK references to `contacts` already declare
`on_delete: :delete_all` (terrain_profiles, contact_edits,
contact_common_volume_radar) so a single Repo.delete!
cascades cleanly.
K8s pods have IPv4-only egress; without :inet in the gen_tcp opts,
BEAM was picking mqtt.pskreporter.info's AAAA record and the
kernel's connect() returned EINVAL, surfacing as :einval and a
30-second retry loop.
emqtt's transitive `quicer` dep needs CMake + msquic submodule
clones to build, which the slim Debian builder image doesn't ship —
the prod CI build was failing inside `mix deps.compile`. Tortoise311
is the obvious pure-Elixir alternative but it bundles a
gen_state_machine + retry/inflight stack we don't use because we're
subscribe-only at QoS 0.
Implementing the wire protocol inline is ~150 lines in
`Microwaveprop.Pskr.Mqtt` (CONNECT/SUBSCRIBE/PINGREQ/DISCONNECT
builders, CONNACK/PUBLISH/SUBACK/PINGRESP parser, varint codec).
Pskr.Client now uses :gen_tcp with `active: :once` framing,
buffers partial reads, schedules its own keepalive, and parses
inbound packets in a tight loop.
Drops `BUILD_WITHOUT_QUIC=1` from the Dockerfile — no longer
relevant since there's no native dep at all in the build chain
now.
Subscribes to mqtt.pskreporter.info:1883 over plain TCP and folds
every reception report into hourly aggregates per
(band, sender_grid_4, receiver_grid_4). Each spot is the empirical
"propagation actually occurred on this path right now" signal we'll
correlate against HRRR atmospheric state to recalibrate the scoring
weights. Aggregating at the path-hour bucket collapses 50-reporter
QSOs to one row instead of 50.
Topic filter anchors on USA (DXCC 291) on either sender or receiver
side so the broker pre-filters out everything outside our HRRR
domain. Bands subscribed: VHF and up (6m, 2m, 70cm, 23cm, +
microwave) — HF is dominated by ionospheric propagation, which this
project doesn't model.
Pskr.Client cluster-elects a singleton via :global.register_name —
all replicas start the GenServer but only one connects to MQTT;
the rest stay in :standby and watch for the leader's nodedown to
re-run election. Off in dev/test, on by default in prod
(PSKR_MQTT_ENABLED=false as kill-switch).
Pskr.Aggregator buffers in memory keyed by Pskr.path_key/1 and
flushes every 60s via Repo.insert_all/3 with an additive
ON CONFLICT (count summed, SNR envelope by GREATEST/LEAST,
modes unioned via unnest+DISTINCT). Idempotent across overlapping
flushes and across leader handover.
Dockerfile sets BUILD_WITHOUT_QUIC=1 — emqtt's transitive quicer
dep wants CMake + OpenSSL headers we'd otherwise have to add to
the builder image just to never use QUIC over plain MQTT. Base
image is unchanged; the new dep compiles cleanly into the existing
prop-base runtime.
phoenix_pubsub_redis 3.1's deprecation message ('Move :url inside the
:redis_opts option') is misleading — wrapping the URL as
`redis_opts: [url: url]` forwards the keyword list to Redix, which
rejects `:url` with `unknown options [:url], valid options are:
[:host, :port, :password, ...]`. New pods on commit b4d6b04 onward
crashed at startup with that NimbleOptions.ValidationError; old pods
running e53a34d (top-level :url) coasted on already-established
connections so the breakage only surfaced when fresh pods rolled.
The library accepts `redis_opts` as either a URL binary OR a
keyword list. Pass the URL as a bare string so it takes the
binary-clause path that hands it directly to Redix.PubSub.start_link/1.
The library deprecated top-level Redis connection keys; passing :url
directly logged a warning on every boot. Functionally identical, just
silences the warning.
Switches the PubSub adapter to Phoenix.PubSub.Redis backed by the
shared Valkey instance (same backend GridCache uses), via the new
phoenix_pubsub_redis dep. Falls back to the default PG2 / Erlang-
distribution adapter when VALKEY_URL is unset, so dev/test boot
unchanged.
Decouples PubSub fan-out from libcluster's BEAM connection. A
flapping Erlang distribution (the OOM-cascade scenario from
2026-05-03 surfaced this — libcluster warnings dominated the logs
while pods CrashLoopBackOff'd) no longer takes broadcasts with it,
so the rest of the cluster keeps publishing while the bad pod
recovers.
node_name uses POD_IP for cluster-wide uniqueness, falling back to
Erlang's node() name in non-K8s environments. Required by the Redis
adapter so loopback filtering doesn't drop a pod's own broadcasts as
if they came from a peer.
Each cached HRRR valid_time is ~32 MiB compressed × cap of 8 = up to
256 MiB per pod. With 4 hot pods + 1 backfill that's ~1.25 GiB of
cluster RAM holding identical content. Replacing the per-pod ETS
replicas with a single shared Valkey copy collapses that to ~256 MiB
total (off-pod) and makes the BEAM heap on each replica meaningfully
smaller — directly addresses the headroom the 2026-05-03 OOM cascade
ate into.
* New `Microwaveprop.Valkey`: single Redix connection (named conn,
sync_connect: false, exit_on_disconnection: false). `start_link/0`
returns `:ignore` when VALKEY_URL is unset so dev/test boot without
Valkey. Helpers wrap GET/MGET/SET-with-TTL/ZADD/ZREVRANGE/SCAN with
`:erlang.term_to_binary` round-tripping under the safe atom flag.
* `Microwaveprop.Weather.GridCache` now branches on
`Valkey.configured?/0`. Public API is identical so callers (Weather,
GridCachePruneWorker, MapLive) don't change. ETS path is preserved
unchanged for dev/test.
* Storage layout when on Valkey:
prop:wg:vts ZSET of valid_time iso8601 strings
prop:wg:<vt>:<lat>:<lon> one chunk per 5°×5° spatial bucket
fetch_bounds MGETs only the chunks intersecting the viewport, so a
regional view is 1-4 round-trips instead of pulling the full grid.
* Per-key TTL = 8 h (matches the ETS valid_time cap). Eviction is
automatic; prune_keep_latest is a no-op on Valkey, prune_older_than
uses ZRANGEBYSCORE + DEL.
* All Valkey errors fall back to disk (ScalarFile.read_bounds), logged
as warnings — page works through Valkey outages, no user-visible
break beyond a one-shot latency bump.
ScoreCache, Microwaveprop.Cache, NexradCache, telemetry/Postgres
protocol/libcluster registry tables remain ETS — they're sub-ms
hot-path lookups where a network hop would be felt.
VALKEY_URL must be present in prop-secrets for production rollout
(applied out-of-band, secret.yaml is git-ignored).
Found while auditing pod memory: :weather_grid_cache is by far the
largest ETS table (~32 MiB compressed per entry, vs ~2 MiB/entry for
ScoreCache) because each cell is a 22-field weather row instead of a
scalar score. Cap of 24 meant the cache could hoard up to ~768 MiB on
a single pod, and hot pods sit at 3-5 GiB against a 6 GiB limit — most
of that headroom belongs to GridCache.
The cap was sized for 'analysis + 18 forecasts + stragglers' but the
real access pattern is current-hour + a short forward scrub. Beyond
that, fall back to ScalarFile.read_bounds (sub-100 ms per file) instead
of permanently parking ~500 MiB on data nobody looks at.
Worst-case GridCache spend drops 768 MiB → 256 MiB. Steady-state win
is smaller (most pods don't fill all 24 slots), but bounds the tail.
Each path-table cell now wraps its content in <.link navigate={...}>,
so the row behaves as a native anchor: hovering shows the destination
URL in the browser status bar, right-click 'open in new tab' works,
middle-click works, and the rendered HTML is meaningful to assistive
tech / scrapers / link-validators.
Replaces the phx-click={JS.navigate(...)} pattern, which gave none of
those affordances and required the user to trust that the row was
clickable. Whole-row hover highlight is preserved via a CSS
:has(a:hover) selector so cell-hover still tints the row.
The /rover-planning index was rendering only band_mhz (the legacy
primary single-band column), so multi-band missions with bands_mhz
populated showed just the first band. Switched the renderer to
arity-2 and now pulls the full bands list via Mission.bands/1, so
multi-band missions render '10 GHz · 24 GHz · 47 GHz' etc. Sort
still keys on band_mhz so the column stays sortable on a single
scalar value.
The default 'We can't find the internet' wording blames the user's
connection, but the typical cause on this site is a server-side issue
(deploy in progress, pod OOM, ingress hiccup). Server-specific wording
matches reality and points users at the right thing to wait on.
The :terrain queue at 3 slots/pod was sized for the lightweight per-QSO
TerrainProfileWorker, but RoverPathProfileWorker now runs the full
PathCompute pipeline (terrain + 9 HRRR profiles + sounding + ionosphere
+ scoring + loss + forecast) — same heap profile as :propagation, which
sits at 1 slot precisely to avoid OOM.
A backfill_paths flood today queued ~200 rover-path jobs. With 4 hot
pods × 3 slots, the cluster stacked 15 concurrent path-computes and
two pods OOM-killed in a loop, taking the libcluster ring with them
(visible as the "unable to connect" warnings on the surviving pods).
New :rover_path queue at 1 slot/pod caps cluster-wide concurrency at
(hot_replicas + 1 backfill), drains the existing 200-job backlog
inside ~30 min, and keeps :terrain free for the cheap workers it was
sized for.
Existing Path rows persisted before the PathCompute extraction shipped
have no "term" key in their result map, so the live /path page hits
the :stale_term branch and falls back to live recompute when the
operator clicks a row. The hourly backfill cron now flips those
:complete rows back to :pending so the path-profile worker rerolls
them with the new full-output term, after which row clicks render
from cache instantly. The flip uses Postgres's jsonb_exists() so it's
a single targeted UPDATE instead of pulling rows into Elixir.
Two stacked wins for the form-submit hot path:
1) create_mission now hands path-matrix population to the same async
reconcile worker update_mission uses, so the request returns
right after the Mission row insert. For a mission with hundreds
of (rover x station x band) tuples this used to block the LiveView
for many seconds.
2) Inside reconcile/enqueue_paths_for, the per-pair Multi.insert
loop is replaced with one Repo.insert_all for the Path rows and
one Oban.insert_all for the worker jobs. N round-trips collapse
to two regardless of pair count.
Three connected changes:
1) Extract PathLive.compute_path/4 + every helper it owned (resolve,
profile grid lookup, sounding/ionosphere readouts, scoring, loss /
power budgets) into Microwaveprop.Propagation.PathCompute. PathLive
now delegates; ~410 lines of dead helpers deleted from PathLive.
2) RoverPathProfileWorker calls PathCompute.compute/4 with the
mission's heights and PathLive's default station params (10 W TX,
30 dBi gains). Stores the full atom-keyed compute output as a
Base64-encoded :erlang.term_to_binary/1 blob alongside the flat
summary fields the rover-planning show table reads. The blob
roundtrips structs and DateTime exactly (Jason.encode would lose
them).
3) PathLive accepts ?rover_path_id=UUID. When set, loads the cached
Path, decodes the term (binary_to_term :safe), assigns it as
@result, and renders normally — no compute_path call. The
rover-planning show table now links rows directly to
/path?rover_path_id=UUID, so a click opens the full Path
Calculator UI from cached data without re-running terrain / HRRR /
sounding lookups.
Bonus prod fixes folded in:
- PathShow's elevation chart attribute (data-* → data-profile JSON)
was crashing the JS hook with 'unexpected character at line 1'.
- Station.changeset now wipes previously-resolved callsign/grid/
lat/lon when the user types into :input — typing 'AA5' early
resolved to a wrong location and locked it; subsequent keystrokes
never re-resolved.
- phx-debounce=600 on the station input so QRZ doesn't get hit on
every keystroke.
Worker now stores the full per-point analysis (beam height,
Fresnel-zone radius, clearance) on Path.result.path_points, and
PathShow renders a real cached page: terrain elevation chart fed by
the same ElevationProfile JS hook /path uses (feet on Y, miles on X,
LOS + Fresnel overlay), distance / bearing / clearance / antenna
heights all in feet, baseline link-budget breakdown, and the cached
midpoint propagation score with computed_at timestamp.
A Recompute live button hands off to /path for the time-varying bits
(current HRRR conditions, scoring breakdown, sounding, ionosphere)
that aren't worth caching.
The cached path-detail URL was a thinned-down view (geometry +
loss-budget summary + score). The user wanted the same rich display
/path renders — terrain chart with feet axes + tooltips, conditions,
scoring, link/power budgets, forecast — so PathShow now resolves the
row's rover/station/band/heights and push_navigates to /path with
those params baked in. The cached result map on Path.result is still
what the rover-planning show table reads from for fast row rendering;
this just bridges the row click to the canonical Path Calculator.
The whole-block try/rescue around fetch_stats meant any single failing
query (a missing dev table, a transient pg blip) zeroed all 10
counters. Each count_exact/1 + count_estimate/1 now uses Repo.query/2
and falls back to 0 only for its own column. Bumps the inline 9-factor
prose to 10-factor and pulls the contacts headline from the live count
instead of a stale '58k+' string.
Adds a CLAUDE.md note prompting future updates to refresh /about's
prose when factor counts, data sources, or schema scale change.
New /rover-planning/:id/paths/:path_id route renders the worker-stored
result map directly: geometry, link-budget components, propagation
score, computed_at. The 'Recompute live' button hands off to /path
when the operator wants HRRR-driven loss against current weather.
Two fixes:
1) Form save / rover-site add/remove no longer runs the path-matrix
reconcile inline. update_mission and the show LiveView now
enqueue a single RoverMissionReconcileWorker job and return —
for missions with many rover sites x bands the inline path was
firing hundreds of Multi inserts + Oban.inserts on the request.
2) RoverPathProfileWorker.load_path/1 now strictly matches a
4-tuple (mission_id, rover_location_id, station_id, band_mhz)
with band_mhz as an integer, plus updates the Oban unique key
set to include band_mhz so multi-band missions enqueue per-band
jobs (not just one). Legacy/malformed args are logged and
dropped instead of crashing the queue with MultipleResultsError.
Clicking the trash icon on a stationary station in :edit mode before
any other phx-change had fired emptied the form's stations params,
which combined with cast_assoc(on_replace: :delete) wiped EVERY
existing station instead of just the one the user clicked. Fall back
to rebuilding the params from the persisted mission's stations when
the form's current params don't already carry them, then delete only
the targeted index.
Mission now carries bands_mhz ({:array, :integer}) — operator picks
one or more bands as multi-checkboxes. enqueue_paths_for builds the
cross product (rover x station x band) and persists each tuple as its
own Path row keyed by (mission_id, rover_location_id, station_id,
band_mhz). The path-profile worker reads band_mhz from the path
itself (legacy single-band jobs without band_mhz in args still resolve
to their unique row).
replace_mission_paths/1 is now a thin alias for reconcile_mission_paths/1
which diffs desired vs actual: stale tuples (old band that the user
unchecked, station they removed, rover-site they deleted) get dropped,
new tuples become :pending and enqueue, surviving :complete rows are
left in place — no more wholesale destruction of already-computed
paths on every edit.
The show table gains a Band column, and band_label() in the mission
summary becomes bands_label() (joins the list with commas).
The add-rover-site form now resolves the input on every change and, if
any existing rover-location shares the same 6-char Maidenhead cell
(~5 km), surfaces a 'Already saved as' link to that location's detail
page. Avoids users accidentally duplicating a site they (or another
operator) already added.
The path-profile worker now also computes the baseline link-loss
budget — free-space path loss, oxygen absorption, baseline H2O loss
(at 7.5 g/m^3 default humidity), plus the already-cached terrain
diffraction — and stores all four numbers in Path.result. The show
page renders the per-row total in a new Loss column. /path still
recomputes against live HRRR weather; this is the offline-readable
cached snapshot.
The 'Rover sites' card no longer renders the long list of every site —
each rover-location now appears once as a path-profile group heading,
which carries its own delete button next to the n/m progress text. The
add-site form stays in place.
Adds RoverPlanning.backfill_paths/0 that walks every mission and
re-enqueues RoverPathProfileWorker jobs for any (rover × station) pair
whose Path is missing or not :complete. The path-profile worker
short-circuits on :complete, so this is idempotent.
Wires a new RoverMissionBackfillWorker into both dev (config.exs) and
prod (runtime.exs) crons at HH:30 — heals missions stuck in :pending
or :failed from transient elevation-API errors or interrupted
create_mission enqueues without operator action.
The Stationary-stations summary was rendering whatever the user typed
verbatim ('aa5c 33.1889, -96.4517'). Switch to a pair of helpers that
uppercase the callsign (covering legacy mixed-case rows) and synthesize
a 6-char Maidenhead grid from lat/lon when no grid was stored. Bare
coordinates no longer appear in this list — the grid carries the same
location info in a more amateur-radio-native form.
The path-profile worker now also looks up the propagation grid score for
the path midpoint at the mission's band before flipping status to
:complete, so the row never appears 'done' until both terrain and
propagation prediction have run. The show table renders that score as a
red/yellow/green badge in a new Propagation column.
Each rover-location group heading is now a link to /rover-locations/:id
so the user can jump straight to the spot's detail page from the
mission view.
Path table now shows callsign on top with grid muted below, and the
detail-only Min clearance / Diffraction columns are gone. The verdict
badge matched lowercase 'clear/blocked/marginal' but the worker stores
uppercase 'CLEAR/BLOCKED/FRESNEL_PARTIAL/FRESNEL_MINOR' from
TerrainAnalysis, so every cell silently rendered empty. Updated the
guard clauses to the actual upstream strings and added Fresnel variants.
