Two related fixes:
- Drop the global contact_enrichment:{hrrr,weather,solar} subscriptions.
These topics fire for every QSO's enrichment completion, and the
handlers blindly re-queried data for the currently-displayed contact,
causing spurious re-renders whenever any enrichment worker finished
anywhere in the system. The per-contact terrain subscription stays —
that one is correctly scoped.
- Only add a slot to `hydration_pending` when the contact's status
column says the work is genuinely in flight (:queued or :processing).
Previously every connected mount flashed spinners over all eight
enrichment sections, even for contacts whose data was already cached
and about to return instantly from the async task.
Also drop the Donate-to-NTMS footer link.
The antenna-height input only scaled the displayed range labels and
the viewshed max-range — it never fed into the propagation score
itself. Showing the control implied the colored overlay responded to
it, which was misleading. Viewshed now uses a fixed 10 m baseline,
matching the previous 33 ft default.
Three improvements to the contact detail page:
* Track async hydration per source in `hydration_pending` so the
Terrain / Soundings / Atmospheric Profile sections render a loading
spinner while their task is in-flight instead of briefly flashing
"no data available" before the payload arrives.
* Propagation analysis summary now names the origin station by
callsign ("Path is terrain-obstructed at N mi from W5XD") instead
of the generic "station 1" placeholder.
* Elevation profile chart labels the 0-mi and max-mi ticks with the
matching callsign so it's obvious which end of the profile is
which station.
Propagation analysis panel now adds a "Tropo duct detected at N/M HRRR
samples along the path" line whenever any HRRR sample flags ducting,
even if the overall mechanism classification doesn't end up pointing at
the duct (e.g. short paths, clear-LoS contacts). Also rewords the map
caveat to invite users with better coordinates to file a suggested
edit — clickable when logged in, otherwise a login link.
Antenna height now drives typical/extended/exceptional range estimates
and viewshed max range via a sqrt(h_m/10) factor clamped to [0.6, 2.0].
Baseline 33 ft matches the existing defaults. Changing the height input
now pushes update_band_info and refreshes the clicked point's detail
panel plus viewshed so the range-estimate line and coverage shape track
the user's actual station height.
rover_map_hook and weather_map_hook share the same viewport-scoped
data flow as the propagation map — they hit the same blank-tile bug
after tab hide or socket reconnect. Add reconnected() + visibilitychange
handlers and document the requirement in CLAUDE.md so future hooks that
push map_bounds don't regress.
When the tab was hidden or the LiveView socket dropped, the server-side
bounds/scores assigns reset while the hook's scoreGrid retained its old
viewport data. Tiles Leaflet created or regenerated outside that stale
extent painted blank, leaving the overlay visibly truncated.
Deploy stamp was in Layouts.app's nav but the map page uses a custom
full-screen layout that skips the standard nav, so the info wasn't
visible on the page most users land on. Add a compact
"Deployed 2h ago" under the sidebar's data-timestamp/pipeline-status
block with the full UTC time as a tooltip, matching the shape of
Layouts.deploy_stamp.
Three small corrections now that the Richardson gate is actually
wired into Scorer.score_refractivity/5:
- Fix arity reference /4 → /5 in Part 2c.
- Reconcile the backtest section's "bulk_richardson is dead" finding
with the new gate wiring — they're compatible claims. Richardson
is dead as a standalone ML discriminator (stable and unstable
inversions both produce ducts), but useful as a gate on the
native-duct boost (a duct with high Ri is likely to mix out
before the signal gets through).
- Note that nil Richardson is treated as "no info" so older native
profiles without the column still receive the unconditional boost.
Added 5-arity Scorer.score_refractivity/5 that takes bulk_richardson
alongside best_duct_band_ghz. The existing 1.15× boost now only
applies when Richardson is in the stable regime (< 25) — native-
profile duct cells average 8.7-18.4 for ducting vs 38.3 for
non-ducting, so a duct-band reading under turbulent conditions is
more likely to be torn up by mechanical mixing than to carry a
beyond-LOS signal.
Wiring:
- Weather.nearest_native_duct_info/3 returns {ghz, richardson}; the
bare nearest_native_duct_ghz/3 now delegates.
- PathLive and Propagation.score_with_algorithm/7 both pull the
Richardson value into the conditions map alongside best_duct_band.
- Scorer.composite_score/2 reads conditions[:bulk_richardson] and
passes it into score_refractivity/5.
- Backward compat: 4-arity variant and nil Richardson keep the old
unconditional-boost behaviour so existing callers don't regress.
HRRR caps at f018, so beyond ~18 hours the map and path calculator
go blank. This plan scopes a GFS open-data ingest that runs
alongside HRRR to drive the 18-240 hour forecast window, with a
visible seam on the timeline where the resolution drops from 3 km
to 0.25°.
Scope: GFS 0.25° CONUS subset from AWS S3, four runs/day on the
standard cycle, chained fetch/score workers that mirror the HRRR
pipeline shape, per-source binary score files so HRRR and GFS stay
independent. ECMWF IFS HRES listed as a follow-up once GFS works.
algo.md rewrite:
- Strip retrospective language (was/previously/↑ from/Reverted/prior)
so the doc describes the current algorithm instead of its history.
Parts 2b/2c/2d lose their dated headings.
- Replace ERA5 references with NARR — ERA5 was never populated in
prod; NARR is the real historical-backfill path (1979–2014 via
NCEP / NCEI).
- Fold the sounding ducting-probability table, HPBL binned distance,
and per-band signal summary into the current-state discussion
instead of "changes since last refresh" tables.
- Note the bulk_richardson gating on the native-duct boost (code
follow-up in a separate commit).
Algo page UI: widen .markdown-content from 64rem → 88rem so the
per-band weight + correlation matrices stop overflowing on typical
screens. Wide tables inside markdown content now get their own
horizontal scroll (display: block + overflow-x: auto) rather than
forcing the whole container to scroll.
Path calculator conditions map was missing :latitude and
:best_duct_band_ghz. Scorer read them with bracket-notation so the
behaviour was defensively correct but we were leaving signal on the
floor:
- latitude: midpoint of src/dst so `score_season` picks up the right
latitude-based seasonal shift for northern vs southern paths
- best_duct_band_ghz: queried via new Weather.nearest_native_duct_ghz/3
at the midpoint so score_refractivity/4 applies the 1.15× boost
when a native-resolution duct supports the target band
Recalibrator.fit/1 was serialising ~10k per-contact HRRR lookups
(5000 positives + 5000 negatives, one Weather.find_nearest_hrrr each).
Parallelised both sides with Task.async_stream at 20 concurrency —
well under the prod db pool of 30. Cuts a band recalibration from
~minutes of query round-trips to seconds.
Audit pass found two real issues (most agent-flagged items were false
positives — the hrrr_native_profiles composite index exists,
pwat_mm/bl_depth_m are already in the path conditions map, and the
/1.0 in MechanismClassifier is the standard Elixir int→float coerce).
1. Path calculator's 9-point HRRR sampling was running sequentially,
~9× the latency of a single query. Task.async_stream with
max_concurrency: 9 makes them concurrent so path calc pays roughly
one query's worth of time instead of nine.
2. MechanismClassifyWorker's meteor-shower window used hardcoded 2024
peak dates with `%{date | year: peak.year}` projection. That broke
across year boundaries — a Dec 30 contact couldn't match a Jan 4
Quadrantids peak even in-range of the ±3 day window. Switched to
{month, day} tuples and check the peak projected onto year N-1, N,
N+1 so the window works correctly near Dec 31 / Jan 1.
The last attempt still blew past the CI runner's /var cap because
apt's pre-download free-space check runs against the archives dir,
and the image overlay is where that sits. Moving /var/cache/apt and
/var/lib/apt onto BuildKit cache mounts puts the .deb storage on the
builder's cache volume (outside the image overlay), so the free-space
check runs against whatever disk backs the cache — which on the CI
runner has room.
Requires the `syntax=docker/dockerfile:1.6` directive at the top for
the --mount flag. Also removes /etc/apt/apt.conf.d/docker-clean so
the Debian base's post-invoke hook doesn't wipe the cache mount on
every apt run.
The cdo install pulls ~1 GB of transitive deps (netCDF/HDF5/ecCodes/
Fortran) and was exhausting CI's /var/cache/apt partition. Two fixes:
- `APT::Keep-Downloaded-Packages=false` deletes each .deb immediately
after it's extracted, so the archive cache doesn't retain every
downloaded package through the end of the install.
- Split cdo into its own RUN layer so the base runtime deps finish
installing and clean before cdo's cache starts growing.
The "Plot path to beacon" action now includes `tx_power_dbm` and
`src_gain_dbi` in the /path query string so the path calculator's TX
side emits the same EIRP as the beacon out of the box. The beacon
schema stores `power_mw` as EIRP already (per Beacons.RangeEstimate),
so we take 10·log10(power_mw), split against a default 30 dBi dish,
and clamp the TX power to a 0 dBm floor so very-low-EIRP beacons
produce a sane gain/power split instead of -10 dBm outputs.
Examples:
10 W EIRP (40 dBm) → 30 dBi dish + 10 dBm TX
100 W EIRP (50 dBm) → 30 dBi dish + 20 dBm TX
100 mW EIRP (20 dBm) → 20 dBi + 0 dBm (clamped)
Path calculator now samples nine HRRR grid cells evenly along the
source→destination line (was three: Source/Midpoint/Destination) so
mid-path ducts that endpoint-only queries missed show up in the duct
count. The UI reports "Duct at N / 9 HRRR points" instead of a binary
flag, and the table lists every sample with its percent-of-path label.
Independent RAOB signal: uses the new `Weather.nearest_sounding_to/3`
to find the closest sounding within 300 km / ±3 h of the midpoint. If
the sounding's `ducting_detected` is true we surface a "RAOB <code>
confirms duct" badge — HRRR under-reads thin surface ducts that
soundings resolve, so the two signals together are the right
cross-check for live duct prediction.
Distance formatting: all km values on the path results (top Distance
summary, sounding "km from midpoint") now render as
`123 km (76 mi)`; under 10 km we keep one decimal so near-LOS paths
don't round to zero.
Spatial + temporal nearest-RAOB lookup (defaults to 300 km / ±3 h)
joining weather_stations → soundings. Returns the full sounding row
so callers can read the derived ducting_detected / min_refractivity_
gradient fields that HRRR's pressure-level product systematically
under-reads for thin surface ducts.
Prep for the path-calculator enrichment: a sounding within range at
the path midpoint gives an independent duct signal to cross-check
the nine HRRR samples already taken along the path.
Classifies every contact's likely non-LOS propagation mechanism and
persists the result on contacts.propagation_mechanism. Mechanism is
determined in priority order:
1. user_declared_prop_mode (ADIF PROP_MODE from the operator log)
2. EME — moon-ephemeris check, ≥2m band, >1800 km path
3. aurora — Kp≥5 + high-lat path, 50-432 MHz
4. sporadic-E — foEs × 5 ≥ band_mhz, 400-2500 km path
5. meteor_scatter — ±3 days of a shower peak, VHF/UHF
6. rain_scatter — common-volume radar heavy rain, 5-11 GHz ≤800 km
7. tropo_duct — HRRR native_best_duct ≥ band or ducting_detected
8. line_of_sight — ≤50 km path
9. troposcatter — default
Persisted via MechanismClassifyWorker (queue: :mechanism, unique on
contact_id). Submit-time enqueue path includes :mechanism by default;
BackfillEnqueueWorker cron now handles :mechanism alongside existing
types so prod continuously backfills any contact with
propagation_mechanism_status in (:pending, :queued, :failed). Also
added :radar to the cron's type list so common-volume radar backfill
runs automatically rather than only via `mix radar_backfill`.
New modules:
- Microwaveprop.Propagation.MoonEphemeris — Meeus low-precision moon
position, accuracy ±1° — enough for the mutual-visibility EME test
- Microwaveprop.Propagation.MechanismClassifier — plug-in priority
chain over the evidence map
- Microwaveprop.Workers.MechanismClassifyWorker — assembles inputs
from HRRR / native profiles / common-volume radar / solar_indices /
ionosonde + calls the classifier
ADIF importer now reads PROP_MODE into user_declared_prop_mode so
operator-tagged mechanisms (EME/ES/MS/RS/AS/AUR) become ground truth.
Ran recalibrate_algo.py against the full local prop_dev (81,994 contacts,
18.6M HRRR rows) and derived per-band composite weights for the nine
bands with >=200 matched contacts. Moisture (dewpoint/PWAT/surface N)
is consistently beneficial through 5.76 GHz, reverses at 24 GHz; rain
scales sqrt(rain_k) not linearly so 24 GHz gets 0.215 rain weight
instead of the linear-ratio 0.95. 10 GHz stays as the reference band
(defaults); 47+ GHz inherits defaults (n<200).
- BandConfig.weights/1 returns per-band override or default fallback
- @band_configs carries :weights on 222/432/902/1296/2304/3400/5760/24G
- Recalibrator.compute_factors/3 + fit(band_mhz:) band-aware fitting
- Scorer + ContactLive.Show pass band_config to weights/1
- algo.md Part 2d documents the 2026-04-18 analysis + derivation rule
- scripts/derive_band_weights.py turns correlations into weight maps
- report preserved at docs/algo-reports/2026-04-18-recalibration.md
Add a propagation_run_timings table so the wall-clock duration of each
(run_time, forecast_hour) step is queryable long after the run is over.
Keyed by (run_time, forecast_hour) with a status column that captures
whether the step succeeded or bailed out, and an error string on
failure.
PropagationGridWorker stamps every step (ok and failed) via
Propagation.record_run_timing/1. Timing inserts are wrapped in rescue +
changeset-error handling so the instrumentation can never brick the
chain.
Swap the SSH+scp-from-backup approach for a direct pg_dump against the
prod Postgres using PROP_PROD_DB_URL already in .envrc. Pin the
pg_dump/pg_restore/psql binaries to the @18 series so they match the
prod server version, and stream per-object progress via --verbose on
both tools.
Rename the local dev database from microwaveprop_dev to prop_dev so
the name matches the production database. Add scripts/restore_prod_to_local.sh,
which pulls the latest pg_dump off the backup server, drops the local
prop_dev, recreates it, and pg_restores into it.
Add a "Forgot your password?" flow off the login page. A 24-hour
reset_password token is emailed on request, the landing page lets the
user pick a new password, and all other tokens for the user are
revoked on success.
The request endpoint returns the same flash regardless of whether the
email matches a user so that attackers can't enumerate accounts.
Adopt the handoff design system: navy-slate surfaces with indigo-violet
accent (#6366F1 dark / #5458E3 light), Inter as the UI font, hairline
1px borders, softer corner radii. Add propagation-scale and amateur-band
CSS vars so JS hooks can pull theme-aware colors via getComputedStyle.
Polish: drop table zebra striping for hover-on-base-200 rows with
uppercase small-caps headers, bump .header to text-2xl, tighten nav
spacing with a hairline divider, fix broken text-brand -> text-primary
on the auth screens.
Wire --band-* into contacts_map_hook: resolve once at mount, re-resolve
on data-theme mutation so polylines recolor when the theme toggles.
Previously the map loaded the latest cached valid_time from GridCache,
which can be any +Nh forecast hour the worker has written while walking
f00-f18. Users expect to see current conditions on load, not a forecast
that's ~11 hours out.
Docker final stage bakes BUILD_TIMESTAMP into /app/BUILD_TIMESTAMP via
build-arg (only consumed after mix compile, so earlier layer caching is
preserved). Application.build_timestamp/0 reads the file, falls back to
DEPLOY_TIMESTAMP env, then to compile time. Navbar renders a compact
"Deployed Xm ago" with the full UTC time in the tooltip.
