CDS rejects `reanalysis-era5-pressure-levels` requests that include
dewpoint_temperature because it's only a single-level 2m variable; no
pressure-level dewpoint exists in ERA5. The failure mode is a 2–3 second
`{"status": "failed"}` response with no message field, which is why
prod was burning through Era5MonthBatchWorker attempts without anything
landing in era5_profiles.
Switch the pressure-level moisture variable to specific_humidity (SPFH),
update the wgrib2 extraction regex, and convert SPFH → dewpoint °C via
Weather.ThetaE.dewpoint_from_spfh/2 in the profile builder. Surface
dewpoint keeps using the single-level 2m dewpoint_temperature, which is
valid and already working.
Three signal sources we already collect but weren't using:
* NEXRAD composite reflectivity → rain rate via Marshall-Palmer, taken
as max of HRRR-derived and NEXRAD-derived rate so fast convective
cells between HRRR hourly analyses can still trigger the rain penalty.
Only active on f00 — forecast hours can't see future radar. New
Scorer.dbz_to_rain_rate_mmhr/1 with 5 dBZ noise floor and 150 mm/hr
hail-safe ceiling.
* hrrr_native_profiles.best_duct_band_ghz → Scorer.score_refractivity/4
applies a 1.15× boost when the cell's native-resolution duct supports
the target band's frequency. HRRR pressure-level gradients
systematically under-read thin trapping layers the native profile can
resolve. Sub-band ducts do NOT boost — they're evidence that the
gradient we have is all there is at the target frequency.
* Commercial LOS link rx_power fading → inverse tropo sensor.
Commercial.link_degradation_at/3 computes the average 7-day-baseline
vs current delta across enabled links within 75 km, ignoring links
where link_state != 1. Scorer.commercial_link_boost/2 adds +2 to +25
to the composite score for 3+ dB of fading. ~150 km radius around
DFW is the only zone this helps today, but it's the first *measured*
signal in the algorithm vs the model-derived proxies.
Also fix a latent test bug exposed by the earlier ERA5 poll-timeout
bump: era5_batch_client_test's "uncached path returns error" tests
hung for up to an hour when run with direnv's real CDS key. New
describe-level setup explicitly unsets the env var so the tests stay
hermetic.
1,359 tests, 0 failures.
Direct queries against prod (75M HRRR profiles, 16,864 soundings, 392k
surface obs, new hrrr_native_profiles table) drive these changes:
* Reinstate shallow-BL bonus as an HPBL multiplier on the refractivity
score (1.10× <200m → 0.78× ≥2000m). The Apr 11 "shallow BL bonus
removed" conclusion was an artifact of the prior matching strategy;
with the cleaner contact↔HRRR join (n=680) the binned data goes 230 km
avg at HPBL <200m vs 100 km at ≥2000m, monotonic across all bins.
* Add 6 missing bands (142, 145, 288, 322, 403, 411 GHz) so contacts in
those bands stop being silently dropped. Coefficients extrapolate
ITU-R P.676/P.838 trends from the existing 134/241 GHz entries.
* Bump ERA5 poll timeout 10 min → 1 hour and Era5MonthBatchWorker
max_attempts 3 → 5 so CDS slowness stops discarding tiles. Also dump
the full failure body when CDS omits the message field — the prior
"ERA5 job failed: nil" was masking real error reasons in oban_jobs.
* Add scripts/recalibrate_algo.py so this analysis can be re-run any
time new data lands without manual SQL. Reads PROP_PROD_DB_URL from
.envrc, drops a Markdown report into docs/algo-reports/.
* Append a dated Part 2c to algo.md documenting the corpus expansion,
the honest accounting of historical HRRR coverage (only ~1,020 of 58k
contacts are precision-matchable), and the empty era5/rtma/climatology
tables. Update the gaseous-absorption and rain-attenuation tables to
include the new bands.
Test suite: 1,335 tests, 0 failures.
IEM's RAOB endpoint stopped keeping Canadian stations current — most
stalled around Sep 2024 — which leaves a gap in refractivity/ducting
calibration over all the CW*/CY* stations already in weather_stations.
MSC's own tephi CSV is fixed-width and rendering-focused; the WMO TEMP
bulletins would need a full decoder. University of Wyoming serves the
same stations in a clean space-delimited format, is current, and
accepts the 3-letter station code (drop the leading C from Canadian
ICAO). Its output shape matches IemClient.parse_raob_json/1 exactly,
so it drops straight into Weather.upsert_sounding/2.
- New UwyoSoundingClient with URL builder, Req-based fetch_sounding,
and a fixed-width parse_sounding_html that extracts PRES/HGHT/TEMP/
DWPT/DRCT/SKNT from the <PRE> block and DateTime from the <H2> header.
- New CanadianSoundingFetchWorker: Oban batch worker that finds all
sounding stations whose code starts with C, picks the most recently
publishable 00Z or 12Z slot (90 min publish delay), calls UWYO per
station, and upserts with SoundingParams-derived refractivity/
ducting fields.
- Oban cron at 01:30Z and 13:30Z (dev + prod) to drive the worker.
- Req.Test plug wiring in config/test.exs.
- Captured Goose Bay fixture as test/support/fixtures/uwyo_sounding_yyr.html.
Also drops two implementation plans under docs/plans/:
- 2026-04-13-rdps-vertical-profiles.md: ~2 day plan for RDPS 10 km
Canadian ducting outside HRRR's Lambert footprint.
- 2026-04-13-hrdps-canadian-prop-grid.md: ~5 day plan for the full
HRDPS 2.5 km Canadian prop grid. Explicitly recommends doing RDPS
first.
TDD throughout, 19 new tests, 1318/1318 passing, credo strict clean.
warm_grid_cache_and_broadcast was re-SELECTing 92k hrrr_profiles rows
with JSONB profile/duct_characteristics columns on every forecast hour.
Row decoding exceeded the 15s default DB connection timeout, killing
the worker before f01-f18 could run. Oban retried from f00 and got
stuck in a loop: for the last several hours, no forecast hours were
being written and even f00 was stale.
Build the weather cache rows directly from the in-memory grid_data
the worker already has (Weather.build_grid_cache_rows/2) and
GridCache.broadcast_put directly. No DB round trip, no JSONB decode.
Also widen the cron from hourly to every 3 hours so a full f00-f18
sweep (~95 min) can actually complete before the next run starts,
and drop the redundant prune_old_scores() at worker start
(PropagationPruneWorker already runs every 15 min). Add a 60s query
timeout to load_weather_grid_from_db as defense-in-depth for the
cold-cache fill path that still uses it.
Instead of shelling out to https://gridmap.org/locate/:callsign for
callsign → lat/lon lookups, ports the resolver pipeline from gridmap-web
into this project so the whole flow runs in-process.
New modules:
- Microwaveprop.Qrz — cache facade over the QRZ.com XML callsign API.
Looks up from qrz_callsigns first, falls back to a live fetch, and
upserts the result with a configurable cache_ttl_hours (default
168h / 7 days).
- Microwaveprop.Qrz.Client — HTTP/XML client against
https://xmldata.qrz.com/xml/current/. Holds the session key in an
Agent, transparently re-logs-in on :session_expired, and parses
responses via xmerl.
- Microwaveprop.Qrz.Callsign — Ecto schema for the qrz_callsigns
cache table, binary_id primary key per project convention.
- Microwaveprop.Qrz.Record — slim struct with only the 11 fields
we actually consume (identity, name, grid, address, lat/lon).
The full XML payload stays in the raw :data jsonb column for
anyone who wants the other ~40 QRZ fields.
- Microwaveprop.Geocoder — Req-based client against the Google Maps
Geocoding API. Only called as a fallback when QRZ has no explicit
<lat>/<lon> for the callsign.
- Microwaveprop.CallsignLocation — orchestrator. Reads the
callsign_locations cache, on miss calls Qrz then either uses QRZ's
coords directly or geocodes the formatted address, snaps to an
8-char Maidenhead grid via Microwaveprop.Radio.Maidenhead, and
upserts the result.
Microwaveprop.Radio.CallsignClient.locate/1 is rewritten to delegate
to CallsignLocation.lookup/1 and shape the response back to the
existing {:ok, %{callsign, gridsquare, lat, lon}} contract so the
callers in PathLive and RoverLive don't change.
Wiring:
- priv/repo/migrations/20260413000000_create_qrz_callsigns_and_callsign_locations.exs
creates qrz_callsigns and callsign_locations with unique indexes
on :callsign.
- Microwaveprop.Qrz.Client added to the application supervision tree
so the session Agent is started.
- :xmerl added to extra_applications so the release bundles it.
- config/test.exs wires Req.Test plugs for Microwaveprop.Qrz.Client
and Microwaveprop.Geocoder, forces cache_ttl_hours: 0 so the cache
never short-circuits test-level stubs, and supplies dummy QRZ
credentials.
- config/runtime.exs pulls QRZ_USERNAME / QRZ_PASSWORD / QRZ_AGENT
and GOOGLE_API_KEY from the environment so prod and dev can
configure both upstream keys out of band.
Tests (ported verbatim from gridmap-web):
- test/microwaveprop/qrz/callsign_test.exs — schema/changeset
- test/microwaveprop/qrz_test.exs — cache hit, cache miss, TTL
behavior, upsert on stale, error passthrough, case-insensitive
input
- test/microwaveprop/geocoder_test.exs — success, zero results,
request denied, transport error
- test/microwaveprop/callsign_location_test.exs — end-to-end flow
including the QRZ-lat/lon shortcut and the missing-address error
path
All 1294 tests still pass. Credo strict clean.
The pending-edits list on the admin review page now sorts through
live_table. Columns for contact, submitted-by user, changed-fields
summary, and submitted timestamp are rendered via 2-arity custom
renderers so the Ecto-preloaded :contact and :user associations stay
accessible — live_table's data_provider path skips select_columns so
the full struct (with preloads) reaches the renderer.
The review drawer, diff table, approve/reject flow, and note textarea
all stay as-is; after approve/reject the page push_patches back to
its current path so live_table re-runs handle_params and the list
refreshes.
Pending count in the header now reads Radio.pending_edit_count/0
instead of length(@edits) since we no longer hold the full list in
socket state.
Replaces the stream-backed static table for approved beacons with
LiveTable.LiveResource, giving users column sorting and full-text
search across callsign and grid. Custom renderers preserve the
frequency/EIRP formatting, keying label, and on-air badge so the
visual output matches the previous daisyUI cells.
A new Beacons.approved_beacons_query/0 returns the base Ecto query
(`where: approved == true`), assigned onto socket.assigns.data_provider
in mount so live_table's handle_params uses it as the root query and
applies sort/search/paginate on top. The leaflet map above the table
keeps its existing phx-hook+phx-update=ignore setup, and the pending-
approval admin section stays as a stream-backed `<.table>` — live_table
is single-resource-per-liveview and the pending list is a distinct
admin concern anyway.
PubSub updates (beacon created/updated/deleted) now push_patch to the
current path so live_table re-runs handle_params and the table
refreshes in place.
The profile page's involving-contacts query now limits to the 100
most recent rows (ordered by qso_timestamp desc). A small note is
shown under the heading when the cap is hit so users know there's
more data they're not seeing.
Add a new card on the profile page listing every contact where the
callsign appears as either station1 or station2, case-insensitive,
newest first. Backed by a new `Radio.list_contacts_involving_callsign/1`
query with unit coverage for station1/station2 matching, case handling,
ordering, and blank input.
Profile page:
* New MicrowavepropWeb.UserProfileLive at /u/:callsign is a public
page showing a user's contacts and beacons. Resolves case-
insensitively so /u/w5isp and /u/W5ISP are the same thing; unknown
callsigns redirect to /. Uses daisyUI card / stats / table
components with an avatar-placeholder initial and hero icons.
* Accounts.get_user_by_callsign/1 (case-insensitive) plus
Radio.list_contacts_for_user/1 and Beacons.list_beacons_for_user/1
back the page. Beacons list includes both approved and pending so
owners see their drafts.
* The top nav bar and the three LiveView sidebars (MapLive,
WeatherMapLive, ContactMapLive) now render the logged-in callsign
as a navigate link to /u/:callsign instead of a static label.
* Nine new tests cover the lookup, the LiveView render, and the
ownership-scoped queries.
Flexible band input:
* New Microwaveprop.Radio.BandResolver module converts any of:
ADIF wavelength labels ("33cm", "1.25cm", "6mm", case/whitespace
insensitive), numeric frequency strings ("903.100", "10368.000"),
and canonical MHz integers into the one of the site's known bands.
Returns the nearest allowed band for numeric inputs >= 900 MHz,
nil otherwise.
* 902 MHz is added to Contact.@allowed_bands, ContactEdit.@allowed_bands,
AdifImport.@allowed_bands, and the BandResolver list so "33cm"
round-trips end-to-end.
* AdifImport and CsvImport now delegate band resolution to
BandResolver, and Radio.create_contact/2 normalizes the :band attr
on the way in so the manual form and any API callers benefit too.
CsvImport's "invalid band" tests previously used 99999 MHz which
the new resolver snaps to the nearest allowed band; swapped to
"notaband" which is truly unresolvable.
Contacts and beacons list UX:
* Remove the "Submitted" column from /contacts — it duplicated info
already visible on the detail page and was pushing the real
columns off narrow viewports. submitted_cell/1 and its three
column-specific tests go with it.
* Hide the Lat / Lon columns from /beacons — six decimal places of
coordinates weren't useful next to the grid square and took a
disproportionate amount of row width.
Submit and contact-detail forms now use a plain text input for the QSO
timestamp with a "YYYY-MM-DD HH:MM" placeholder and regex pattern. The
native datetime-local input ignores lang="en-GB" on macOS/iOS when the
system clock is 12-hour, so it was rendering AM/PM against the user's
wishes. Ecto's :utc_datetime cast already accepts this format (verified
space+seconds, T+seconds, and both with and without seconds/Z).
Contact edit workflow grows a third branch: admins still apply directly,
non-owners still go through the admin review queue, and now logged-in
owners of a contact (user_id == current_user.id) can apply edits
directly too. New helpers Radio.owner?/2 and Radio.apply_owner_edit/3
reuse the existing normalize + diff + apply_edit_to_contact path so the
grid-change → enrichment re-enqueue pipeline kicks in automatically.
Anonymous contacts (user_id nil) and other users' contacts both return
{:error, :not_owner}.
Nine new tests cover owner?/2 and apply_owner_edit/3 including the
no-changes, wrong-user, and anonymous-contact paths.
Regex.scan with `return: :index` reports BYTE offsets, but String.slice/3
uses CHARACTER offsets. When a record contained a multi-byte UTF-8
character in an earlier field (e.g. "café" in NOTES), every subsequent
field's slice shifted one byte left, eventually landing on a literal
">" that crashed String.to_integer/1 with ArgumentError.
Switch to :binary.part/3 which is byte-based and matches what Regex.scan
reports. Also simplify the value offset calculation to use the scanned
tag_len directly instead of re-splitting the tag text. Reproduced in a
new regression test.
Prod stack trace: AdifImport.parse_fields/1 crashed on upload with
binary_to_integer(">") -- a FreeDV/N1MM logger that embedded UTF-8
characters in a NOTES field triggered it.
MRMS
----
Layer the NOAA MRMS PrecipRate product onto the score grid so rain fade
updates every 2 minutes instead of every hour alongside HRRR. New modules:
- Microwaveprop.Weather.MrmsClient: fetches the latest .grib2.gz off the
NCEP mirror (Req auto-decompresses so no gunzip step), writes the raw
GRIB2 to a temp file, and calls the existing wgrib2 wrapper with the
0.125 propagation grid spec to get interpolated cells. Returns a
%{{lat, lon} => mm_per_hour} map with missing-value sentinels dropped.
- Microwaveprop.Weather.MrmsCache: ETS-backed GenServer mirroring
ScoreCache/GridCache. Caches a single "current" entry keyed by
valid_time with PubSub broadcast so peer nodes stay in sync and only
the Oban leader pays the fetch + regrid cost.
- Microwaveprop.Workers.MrmsFetchWorker: cron every 2 minutes, short-
circuits when the cached valid_time already matches the newest file.
Microwaveprop.Propagation.AsosNudge.compute/4 now takes an optional
rain_grid. When a cell has MRMS rain >= 0.1 mm/hr it gets patched onto
the HRRR profile's `precip_mm` field (the scorer already reads it there)
and the cell is re-scored even with no ASOS station nearby. Cells with
MRMS rain below the threshold aren't touched so dry cells keep their
raw HRRR scores (which have the wind/sky/native-gradient signal that
isn't persisted on HrrrProfile rows and would otherwise be lost).
AsosAdjustmentWorker pulls MrmsCache on every tick and passes the grid
through to AsosNudge.compute/4. Also skips the IemClient error branch
that can never happen and handles the ASOS-empty + MRMS-empty case
explicitly. MrmsCache wired into the supervision tree; MrmsFetchWorker
cron entry added to config.exs and dev.exs.
Four new AsosNudge cases cover MRMS-only re-scoring, threshold gating,
and the wet/dry score delta.
Beacons 500
-----------
Beacon.format_freq/1 and format_mw/1 crashed on whole-number floats
(e.g. 24192.0) because `frac == 0.0` could become false under float
rounding while `trim_trailing_zeros/1` stripped the decimal point,
leaving a 1-element list that couldn't be destructured as [_, frac].
Shared format_number/1 helper handles integer input directly and
pattern-matches both the "int-only" and "int + frac" shapes.
Added stream_data property tests covering the whole microwave range for
both integers and floats to catch this class of bug before prod.
UTC clock flash
---------------
The /weather and /map UTC clocks were empty until the JS hook mounted
post-WebSocket, producing a several-second blank spot on initial load
and a clobber risk on sidebar re-renders. Mount now computes a
server-rendered `initial_utc_clock` string and the template seeds the
element with that plus `phx-update="ignore"` so LiveView morphdom won't
overwrite what the hook writes.
Adds Microwaveprop.Propagation.AsosNudge: a pure IDW bias-field module
that takes ASOS observations + HRRR profiles and returns re-scored grid
rows for every cell within 250km of a reporting station. Upper-air
fields (min_refractivity_gradient, pwat_mm, hpbl_m, profile, duct
metadata) pass through unchanged so HRRR's signal isn't clobbered.
The old AsosAdjustmentWorker was unwired and buggy — nil'd out ~22% of
the scoring weight and wrote orphan timestamps. Replaced with a slim
worker that queries the latest HRRR valid_time, fetches live ASOS
currents, calls AsosNudge.compute/3, and upserts onto
(lat, lon, valid_time, band_mhz) so nudged values overwrite the HRRR
hour cleanly instead of polluting available_valid_times. After each
upsert it warms ScoreCache and broadcasts propagation:updated so live
/map clients refresh.
Cron hooked up every 10 minutes in config.exs and dev.exs. Also cleaned
up the stale "dev has propagation disabled" note in CLAUDE.md.
13 new AsosNudge unit tests cover: residual computation (co-located,
out-of-grid, nil fields), IDW weighting (single station, far station,
two equidistant stations, nil component handling), upper-air
preservation, and the compute/3 entry point's shape and radius filter.
Drive-by Styler formatting touched a handful of unrelated files from
`mix format`.
Centralized the From and Reply-To headers in Microwaveprop.Mailer.apply_defaults/1
so UserNotifier and EditNotifier can't drift. Drops the EMAIL_FROM env var
override — both values are hardcoded now.
The /weather LiveView was timing out on pod restart because
latest_weather_grid/1 ran load_weather_grid_from_db/1 synchronously on
cache miss — that query reads 176k hrrr_profiles rows with two huge
JSONB columns (profile array + duct_characteristics), and JSONB
decoding blocked the LiveView process past the 15-second pool
ownership timeout.
On cache miss latest_weather_grid/1 now returns empty immediately and
kicks off a deduped background fill through GridCache.claim_fill/1 (a
per-valid_time ETS lock so N concurrent mounts after restart don't each
fire the slow query). When the fill completes it broadcasts
weather:updated, and the handle_info in WeatherMapLive hits the warm
cache and pushes rows over the socket.
Added load_weather_grid/1 as a synchronous sibling for tests and any
caller that genuinely needs inline data. Tests updated to use it and
to clear GridCache in setup so ETS state doesn't leak between cases.
Also untrack k8s/secret.yaml and add both it and k8s/*-secret.yaml to
.gitignore — those manifests hold plaintext SMTP and database
credentials that should not live in the repo. Secrets already in git
history should be rotated separately.
- AdifImport.normalize_mode/2 maps ADIF MODE (and SUBMODE when present)
onto the fixed set of modes the site accepts: CW/SSB/FM/FT8/FT4/Q65.
SUBMODE takes precedence so `MODE=MFSK, SUBMODE=FT8` correctly imports
as FT8. USB/LSB/DSB all become SSB; NBFM/WFM become FM.
- Unknown modes (RTTY, PSK31, etc.) map to nil so the row still imports
— mode is optional on the submission path now.
- AdifImport.mode_mapping_reference/0 returns the user-facing list of
recognized inputs and their mappings.
- Submit page's ADIF tab renders that list in a table so users can see
upfront what will happen to their file before uploading.
- Add ContactMapController serving /api/contacts/map with pre-gzipped
JSON from Microwaveprop.Cache. Browsers handle Content-Encoding: gzip
natively; payload drops from 5.7 MB raw to 1.5 MB on the wire (73%).
Cache TTL 10 min, invalidated on contact insert alongside the other
contact-map cache keys.
- Move contact map payload out of the LiveView's initial HTML entirely.
Shell renders with the filter chrome (count + band list); the hook
fetches contacts from the HTTP endpoint in parallel with map init.
This also avoids Phoenix.Socket's 64 KB frame limit (push_event would
have required chunked transfer or a global frame size bump).
- Parallelize ContactLive.Show hydrate: five independent DB loads
(weather, solar, hrrr_path, terrain, iemre) now run concurrently via
Task.async + Task.await_many. Total latency drops from sum(each) to
max(each), cutting contact show hydrate time roughly 5x.
- Add Weather.GridCache: ETS cache of derived HRRR grid rows keyed by
valid_time, cluster-synced via PubSub. Eagerly warmed from
PropagationGridWorker after each upsert so /weather map pan/zoom and
weather_point_detail hit zero DB on warm cache.
- Replace latest_weather_grid DB query path with cache-first lookup +
DB fallback. hrrr_profiles is 42M rows partitioned; pulling 3-10k
rows per viewport on every pan was the main cost.
- ContactLive.Show: defer the heavy enrichment loads (weather, solar,
HRRR, terrain, IEMRE, elevation profile, ITU-R propagation analysis,
data_sources) into a handle_info(:hydrate) that runs after the shell
renders. Initial mount now returns nil placeholders; template already
had :if guards for all of them. Shell-to-first-paint goes from
~500ms-2s down to ~20ms.
- Cache fetch_queue_counts for 5s in ContactLive.Show — oban_jobs group
by query was running on every contact page view.
- Backfill stats: wrap count_unprocessed, fetch_stats, fetch_db_stats
in Microwaveprop.Cache with 2-5s TTLs; bump refresh debounce from 1s
to 2s so bulk enrichment events don't thrash the DB.
- Add contacts_inserted_at_desc_id_desc_idx so /contacts list runs as an
Index Scan (~0.4ms) instead of a Seq Scan + top-N heapsort (~77ms on
58k rows)
- Add Microwaveprop.Cache: tiny ETS-backed TTL cache for memoizing
expensive-but-stale-tolerant values
- Cache total_entries for unsearched /contacts page loads (30s TTL,
invalidated on insert)
- Cache the entire /contacts/map payload (pre-encoded JSON + band list),
moving load_contacts into Radio.contact_map_payload; invalidated on
new contact insert. Mount goes from ~150-300ms to ~5ms.
- Fix /map overlay blinking on load: reuse the Leaflet GridLayer across
renderScores calls and just redraw() against the updated ScoreGrid,
instead of removeLayer + new layer which flashed between tile regens
- Make mode optional on submission: schema change (null: true),
submission_changeset no longer requires it, blank strings normalise to
nil, CSV import accepts 6-column (no mode) or 7-column layouts
- core_components .input adds a red asterisk to labels when required,
giving users a visual cue for which fields must be filled on /submit
- ScoreCache stores {band, valid_time} as %{{lat, lon} => score} map so
point lookups are O(1); adds fetch_point/4 and valid_times/1
- available_valid_times/1 reads directly from ScoreCache when warm,
falls back to DB on cold start
- point_forecast/3 iterates cached valid_times and uses fetch_point/4
instead of hitting the DB per click
- NexradCache: node-local ETS cache of decoded n0q PNG pixel buffers
keyed by 5-minute rounded timestamp; skips ~1-5s HTTP+decode on
concurrent/repeat clicks within the same window
- MapLive: start_async the rain_scatter fetch so point_detail renders
immediately with a pending marker; push rain_scatter_update when
NEXRAD resolves
- MapLive: preload all 18 remaining forecast hours for the current
viewport after mount/band change/propagation_updated; client caches
them and renders timeline scrubs instantly without a server roundtrip.
Adds set_selected_time event for fast-path state sync.
- Propagation map JS: forecastCache map + drawScatterMarkers helper,
timeline click uses preloaded cache when available
- Add ScoreCache GenServer with node-local ETS table keyed by
{band, valid_time}, subscribed to "propagation:cache" PubSub topic so
every pod stays in sync with a single hourly compute
- scores_at/3 checks cache first, falls back to DB and populates on miss
- PropagationGridWorker warms and broadcasts the cache for each band
after every forecast hour upsert; prunes >2h old entries
- Replace per-pixel string-keyed Map with flat Int8Array over the CONUS
grid in propagation_map_hook.ts to eliminate allocations in the tile
rasterization hot loop (interpolateScore / propagationReach)
1. Prevent user_id impersonation in contact submissions
- Remove user_id from submission_changeset cast fields
- Pass user_id as a separate server-side argument to create_contact/2
- Client form params can no longer forge account-linked contacts
2. Restrict flag toggle to admins only
- Add admin? guard to toggle_flag event handler
- Hide flag button from non-admin users in template
3. Reset enrichment on timestamp edits
- Add qso_timestamp to the fields that trigger enrichment reset
- Prevents stale weather/HRRR/IEMRE data after time changes
Aliases: add module aliases for 9 nested module references
Apply: replace apply/3 with direct module attribute calls
Line length: break 1 long spec line
Refactoring: extract helpers to reduce complexity and nesting
in show.ex, radio.ex, weather workers, terrain, duct detection,
backfill dashboard, contact map, and mix tasks
- Replace %Struct{} with Struct.t() in all @spec annotations
- Replace length(x) > 0 with x != [] in test assertions
- Fix multi-line spec struct references in weather.ex
- Add belongs_to :user on Beacon schema, preload in get_beacon!/1
- Show "Submitted by {callsign}" on beacon detail page
- Add NTMS donation link on the about page
- Change grid preference text to 8 characters (e.g. EM12kp37)
Parse ADIF tagged-data format with fuzzy frequency-to-band matching
(nearest microwave band for any freq >= 900 MHz). Supports STATION_CALLSIGN,
OPERATOR, CALL, GRIDSQUARE, MY_GRIDSQUARE, FREQ, BAND, QSO_DATE, TIME_ON.
Same dedup and preview/commit flow as CSV upload.
Also fix select dropdown text alignment in daisyUI.
SRTM profiles no longer abort on missing tiles — water/ocean areas
return 0m elevation instead of failing the entire profile and falling
back to the rate-limited API. Elevation client and viewshed now pass
download: true to auto-fetch missing tiles from S3. NFS mount changed
to writable so downloaded tiles persist across pods.
Registered users can suggest edits to any contact's core fields
(callsigns, grids, band, mode, timestamp). Edits enter an admin
approval queue with field-by-field diff view. On approve, changes
are applied and enrichment re-enqueued if grids/band changed.
Users receive email notification on approve or reject.
Also updates dependabot.yml for mix ecosystem.
When clicking a grid point, fetches the latest NEXRAD composite
reflectivity and identifies rain cells within 300 km that could
enable rain scatter contacts. Shows:
- Scatter classification (excellent/good/marginal/none)
- Top 3 cells with dBZ, distance, bearing, and relative signal
- Colored circle markers on the map at rain cell locations
- Markers sized by reflectivity, colored by intensity
Uses simplified bistatic radar equation accounting for reflectivity,
frequency-dependent scattering (Rayleigh/Mie), and R^4 path loss.
NEXRAD cells sampled every ~5 km within bounding box for efficiency.
Terrain elevations now include the earth bulge correction, so the
profile visually humps up in the middle relative to the straight
LOS beam — showing how the curved earth rises into the signal path.
Uses actual k-factor from HRRR refractivity when available, otherwise
standard 4/3. For a 300 km path at k=4/3, the midpoint bulge is
~2,650 m (8,700 ft).
When clicking a grid point with ducting, the panel now shows each
duct layer with base-top height in feet, thickness in meters, and
minimum trapped frequency. Data flows from Duct.analyze through
the scoring factors as a ducts array.
- Replace stale April 2026 manual weights with recalibrated values
- Document native hybrid-sigma data in data flow section
- Note refractivity factor now uses native 10-50m resolution
- Add hourly grid integration section to Part 12
- Expose duct_info (count, freq, thickness) in scoring factors for UI
The PropagationGridWorker now fetches native hybrid-sigma levels
(TMP, SPFH, HGT, PRES × 50 levels) alongside the standard surface
and pressure products. Native data provides 10-50m vertical spacing
vs 250m from pressure levels, detecting thin surface ducts invisible
to the standard product.
Key design: cell-by-cell reducer in Wgrib2.extract_grid_from_file_mapped
processes each of the 95k CONUS cells through a duct analysis function
inline, keeping only scalar metrics per cell. Peak memory ~86 MB
instead of ~1.8 GB for the full grid map.
Per-cell output: native_min_gradient, best_duct_freq_ghz,
max_duct_thickness_m, duct_count. The scorer prefers the native
gradient over the pressure-level gradient when available.
Native fetch is optional — if it fails, scoring continues with
pressure-level data only.
Backtest on 11,431 native profiles (2026-04-11):
- Drop duct_usable_10/24/47ghz (always 1.0, no discrimination)
- Drop bulk_richardson (near-identical QSO vs baseline means)
- Document all feature results with signal strength assessment
- theta_e_jump is strongest native discriminator (44% lift)
- best_duct_freq and duct_thickness show clear physical signal
- Round grid-derived lat/lon in maybe_fill_latlon changeset step
- Format lat/lon to 6 decimal places on index table
- Migrate existing beacon data to 6 decimal precision
- Add comma separators to EIRP mW display (e.g. 10,000 mW)
- Extract shared add_commas helper for format_freq and format_mw
Height in feet doesn't need decimal precision. Migrates the DB
column, updates schema type, and strips trailing .0 from form
input so the integer cast succeeds.
- Add CommaNumber JS hook for live comma formatting while typing
frequency MHz in the beacon form
- Strip commas server-side before changeset validation
- Order beacon list by most recently added (desc inserted_at)
- Round lat/lon to 6 decimal places in changeset and display
- Round height_ft to integer in changeset and display
- Display coords at 6 decimal places on show page
