docs(about): update for HRDPS, Rust pipeline, perf work, remove ML model refs
Some checks failed
Build and Push / Build and Push Docker Image (push) Failing after 4m38s

- Add Canadian HRDPS model coverage (to 60°N)
- Document Rust pipeline (prop-grid-rs) and its performance characteristics
- Add .sgrid/.pgrid binary format details
- Note ~6s chain step and 23-band fused pass
- Remove Nx/Axon/ExLA ML model references (scaffolding was removed)
- Update roadmap: drop model training, add PSKR calibration, propagation alerts
- Fix factor count (was 9, is 10) in roadmap item
This commit is contained in:
Graham McIntire 2026-08-01 14:39:10 -05:00
parent 95976b6b90
commit e490f65359
No known key found for this signature in database
GPG key ID: F4ABF488E6029E59

View file

@ -56,10 +56,11 @@ defmodule MicrowavepropWeb.AboutLive do
<div role="alert" class="alert alert-info">
<p>
<strong>Coverage:</strong>
right now this only covers the continental US. Every upstream source
we're using — NOAA HRRR, ASOS, IEMRE, the ITU terrain data — is
CONUS-only. Hawaii, Alaska, Canada, and Mexico will need different
data sources and aren't wired up yet.
right now this focuses on the continental US. NOAA HRRR, ASOS,
IEMRE, and the ITU terrain data are CONUS-only. The Canadian HRDPS
model extends coverage north to 60°N (southern Canada), and SRTM
terrain tiles cover that range. Hawaii, Alaska, northern Canada,
and Mexico will need different data sources and aren't wired up yet.
</p>
</div>
</div>
@ -72,13 +73,16 @@ defmodule MicrowavepropWeb.AboutLive do
<ol class="list-decimal list-outside pl-5 space-y-2">
<li>
<strong>The weather.</strong>
Hourly 3 km HRRR forecasts give surface fields and pressure-level
profiles. We derive refractivity profiles, minimum refractivity
gradient, ducting detection, boundary-layer depth, and precipitable
water for every grid cell. ASOS surface obs, 12-hourly upper-air
soundings, and gridded IEMRE reanalysis fill in the gaps. A
10-factor composite score is written out for every 0.125° cell
across CONUS, for each hour of a 48-hour forecast.
Hourly 3 km NOAA HRRR forecasts give surface fields and pressure-level
profiles across CONUS, while the Canadian HRDPS model (0.125°
resolution, 4× daily) extends coverage north to 60°N. We derive
refractivity profiles, minimum refractivity gradient, ducting
detection, boundary-layer depth, and precipitable water for every
grid cell. ASOS surface obs, 12-hourly upper-air soundings, and
gridded IEMRE reanalysis fill in the gaps. A 10-factor composite
score is written for every 0.125° cell (~95k across CONUS) across
23 microwave bands (902 MHz241 GHz), for each hour of a 48-hour
forecast.
</li>
<li>
<strong>The contacts.</strong>
@ -154,6 +158,16 @@ defmodule MicrowavepropWeb.AboutLive do
(HRRR pulls, terrain, ASOS, soundings, IEMRE, solar indices,
commercial links).
</li>
<li>
A Rust pipeline (prop-grid-rs) handles the heavy lifting:
fetching GRIB2 data from NOAA S3, decoding via wgrib2, deriving
weather scalars, and scoring all 23 bands in a single fused pass
over ~95k cells. Writes dense binary artifacts (.pgrid profiles,
.sgrid weather scalars, and per-band .prop score files) to NFS
for sub-millisecond single-cell reads from Elixir. A full
forecast-hour chain step runs in ~6 seconds ~3× faster than
the previous Elixir-only pipeline.
</li>
<li>
Leaflet + Canvas tile layers for the propagation heatmap we
render 0.125° cells at interactive frame rates. Tailwind v4 +
@ -167,16 +181,12 @@ defmodule MicrowavepropWeb.AboutLive do
</li>
<li>
Data sources: NOAA HRRR (AWS S3, hourly analysis + 18 h
forecasts), Canadian HRDPS (MSC Datamart, 4× daily 48 h
forecasts), Iowa Environmental Mesonet (ASOS and upper-air
soundings), IEMRE gridded reanalysis, SRTM 90 m terrain, SNMP
polling on seven commercial microwave links near DFW at 5-minute
intervals, and NCEP NARR for pre-2014 contact enrichment.
</li>
<li>
There's Nx/Axon/EXLA scaffolding for a small feed-forward model
(13 features 64 32 1 sigmoid). Not trained yet waiting
on enough calibrated data to not overfit.
</li>
</ul>
</div>
@ -191,20 +201,21 @@ defmodule MicrowavepropWeb.AboutLive do
</li>
<li>
<strong>Weight calibration.</strong>
Fit the 9 factor weights against recorded distances / QSO counts
per band so the score reflects real propagation, not my
intuition.
</li>
<li>
<strong>Training the model.</strong>
Once there are enough clean (contact, conditions) pairs, train
the Axon model to augment or replace the hand-tuned scoring.
Re-fit the 10 factor weights against recorded distances / QSO
counts per band so the score reflects real propagation, not hand
tuning. The PSKR spot ingestion pipeline (sampling every 5 min)
is building the calibration dataset automatically.
</li>
<li>
<strong>Better inputs.</strong> MRMS for precipitation at 24+ GHz where rain attenuation
dominates, RTMA/URMA surface analysis blending, GOES total
precipitable water.
</li>
<li>
<strong>Expanded coverage.</strong> The Canadian HRDPS model gives
us southern Canada (to 60°N). Alaska, Hawaii, Mexico, and the
Caribbean need their own data sources and scoring pipelines.
</li>
</ul>
</div>
<div role="alert" class="alert alert-info text-center">