docs(about): update for HRDPS, Rust pipeline, perf work, remove ML model refs
Some checks failed
Build and Push / Build and Push Docker Image (push) Failing after 4m38s

- Add Canadian HRDPS model coverage (to 60°N)
- Document Rust pipeline (prop-grid-rs) and its performance characteristics
- Add .sgrid/.pgrid binary format details
- Note ~6s chain step and 23-band fused pass
- Remove Nx/Axon/ExLA ML model references (scaffolding was removed)
- Update roadmap: drop model training, add PSKR calibration, propagation alerts
- Fix factor count (was 9, is 10) in roadmap item
This commit is contained in:
Graham McIntire 2026-08-01 14:39:10 -05:00
parent 95976b6b90
commit e490f65359
No known key found for this signature in database
GPG key ID: F4ABF488E6029E59

View file

@ -56,10 +56,11 @@ defmodule MicrowavepropWeb.AboutLive do
<div role="alert" class="alert alert-info"> <div role="alert" class="alert alert-info">
<p> <p>
<strong>Coverage:</strong> <strong>Coverage:</strong>
right now this only covers the continental US. Every upstream source right now this focuses on the continental US. NOAA HRRR, ASOS,
we're using — NOAA HRRR, ASOS, IEMRE, the ITU terrain data — is IEMRE, and the ITU terrain data are CONUS-only. The Canadian HRDPS
CONUS-only. Hawaii, Alaska, Canada, and Mexico will need different model extends coverage north to 60°N (southern Canada), and SRTM
data sources and aren't wired up yet. terrain tiles cover that range. Hawaii, Alaska, northern Canada,
and Mexico will need different data sources and aren't wired up yet.
</p> </p>
</div> </div>
</div> </div>
@ -72,13 +73,16 @@ defmodule MicrowavepropWeb.AboutLive do
<ol class="list-decimal list-outside pl-5 space-y-2"> <ol class="list-decimal list-outside pl-5 space-y-2">
<li> <li>
<strong>The weather.</strong> <strong>The weather.</strong>
Hourly 3 km HRRR forecasts give surface fields and pressure-level Hourly 3 km NOAA HRRR forecasts give surface fields and pressure-level
profiles. We derive refractivity profiles, minimum refractivity profiles across CONUS, while the Canadian HRDPS model (0.125°
gradient, ducting detection, boundary-layer depth, and precipitable resolution, 4× daily) extends coverage north to 60°N. We derive
water for every grid cell. ASOS surface obs, 12-hourly upper-air refractivity profiles, minimum refractivity gradient, ducting
soundings, and gridded IEMRE reanalysis fill in the gaps. A detection, boundary-layer depth, and precipitable water for every
10-factor composite score is written out for every 0.125° cell grid cell. ASOS surface obs, 12-hourly upper-air soundings, and
across CONUS, for each hour of a 48-hour forecast. gridded IEMRE reanalysis fill in the gaps. A 10-factor composite
score is written for every 0.125° cell (~95k across CONUS) across
23 microwave bands (902 MHz241 GHz), for each hour of a 48-hour
forecast.
</li> </li>
<li> <li>
<strong>The contacts.</strong> <strong>The contacts.</strong>
@ -154,6 +158,16 @@ defmodule MicrowavepropWeb.AboutLive do
(HRRR pulls, terrain, ASOS, soundings, IEMRE, solar indices, (HRRR pulls, terrain, ASOS, soundings, IEMRE, solar indices,
commercial links). commercial links).
</li> </li>
<li>
A Rust pipeline (prop-grid-rs) handles the heavy lifting:
fetching GRIB2 data from NOAA S3, decoding via wgrib2, deriving
weather scalars, and scoring all 23 bands in a single fused pass
over ~95k cells. Writes dense binary artifacts (.pgrid profiles,
.sgrid weather scalars, and per-band .prop score files) to NFS
for sub-millisecond single-cell reads from Elixir. A full
forecast-hour chain step runs in ~6 seconds ~3× faster than
the previous Elixir-only pipeline.
</li>
<li> <li>
Leaflet + Canvas tile layers for the propagation heatmap we Leaflet + Canvas tile layers for the propagation heatmap we
render 0.125° cells at interactive frame rates. Tailwind v4 + render 0.125° cells at interactive frame rates. Tailwind v4 +
@ -167,16 +181,12 @@ defmodule MicrowavepropWeb.AboutLive do
</li> </li>
<li> <li>
Data sources: NOAA HRRR (AWS S3, hourly analysis + 18 h Data sources: NOAA HRRR (AWS S3, hourly analysis + 18 h
forecasts), Canadian HRDPS (MSC Datamart, 4× daily 48 h
forecasts), Iowa Environmental Mesonet (ASOS and upper-air forecasts), Iowa Environmental Mesonet (ASOS and upper-air
soundings), IEMRE gridded reanalysis, SRTM 90 m terrain, SNMP soundings), IEMRE gridded reanalysis, SRTM 90 m terrain, SNMP
polling on seven commercial microwave links near DFW at 5-minute polling on seven commercial microwave links near DFW at 5-minute
intervals, and NCEP NARR for pre-2014 contact enrichment. intervals, and NCEP NARR for pre-2014 contact enrichment.
</li> </li>
<li>
There's Nx/Axon/EXLA scaffolding for a small feed-forward model
(13 features 64 32 1 sigmoid). Not trained yet waiting
on enough calibrated data to not overfit.
</li>
</ul> </ul>
</div> </div>
@ -191,20 +201,21 @@ defmodule MicrowavepropWeb.AboutLive do
</li> </li>
<li> <li>
<strong>Weight calibration.</strong> <strong>Weight calibration.</strong>
Fit the 9 factor weights against recorded distances / QSO counts Re-fit the 10 factor weights against recorded distances / QSO
per band so the score reflects real propagation, not my counts per band so the score reflects real propagation, not hand
intuition. tuning. The PSKR spot ingestion pipeline (sampling every 5 min)
</li> is building the calibration dataset automatically.
<li>
<strong>Training the model.</strong>
Once there are enough clean (contact, conditions) pairs, train
the Axon model to augment or replace the hand-tuned scoring.
</li> </li>
<li> <li>
<strong>Better inputs.</strong> MRMS for precipitation at 24+ GHz where rain attenuation <strong>Better inputs.</strong> MRMS for precipitation at 24+ GHz where rain attenuation
dominates, RTMA/URMA surface analysis blending, GOES total dominates, RTMA/URMA surface analysis blending, GOES total
precipitable water. precipitable water.
</li> </li>
<li>
<strong>Expanded coverage.</strong> The Canadian HRDPS model gives
us southern Canada (to 60°N). Alaska, Hawaii, Mexico, and the
Caribbean need their own data sources and scoring pipelines.
</li>
</ul> </ul>
</div> </div>
<div role="alert" class="alert alert-info text-center"> <div role="alert" class="alert alert-info text-center">