docs(about): update for HRDPS, Rust pipeline, perf work, remove ML model refs
Some checks failed
Build and Push / Build and Push Docker Image (push) Failing after 4m38s
Some checks failed
Build and Push / Build and Push Docker Image (push) Failing after 4m38s
- Add Canadian HRDPS model coverage (to 60°N) - Document Rust pipeline (prop-grid-rs) and its performance characteristics - Add .sgrid/.pgrid binary format details - Note ~6s chain step and 23-band fused pass - Remove Nx/Axon/ExLA ML model references (scaffolding was removed) - Update roadmap: drop model training, add PSKR calibration, propagation alerts - Fix factor count (was 9, is 10) in roadmap item
This commit is contained in:
parent
95976b6b90
commit
e490f65359
1 changed files with 35 additions and 24 deletions
|
|
@ -56,10 +56,11 @@ defmodule MicrowavepropWeb.AboutLive do
|
||||||
<div role="alert" class="alert alert-info">
|
<div role="alert" class="alert alert-info">
|
||||||
<p>
|
<p>
|
||||||
<strong>Coverage:</strong>
|
<strong>Coverage:</strong>
|
||||||
right now this only covers the continental US. Every upstream source
|
right now this focuses on the continental US. NOAA HRRR, ASOS,
|
||||||
we're using — NOAA HRRR, ASOS, IEMRE, the ITU terrain data — is
|
IEMRE, and the ITU terrain data are CONUS-only. The Canadian HRDPS
|
||||||
CONUS-only. Hawaii, Alaska, Canada, and Mexico will need different
|
model extends coverage north to 60°N (southern Canada), and SRTM
|
||||||
data sources and aren't wired up yet.
|
terrain tiles cover that range. Hawaii, Alaska, northern Canada,
|
||||||
|
and Mexico will need different data sources and aren't wired up yet.
|
||||||
</p>
|
</p>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
@ -72,13 +73,16 @@ defmodule MicrowavepropWeb.AboutLive do
|
||||||
<ol class="list-decimal list-outside pl-5 space-y-2">
|
<ol class="list-decimal list-outside pl-5 space-y-2">
|
||||||
<li>
|
<li>
|
||||||
<strong>The weather.</strong>
|
<strong>The weather.</strong>
|
||||||
Hourly 3 km HRRR forecasts give surface fields and pressure-level
|
Hourly 3 km NOAA HRRR forecasts give surface fields and pressure-level
|
||||||
profiles. We derive refractivity profiles, minimum refractivity
|
profiles across CONUS, while the Canadian HRDPS model (0.125°
|
||||||
gradient, ducting detection, boundary-layer depth, and precipitable
|
resolution, 4× daily) extends coverage north to 60°N. We derive
|
||||||
water for every grid cell. ASOS surface obs, 12-hourly upper-air
|
refractivity profiles, minimum refractivity gradient, ducting
|
||||||
soundings, and gridded IEMRE reanalysis fill in the gaps. A
|
detection, boundary-layer depth, and precipitable water for every
|
||||||
10-factor composite score is written out for every 0.125° cell
|
grid cell. ASOS surface obs, 12-hourly upper-air soundings, and
|
||||||
across CONUS, for each hour of a 48-hour forecast.
|
gridded IEMRE reanalysis fill in the gaps. A 10-factor composite
|
||||||
|
score is written for every 0.125° cell (~95k across CONUS) across
|
||||||
|
23 microwave bands (902 MHz–241 GHz), for each hour of a 48-hour
|
||||||
|
forecast.
|
||||||
</li>
|
</li>
|
||||||
<li>
|
<li>
|
||||||
<strong>The contacts.</strong>
|
<strong>The contacts.</strong>
|
||||||
|
|
@ -154,6 +158,16 @@ defmodule MicrowavepropWeb.AboutLive do
|
||||||
(HRRR pulls, terrain, ASOS, soundings, IEMRE, solar indices,
|
(HRRR pulls, terrain, ASOS, soundings, IEMRE, solar indices,
|
||||||
commercial links).
|
commercial links).
|
||||||
</li>
|
</li>
|
||||||
|
<li>
|
||||||
|
A Rust pipeline (prop-grid-rs) handles the heavy lifting:
|
||||||
|
fetching GRIB2 data from NOAA S3, decoding via wgrib2, deriving
|
||||||
|
weather scalars, and scoring all 23 bands in a single fused pass
|
||||||
|
over ~95k cells. Writes dense binary artifacts (.pgrid profiles,
|
||||||
|
.sgrid weather scalars, and per-band .prop score files) to NFS
|
||||||
|
for sub-millisecond single-cell reads from Elixir. A full
|
||||||
|
forecast-hour chain step runs in ~6 seconds — ~3× faster than
|
||||||
|
the previous Elixir-only pipeline.
|
||||||
|
</li>
|
||||||
<li>
|
<li>
|
||||||
Leaflet + Canvas tile layers for the propagation heatmap — we
|
Leaflet + Canvas tile layers for the propagation heatmap — we
|
||||||
render 0.125° cells at interactive frame rates. Tailwind v4 +
|
render 0.125° cells at interactive frame rates. Tailwind v4 +
|
||||||
|
|
@ -167,16 +181,12 @@ defmodule MicrowavepropWeb.AboutLive do
|
||||||
</li>
|
</li>
|
||||||
<li>
|
<li>
|
||||||
Data sources: NOAA HRRR (AWS S3, hourly analysis + 18 h
|
Data sources: NOAA HRRR (AWS S3, hourly analysis + 18 h
|
||||||
|
forecasts), Canadian HRDPS (MSC Datamart, 4× daily 48 h
|
||||||
forecasts), Iowa Environmental Mesonet (ASOS and upper-air
|
forecasts), Iowa Environmental Mesonet (ASOS and upper-air
|
||||||
soundings), IEMRE gridded reanalysis, SRTM 90 m terrain, SNMP
|
soundings), IEMRE gridded reanalysis, SRTM 90 m terrain, SNMP
|
||||||
polling on seven commercial microwave links near DFW at 5-minute
|
polling on seven commercial microwave links near DFW at 5-minute
|
||||||
intervals, and NCEP NARR for pre-2014 contact enrichment.
|
intervals, and NCEP NARR for pre-2014 contact enrichment.
|
||||||
</li>
|
</li>
|
||||||
<li>
|
|
||||||
There's Nx/Axon/EXLA scaffolding for a small feed-forward model
|
|
||||||
(13 features → 64 → 32 → 1 sigmoid). Not trained yet — waiting
|
|
||||||
on enough calibrated data to not overfit.
|
|
||||||
</li>
|
|
||||||
</ul>
|
</ul>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
|
@ -191,20 +201,21 @@ defmodule MicrowavepropWeb.AboutLive do
|
||||||
</li>
|
</li>
|
||||||
<li>
|
<li>
|
||||||
<strong>Weight calibration.</strong>
|
<strong>Weight calibration.</strong>
|
||||||
Fit the 9 factor weights against recorded distances / QSO counts
|
Re-fit the 10 factor weights against recorded distances / QSO
|
||||||
per band so the score reflects real propagation, not my
|
counts per band so the score reflects real propagation, not hand
|
||||||
intuition.
|
tuning. The PSKR spot ingestion pipeline (sampling every 5 min)
|
||||||
</li>
|
is building the calibration dataset automatically.
|
||||||
<li>
|
|
||||||
<strong>Training the model.</strong>
|
|
||||||
Once there are enough clean (contact, conditions) pairs, train
|
|
||||||
the Axon model to augment or replace the hand-tuned scoring.
|
|
||||||
</li>
|
</li>
|
||||||
<li>
|
<li>
|
||||||
<strong>Better inputs.</strong> MRMS for precipitation at 24+ GHz where rain attenuation
|
<strong>Better inputs.</strong> MRMS for precipitation at 24+ GHz where rain attenuation
|
||||||
dominates, RTMA/URMA surface analysis blending, GOES total
|
dominates, RTMA/URMA surface analysis blending, GOES total
|
||||||
precipitable water.
|
precipitable water.
|
||||||
</li>
|
</li>
|
||||||
|
<li>
|
||||||
|
<strong>Expanded coverage.</strong> The Canadian HRDPS model gives
|
||||||
|
us southern Canada (to 60°N). Alaska, Hawaii, Mexico, and the
|
||||||
|
Caribbean need their own data sources and scoring pipelines.
|
||||||
|
</li>
|
||||||
</ul>
|
</ul>
|
||||||
</div>
|
</div>
|
||||||
<div role="alert" class="alert alert-info text-center">
|
<div role="alert" class="alert alert-info text-center">
|
||||||
|
|
|
||||||
Loading…
Add table
Reference in a new issue