graham/prop
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Rewrite /about in a less-LLM voice

Browse source 
Drops the formal "What we're trying to do / The approach / The stack"
scaffolding for a shorter, first-person telling: line-of-sight is
wrong, here's what we're doing about it, here's what's in the box,
here's what's next. Adds a highlighted note that the scoring is
hand-calibrated and not updated in real time, with links to the
algorithm page and the submit page so readers can help close the
loop.
This commit is contained in:
Graham McIntire 2026-04-09 13:01:19 -05:00
parent 5724b33e8f
commit 087517252b
1 changed files with 87 additions and 73 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

160
lib/microwaveprop_web/live/about_live.ex
View file

@ -23,59 +23,76 @@ defmodule MicrowavepropWeb.AboutLive do
<section class="space-y-8 text-sm leading-relaxed"> <section class="space-y-8 text-sm leading-relaxed">
<div class="space-y-3"> <div class="space-y-3">
<h2 class="text-xl font-bold">What we're trying to do</h2>
<p> <p>
Microwave propagation above 10 GHz is dominated by the lower atmosphere: Line-of-sight is wrong for the microwave bands. Every operator on
humidity gradients, temperature inversions, ducting layers, rain cells, 902 MHz and up has made contacts over hills, through trees, and 200 km
and hyper-local refractivity structure. At 10, 24, 47, 76, 122, and 241 GHz past the horizon — contacts that shouldn't have worked if LOS were the
these effects decide whether a contact happens at 50 km or 500 km. This project was whole story. What's actually going on is the lower atmosphere:
started by Jim KM5PO, then picked up by Graham W5ISP. temperature inversions, humidity gradients, ducting layers, and rain
cells that bend, trap, and absorb signals in ways that change by the
hour. The window for a good contact is often minutes, not days.
</p> </p>
<p> <p>
Traditionally microwave contacts are ruled by line of sight, but we've Jim KM5PO started this project to try to predict those windows from
seen that is incorrect. This project is an attempt to build a data-driven weather data. Graham W5ISP picked it up and is iterating on it from
propagation prediction model specifically for the amateur microwave bands, there. The goal: a map that tells you when to get on the air, per
using hourly numerical weather forecasts, historical contact records, and band, based on real atmospheric conditions and what we've learned
eventually calibrated beacon measurements. from the contacts you and everyone else have already made.
</p> </p>
<div class="rounded-box border border-warning/30 bg-warning/10 p-3">
<p>
<strong>Heads up:</strong>
the scoring you see is hand-calibrated from the data we have so far.
It isn't updated in real time — I (Graham) re-fit the weights
manually when there's enough new data to move the needle. The more
contacts and beacon reports we collect, the better it gets. See the
<.link navigate={~p"/algo"} class="link link-hover font-semibold">algorithm page</.link>
for what's actually under the hood, and please <.link
navigate={~p"/submit"}
class="link link-hover font-semibold"
>submit your contacts</.link>.
</p>
</div>
</div> </div>
<div class="space-y-3"> <div class="space-y-3">
<h2 class="text-xl font-bold">The approach</h2> <h2 class="text-xl font-bold">How it works, roughly</h2>
<p>We pair two things that most propagation tools keep separate:</p> <p>
Two things, paired up. Most propagation tools keep them separate.
</p>
<ol class="list-decimal list-outside pl-5 space-y-2"> <ol class="list-decimal list-outside pl-5 space-y-2">
<li> <li>
<strong>Atmospheric state</strong> — hourly 3 km HRRR forecasts <strong>The weather.</strong>
(surface fields plus pressure-level profiles), hourly surface Hourly 3 km HRRR forecasts give surface fields and pressure-level
observations from ASOS, 12-hourly upper-air soundings, and gridded profiles. We derive refractivity profiles, minimum refractivity
IEMRE reanalysis. From these we derive refractivity profiles, gradient, ducting detection, boundary-layer depth, and precipitable
ducting detection, minimum refractivity gradient, precipitable water, water for every grid cell. ASOS surface obs, 12-hourly upper-air
boundary-layer depth, and a 9-factor composite score for every soundings, and gridded IEMRE reanalysis fill in the gaps. A 9-factor
0.125° cell on a CONUS grid, for each hour of an 18-hour forecast. composite score is written out for every 0.125° cell across CONUS,
for each hour of an 18-hour forecast.
</li> </li>
<li> <li>
<strong>Historical contacts</strong> — 58k+ amateur microwave QSOs <strong>The contacts.</strong>
tagged with the atmospheric conditions at both ends and along the 58k+ amateur microwave QSOs, each tagged with the atmosphere at
path at the exact time they happened. That gives us a ground-truth both ends of the path (and along it) at the time the contact
dataset we can use to calibrate the scoring weights and, eventually, happened. That's a ground-truth dataset we can actually fit against
train a machine learning model that predicts contact success given — "when you had this score, how far did the contact go, and did it
conditions. happen at all?"
</li> </li>
</ol> </ol>
<p> <p>
The current scoring algorithm is a weighted sum of nine factors The scoring is a weighted sum of nine factors: humidity, time of
(humidity, time of day, T–Td depression, refractivity gradient, sky day, T–Td depression, refractivity gradient, sky cover, season, rain,
cover, season, rain, wind, pressure) with band-dependent weights — wind, pressure. The weights are band-dependent because the physics
humidity helps at 10 GHz via enhanced refractivity but hurts at changes as you go up: humidity helps at 10 GHz (more refractivity)
24+ GHz via absorption, for example. Every coefficient in that and hurts at 24+ GHz (absorption). Every coefficient is a hypothesis
formula is a hypothesis waiting to be tested against the contact I'm checking against the contact and beacon data.
and beacon data.
</p> </p>
</div> </div>
<div class="space-y-3"> <div class="space-y-3">
<h2 class="text-xl font-bold">What we've collected</h2> <h2 class="text-xl font-bold">What we've collected so far</h2>
<p>Live counts from the production database:</p> <p>Straight from the production database:</p>
<div class="grid grid-cols-2 md:grid-cols-3 gap-3"> <div class="grid grid-cols-2 md:grid-cols-3 gap-3">
<.stat_card label="Contacts" value={@stats.contacts} /> <.stat_card label="Contacts" value={@stats.contacts} />
<.stat_card label="Weather stations" value={@stats.weather_stations} /> <.stat_card label="Weather stations" value={@stats.weather_stations} />
@ -91,67 +108,64 @@ defmodule MicrowavepropWeb.AboutLive do
</div> </div>
<div class="space-y-3"> <div class="space-y-3">
<h2 class="text-xl font-bold">The stack</h2> <h2 class="text-xl font-bold">How it's built</h2>
<ul class="list-disc list-outside pl-5 space-y-2"> <ul class="list-disc list-outside pl-5 space-y-2">
<li> <li>
<strong>Backend:</strong> Elixir / Phoenix 1.8 with LiveView for the Elixir / Phoenix 1.8 + LiveView for everything web-facing. Ecto
real-time map, Ecto on PostgreSQL for storage, Oban for the on Postgres for storage. Oban runs the background pipelines
background data pipelines (HRRR fetch, terrain, ASOS, soundings, (HRRR pulls, terrain, ASOS, soundings, IEMRE, solar indices,
IEMRE, solar indices, commercial links), Bandit as the HTTP server. commercial links).
</li> </li>
<li> <li>
<strong>Frontend:</strong> LiveView with Leaflet for maps, Canvas Leaflet + Canvas tile layers for the propagation heatmap — we
tile layers for the HRRR heatmap (we render 0.125° cells at render 0.125° cells at interactive frame rates. Tailwind v4 +
interactive frame rates), Tailwind v4 + daisyUI for layout, esbuild daisyUI for layout, esbuild for JS bundling.
for JS bundling.
</li> </li>
<li> <li>
<strong>Physics:</strong> ITU-R P.526-16 knife-edge + Deygout Physics: ITU-R P.526-16 knife-edge + Deygout 3-edge terrain
3-edge terrain diffraction, ITU-R P.838-3 rain attenuation, diffraction, ITU-R P.838-3 rain attenuation, dynamic k-factor
dynamic k-factor from live HRRR refractivity gradients, great-circle from live HRRR refractivity gradients, great-circle geometry,
geometry for link budgets, Free-Space Path Loss with frequency- FSPL with frequency-dependent O₂ and H₂O absorption.
dependent O₂ and H₂O absorption per band.
</li> </li>
<li> <li>
<strong>Machine learning:</strong> Nx / Axon / EXLA scaffolding for Data sources: NOAA HRRR (AWS S3, hourly analysis + 18 h
a 13-feature (8 atmospheric + 4 cyclical temporal + 1 log-frequency) forecasts), Iowa Environmental Mesonet (ASOS and upper-air
feed-forward network, 64 → 32 → 1 sigmoid. Not trained yet — soundings), IEMRE gridded reanalysis, SRTM 90 m terrain, SNMP
waiting on a larger calibration dataset. polling on seven commercial microwave links near DFW at 5-minute
intervals, and Copernicus ERA5 for pre-2014 contact enrichment.
</li> </li>
<li> <li>
<strong>Data sources:</strong> NOAA HRRR model (AWS S3, hourly There's Nx/Axon/EXLA scaffolding for a small feed-forward model
analysis + 18 h forecasts), Iowa Environmental Mesonet (ASOS &amp; (13 features → 64 → 32 → 1 sigmoid). Not trained yet — waiting
upper-air soundings), IEMRE gridded reanalysis, SRTM 90 m terrain on enough calibrated data to not overfit.
tiles, SNMP polling of seven commercial microwave links near DFW
at 5-minute intervals, Copernicus ERA5 for pre-2014 contact
enrichment.
</li> </li>
</ul> </ul>
</div> </div>
<div class="space-y-3"> <div class="space-y-3">
<h2 class="text-xl font-bold">Where this goes next</h2> <h2 class="text-xl font-bold">What's next</h2>
<ul class="list-disc list-outside pl-5 space-y-2"> <ul class="list-disc list-outside pl-5 space-y-2">
<li> <li>
<strong>Beacon calibration.</strong> A distributed network of <strong>Beacon calibration.</strong>
amateur receivers continuously reporting CW beacon signal levels, A distributed network of amateur receivers reporting CW beacon
feeding ground-truth measurements back into the scoring algorithm. signal levels on a schedule. Beacon submissions are already open
Beacon submissions are now open to anyone via the Beacons page. to anyone via the Beacons page.
</li> </li>
<li> <li>
<strong>Scoring weight calibration.</strong> Fit the 9-factor <strong>Weight calibration.</strong>
weights against recorded QSO distances / counts per band so the Fit the 9 factor weights against recorded distances / QSO counts
composite score reflects real propagation rather than intuition. per band so the score reflects real propagation, not my
intuition.
</li> </li>
<li> <li>
<strong>ML model training.</strong> Once we have enough labeled <strong>Training the model.</strong>
contact+condition pairs, train the Axon model to replace or augment Once there are enough clean (contact, conditions) pairs, train
the hand-tuned scoring function. the Axon model to augment or replace the hand-tuned scoring.
</li> </li>
<li> <li>
<strong>Better inputs.</strong> MRMS for precipitation at <strong>Better inputs.</strong> MRMS for precipitation at 24+ GHz where rain attenuation
24+ GHz (where rain attenuation dominates), RTMA/URMA surface dominates, RTMA/URMA surface analysis blending, GOES total
analysis blending, GOES total precipitable water. precipitable water.
</li> </li>
</ul> </ul>
</div> </div>