diff --git a/lib/microwaveprop_web/live/about_live.ex b/lib/microwaveprop_web/live/about_live.ex index b1b7eec4..779a9369 100644 --- a/lib/microwaveprop_web/live/about_live.ex +++ b/lib/microwaveprop_web/live/about_live.ex @@ -23,59 +23,76 @@ defmodule MicrowavepropWeb.AboutLive do
-

What we're trying to do

- Microwave propagation above 10 GHz is dominated by the lower atmosphere: - humidity gradients, temperature inversions, ducting layers, rain cells, - and hyper-local refractivity structure. At 10, 24, 47, 76, 122, and 241 GHz - these effects decide whether a contact happens at 50 km or 500 km. This project was - started by Jim KM5PO, then picked up by Graham W5ISP. + Line-of-sight is wrong for the microwave bands. Every operator on + 902 MHz and up has made contacts over hills, through trees, and 200 km + past the horizon — contacts that shouldn't have worked if LOS were the + whole story. What's actually going on is the lower atmosphere: + temperature inversions, humidity gradients, ducting layers, and rain + cells that bend, trap, and absorb signals in ways that change by the + hour. The window for a good contact is often minutes, not days.

- Traditionally microwave contacts are ruled by line of sight, but we've - seen that is incorrect. This project is an attempt to build a data-driven - propagation prediction model specifically for the amateur microwave bands, - using hourly numerical weather forecasts, historical contact records, and - eventually calibrated beacon measurements. + Jim KM5PO started this project to try to predict those windows from + weather data. Graham W5ISP picked it up and is iterating on it from + there. The goal: a map that tells you when to get on the air, per + band, based on real atmospheric conditions and what we've learned + from the contacts you and everyone else have already made.

+
+

+ Heads up: + the scoring you see is hand-calibrated from the data we have so far. + It isn't updated in real time — I (Graham) re-fit the weights + manually when there's enough new data to move the needle. The more + contacts and beacon reports we collect, the better it gets. See the + <.link navigate={~p"/algo"} class="link link-hover font-semibold">algorithm page + for what's actually under the hood, and please <.link + navigate={~p"/submit"} + class="link link-hover font-semibold" + >submit your contacts. +

+
-

The approach

-

We pair two things that most propagation tools keep separate:

+

How it works, roughly

+

+ Two things, paired up. Most propagation tools keep them separate. +

  1. - Atmospheric state — hourly 3 km HRRR forecasts - (surface fields plus pressure-level profiles), hourly surface - observations from ASOS, 12-hourly upper-air soundings, and gridded - IEMRE reanalysis. From these we derive refractivity profiles, - ducting detection, minimum refractivity gradient, precipitable water, - boundary-layer depth, and a 9-factor composite score for every - 0.125° cell on a CONUS grid, for each hour of an 18-hour forecast. + The weather. + Hourly 3 km HRRR forecasts give surface fields and pressure-level + profiles. We derive refractivity profiles, minimum refractivity + gradient, ducting detection, boundary-layer depth, and precipitable + water for every grid cell. ASOS surface obs, 12-hourly upper-air + soundings, and gridded IEMRE reanalysis fill in the gaps. A 9-factor + composite score is written out for every 0.125° cell across CONUS, + for each hour of an 18-hour forecast.
  2. - Historical contacts — 58k+ amateur microwave QSOs - tagged with the atmospheric conditions at both ends and along the - path at the exact time they happened. That gives us a ground-truth - dataset we can use to calibrate the scoring weights and, eventually, - train a machine learning model that predicts contact success given - conditions. + The contacts. + 58k+ amateur microwave QSOs, each tagged with the atmosphere at + both ends of the path (and along it) at the time the contact + happened. That's a ground-truth dataset we can actually fit against + — "when you had this score, how far did the contact go, and did it + happen at all?"

- The current scoring algorithm is a weighted sum of nine factors - (humidity, time of day, T–Td depression, refractivity gradient, sky - cover, season, rain, wind, pressure) with band-dependent weights — - humidity helps at 10 GHz via enhanced refractivity but hurts at - 24+ GHz via absorption, for example. Every coefficient in that - formula is a hypothesis waiting to be tested against the contact - and beacon data. + The scoring is a weighted sum of nine factors: humidity, time of + day, T–Td depression, refractivity gradient, sky cover, season, rain, + wind, pressure. The weights are band-dependent because the physics + changes as you go up: humidity helps at 10 GHz (more refractivity) + and hurts at 24+ GHz (absorption). Every coefficient is a hypothesis + I'm checking against the contact and beacon data.

-

What we've collected

-

Live counts from the production database:

+

What we've collected so far

+

Straight from the production database:

<.stat_card label="Contacts" value={@stats.contacts} /> <.stat_card label="Weather stations" value={@stats.weather_stations} /> @@ -91,67 +108,64 @@ defmodule MicrowavepropWeb.AboutLive do
-

The stack

+

How it's built

  • - Backend: Elixir / Phoenix 1.8 with LiveView for the - real-time map, Ecto on PostgreSQL for storage, Oban for the - background data pipelines (HRRR fetch, terrain, ASOS, soundings, - IEMRE, solar indices, commercial links), Bandit as the HTTP server. + Elixir / Phoenix 1.8 + LiveView for everything web-facing. Ecto + on Postgres for storage. Oban runs the background pipelines + (HRRR pulls, terrain, ASOS, soundings, IEMRE, solar indices, + commercial links).
  • - Frontend: LiveView with Leaflet for maps, Canvas - tile layers for the HRRR heatmap (we render 0.125° cells at - interactive frame rates), Tailwind v4 + daisyUI for layout, esbuild - for JS bundling. + Leaflet + Canvas tile layers for the propagation heatmap — we + render 0.125° cells at interactive frame rates. Tailwind v4 + + daisyUI for layout, esbuild for JS bundling.
  • - Physics: ITU-R P.526-16 knife-edge + Deygout - 3-edge terrain diffraction, ITU-R P.838-3 rain attenuation, - dynamic k-factor from live HRRR refractivity gradients, great-circle - geometry for link budgets, Free-Space Path Loss with frequency- - dependent O₂ and H₂O absorption per band. + Physics: ITU-R P.526-16 knife-edge + Deygout 3-edge terrain + diffraction, ITU-R P.838-3 rain attenuation, dynamic k-factor + from live HRRR refractivity gradients, great-circle geometry, + FSPL with frequency-dependent O₂ and H₂O absorption.
  • - Machine learning: Nx / Axon / EXLA scaffolding for - a 13-feature (8 atmospheric + 4 cyclical temporal + 1 log-frequency) - feed-forward network, 64 → 32 → 1 sigmoid. Not trained yet — - waiting on a larger calibration dataset. + Data sources: NOAA HRRR (AWS S3, hourly analysis + 18 h + forecasts), Iowa Environmental Mesonet (ASOS and upper-air + soundings), IEMRE gridded reanalysis, SRTM 90 m terrain, SNMP + polling on seven commercial microwave links near DFW at 5-minute + intervals, and Copernicus ERA5 for pre-2014 contact enrichment.
  • - Data sources: NOAA HRRR model (AWS S3, hourly - analysis + 18 h forecasts), Iowa Environmental Mesonet (ASOS & - upper-air soundings), IEMRE gridded reanalysis, SRTM 90 m terrain - tiles, SNMP polling of seven commercial microwave links near DFW - at 5-minute intervals, Copernicus ERA5 for pre-2014 contact - enrichment. + There's Nx/Axon/EXLA scaffolding for a small feed-forward model + (13 features → 64 → 32 → 1 sigmoid). Not trained yet — waiting + on enough calibrated data to not overfit.
-

Where this goes next

+

What's next

  • - Beacon calibration. A distributed network of - amateur receivers continuously reporting CW beacon signal levels, - feeding ground-truth measurements back into the scoring algorithm. - Beacon submissions are now open to anyone via the Beacons page. + Beacon calibration. + A distributed network of amateur receivers reporting CW beacon + signal levels on a schedule. Beacon submissions are already open + to anyone via the Beacons page.
  • - Scoring weight calibration. Fit the 9-factor - weights against recorded QSO distances / counts per band so the - composite score reflects real propagation rather than intuition. + Weight calibration. + Fit the 9 factor weights against recorded distances / QSO counts + per band so the score reflects real propagation, not my + intuition.
  • - ML model training. Once we have enough labeled - contact+condition pairs, train the Axon model to replace or augment - the hand-tuned scoring function. + Training the model. + Once there are enough clean (contact, conditions) pairs, train + the Axon model to augment or replace the hand-tuned scoring.
  • - Better inputs. MRMS for precipitation at - 24+ GHz (where rain attenuation dominates), RTMA/URMA surface - analysis blending, GOES total precipitable water. + Better inputs. MRMS for precipitation at 24+ GHz where rain attenuation + dominates, RTMA/URMA surface analysis blending, GOES total + precipitable water.