Add meteorological foundations section to algo.md

Comprehensive description of atmospheric science behind the predictions
for meteorologist review: refractivity-propagation connection, four
ducting mechanisms, HRRR model usage, frequency-dependent effects,
diurnal/seasonal cycles, and known limitations.

algo.md

@@ -29,6 +29,170 @@ The regime distinction matters because refractivity effects are *inverted* betwe
 
 ---
 
+## Meteorological Foundations
+
+This section describes the atmospheric science underlying the propagation predictions, written for review by meteorologists. The system predicts microwave radio propagation conditions (10-241 GHz) using NWP model output and surface observations, focusing on tropospheric ducting and gaseous absorption.
+
+### Why Meteorology Matters for Microwave Radio
+
+Microwave radio signals (wavelengths 1 mm to 3 cm) travel through the troposphere and are affected by the same atmospheric structure that meteorologists analyze daily. Unlike HF radio which depends on the ionosphere, microwave propagation is entirely a tropospheric phenomenon governed by:
+
+- **Temperature and moisture profiles** in the lowest 2 km AGL
+- **Boundary layer structure** (mixed layer depth, capping inversions, nocturnal stable layers)
+- **Synoptic-scale weather patterns** (frontal boundaries, subsidence, marine layers)
+- **Gaseous composition** (water vapor and O2 density along the path)
+
+Under standard atmospheric conditions, microwave signals are limited to roughly line-of-sight range (~50 km for ground-level antennas). Enhanced propagation occurs when atmospheric structure bends the radio beam downward (super-refraction) or traps it in a waveguide (ducting), extending range to hundreds or thousands of kilometers.
+
+### The Refractivity-Propagation Connection
+
+Radio refractivity N is the key bridge between meteorology and propagation. N is computed from standard meteorological variables:
+
+```
+N = 77.6 * P/T + 3.73×10⁵ * e/T²
+```
+
+where P is pressure (hPa), T is temperature (K), and e is water vapor pressure (hPa). The first term is the "dry" component (pressure/temperature), the second is the "wet" component (moisture). At microwave frequencies, both terms matter — the wet term contributes roughly 20-40% of total N in the boundary layer.
+
+The **vertical gradient of N** (dN/dh) determines how radio beams bend:
+
+| dN/dh (N-units/km) | Propagation Regime | Meteorological Condition |
+|---|---|---|
+| > 0 | Sub-refraction (beam bends upward) | Strong lapse, well-mixed dry BL |
+| 0 to −79 | Normal refraction | Standard atmosphere |
+| −79 to −157 | Super-refraction (beam bends toward earth) | Moderate inversion or moisture lapse |
+| < −157 | Ducting (beam trapped in waveguide) | Strong temperature inversion with moisture contrast |
+
+The critical threshold is **−157 N/km**: below this, the radio beam curves faster than the Earth's surface, creating a tropospheric duct.
+
+### Ducting Mechanisms
+
+Four meteorological mechanisms produce ducting conditions relevant to this system. Each creates a sharp negative refractivity gradient through a different combination of temperature increase and/or moisture decrease with height:
+
+**1. Radiation/Nocturnal Inversion Ducts**
+- **Mechanism**: Radiative cooling of the surface after sunset creates a strong surface-based temperature inversion. Moisture is often trapped below the inversion cap.
+- **Meteorological signature**: Clear skies, light winds, high pressure. Surface temperature drops rapidly after sunset while air aloft remains warm.
+- **Timing**: Forms 1-3 hours after sunset, strongest just before sunrise. Dissipates within 1-2 hours of solar heating.
+- **Vertical extent**: Typically 50-300m AGL (shallow boundary layer).
+- **dN/dh**: −100 to −300 N/km in the inversion layer.
+- **This is the primary ducting mechanism in our dataset** — the majority of amateur microwave contacts occur during dawn/sunrise windows specifically to exploit these ducts.
+
+**2. Advection Ducts (Marine/Coastal)**
+- **Mechanism**: Warm, dry air advects over a cooler surface (ocean, lake, or cold ground). Temperature increases with height while moisture decreases sharply at the air-mass boundary.
+- **Meteorological signature**: Onshore warm-air advection, sea breeze fronts, warm sector of extratropical cyclones passing over cool water.
+- **Timing**: Can persist for hours to days, not limited to diurnal cycle.
+- **Vertical extent**: 50-500m, can be elevated.
+- **dN/dh**: −200 to −500+ N/km. Among the strongest ducts observed.
+- **Relevance**: Gulf Coast, California coast, and Great Lakes regions. Our dataset shows Florida/Gulf Coast (EL) and California (CM) outperform inland regions at 10 GHz.
+
+**3. Subsidence Inversion Ducts**
+- **Mechanism**: Large-scale sinking motion (subsidence) in high-pressure systems creates a temperature inversion aloft. Moisture below the subsidence inversion is trapped, creating a sharp dN/dh gradient at the inversion base.
+- **Meteorological signature**: Ridge axis or high-pressure center, clear skies above inversion, possibly hazy/humid below. Visible on soundings as a sharp temperature increase at 800-900 hPa with dewpoint drop above.
+- **Timing**: Persistent, follows synoptic-scale ridging.
+- **Vertical extent**: 500-2000m AGL (elevated duct).
+- **dN/dh**: −100 to −200 N/km (weaker than advection ducts but larger scale).
+
+**4. Frontal/Boundary Ducts**
+- **Mechanism**: Temperature and moisture gradients along frontal boundaries, outflow boundaries, and drylines create localized refractivity gradients.
+- **Meteorological signature**: Cold fronts (sharp moisture drop behind front), warm fronts (overrunning warm moist air), dryline passages.
+- **Timing**: Transient, associated with frontal passage.
+- **Our data confirms**: Low-pressure systems correlate with longer propagation distances (262 km avg vs 196 km for high pressure at 10 GHz), likely due to frontal boundary structure rather than pressure itself.
+
+### HRRR Model Data in the Prediction System
+
+The system ingests NOAA's High-Resolution Rapid Refresh (HRRR) model output as its primary atmospheric data source:
+
+- **Model**: HRRR v4, 3 km horizontal resolution, hourly runs, 18-hour forecasts
+- **Source**: NOAA AWS S3 archive (GRIB2 format)
+- **Extracted fields**:
+  - Surface: 2m temperature, 2m dewpoint, surface pressure, 10m wind (u/v), total cloud cover, precipitation
+  - Boundary layer: HPBL (planetary boundary layer height), PWAT (precipitable water)
+  - Upper air: temperature, dewpoint, and geopotential height at 1000, 975, 950, 925, 900, 850, 800, 700 hPa
+
+From these fields, the system derives:
+
+1. **Surface refractivity N** (ITU-R P.453-14 formula)
+2. **Refractivity profile** N(h) at each pressure level
+3. **Minimum refractivity gradient** dN/dh — the primary ducting indicator
+4. **Modified refractivity** M(h) = N + 0.157h for duct detection (ducting where dM/dh < 0)
+5. **Absolute humidity** from temperature and dewpoint
+6. **Effective earth radius factor k** for terrain diffraction analysis
+
+**HRRR vs radiosonde comparison**: HRRR profiles have 8 standard pressure levels in the lowest 3 km, while radiosondes report at ~10m vertical resolution. This means HRRR can detect large-scale inversions but may miss thin ducting layers (<100m). Our gradient statistics reflect this: HRRR-derived gradients cluster between −40 and −130 N/km (median −70), while radiosonde-derived gradients extend to −500+ N/km. The scoring thresholds are calibrated separately for HRRR data.
+
+### Surface Observations (ASOS)
+
+ASOS stations provide ground-truth observations that complement the HRRR model:
+
+- **Temperature and dewpoint**: Used to compute absolute humidity and T-Td depression
+- **Wind speed**: Light winds favor inversion persistence; strong winds indicate mechanical mixing
+- **Sky condition**: Clear skies promote radiative cooling (nocturnal inversions); overcast prevents it
+- **Pressure**: Used for absolute refractivity calculation and trend analysis
+- **Precipitation**: Rain causes direct path attenuation (ITU-R P.838-3), scaling with frequency
+
+### Frequency-Dependent Atmospheric Effects
+
+A critical meteorological nuance: the same atmospheric state produces opposite effects at different frequencies.
+
+**At 10 GHz (3 cm wavelength)**:
+- Gaseous absorption is negligible (0.012 dB/km total)
+- Propagation is dominated by refractivity structure — ducts and inversions
+- More moisture increases N, increasing beam bending — **humidity helps**
+- Rain attenuation is mild (0.05 dB/km at 4 mm/hr)
+
+**At 24 GHz (1.25 cm wavelength)**:
+- Near the 22.235 GHz water vapor absorption line — humidity causes significant loss
+- 0.012 dB/km per g/m³ of water vapor (10x the rate at 10 GHz)
+- **Humidity hurts** — dry air allows longer paths despite reduced ducting benefit
+- Rain attenuation 6x higher than 10 GHz
+
+**At 47+ GHz (sub-cm wavelength)**:
+- Increasingly dominated by gaseous absorption
+- 68 GHz is near the 60 GHz O2 absorption complex (0.9 dB/km baseline)
+- 241 GHz is between H2O lines at 183 and 325 GHz (0.3 dB/km per g/m³)
+- Propagation windows exist at 75 GHz and 134 GHz
+- **Ducting is the only mechanism** enabling contacts beyond ~20 km at these frequencies
+
+### Diurnal Cycle and Timing
+
+The boundary layer diurnal cycle is the most predictable propagation driver:
+
+1. **Sunset to sunrise**: Radiative cooling builds surface inversions. BL collapses from 1000-2000m (afternoon) to 100-300m. Ducting probability rises.
+2. **Dawn (sunrise ± 1.5h)**: Peak inversion strength. BL is shallowest. This is the preferred operating window for amateur microwave contacts.
+3. **Morning transition (sunrise + 1.5 to +3h)**: Solar heating erodes inversion from below. Ducting weakens.
+4. **Afternoon (sunrise + 6h onward)**: Full convective mixing. BL deepest. Worst propagation conditions. Turbulent scattering dominates any residual refractivity gradients.
+
+Our data quantifies this diurnal effect by frequency: the night/dawn enhancement over afternoon baseline is +4% at 10 GHz, +28% at 24 GHz, +36% at 47 GHz, and +360% at 75 GHz. At the highest frequencies, **time of day is the dominant predictor** — more important than any single weather variable.
+
+### Seasonal Cycle
+
+Ducting probability from our radiosonde analysis (3,901 profiles, continental US):
+
+- **Peak**: June-July (69-77% ducting frequency). Warm, moist boundary layers with strong nocturnal inversions.
+- **Secondary peak**: October-November (55-65%). Autumn radiative cooling with residual moisture.
+- **Minimum**: March (10.8%). Transitional season with frequent mixing events.
+- **Winter**: December-February (12-20%). Cold, dry atmosphere limits refractivity gradients.
+
+At 24+ GHz, the seasonal pattern inverts: **winter is best** because low absolute humidity minimizes absorption, even though ducting is less frequent. The system's band-specific seasonal scoring reflects this inversion.
+
+### Known Limitations and Areas for Meteorologist Review
+
+1. **HRRR vertical resolution**: 8 pressure levels below 700 hPa may miss thin ducting layers. Soundings show gradients 3-5x stronger than HRRR at the same locations. The system uses HRRR-calibrated thresholds, but this may underestimate ducting potential.
+
+2. **Mesoscale effects**: Sea breeze fronts, outflow boundaries, and terrain-induced convergence zones can create localized ducts not resolved by the 3 km HRRR grid. The system does not currently account for these sub-grid phenomena.
+
+3. **Precipitation model**: Rain attenuation uses ITU-R P.838-3 coefficients but has not been validated against measured data (no rain events in the commercial link dataset). Rainscatter (using rain as a scattering medium for propagation) is observed at 24 GHz but not modeled.
+
+4. **Fog and cloud effects**: The system uses cloud cover percentage as a sky condition proxy but does not model fog/cloud attenuation directly. At 10 GHz, fog attenuation is negligible; at 75+ GHz, dense fog can add 1-5 dB/km.
+
+5. **Turbulence scintillation**: Not modeled. On long LOS paths, convective turbulence causes amplitude scintillation (rapid signal fluctuations) that is distinct from multipath fading.
+
+6. **Time resolution**: The system updates hourly using HRRR analysis/forecast. Rapidly evolving mesoscale events (convective outflows, sea breeze onset) may be captured by the forecast hours but with reduced accuracy beyond the analysis hour.
+
+We welcome feedback from operational meteorologists on these assumptions and any atmospheric processes we may be underweighting or overlooking.
+
+---
+
 ## Part 1: Atmospheric Physics
 
 ### Absolute Humidity