Document ERA5, RTMA, and path-integrated HRRR in algo.md

Add sections for all three new data sources in Meteorological
Foundations: path-integrated scoring strategy, ERA5 reanalysis
for pre-2014 contacts, RTMA 15-minute surface analysis. Update
calibration dataset summary with ERA5/RAOB/RTMA coverage.

algo.md

@@ -12,7 +12,7 @@ The regime distinction matters because refractivity effects are *inverted* betwe
 
 ### Calibration Dataset
 
-**QSO data:** 58,367 total QSOs across 13 bands (ARRL Microwave Contest, 1992-2024). All tropospheric (distance < 3,000 km). Enriched with 41,071,398 HRRR model profiles (37,925 matched to contacts, 65% coverage), 15,092 IEMRE gridded hourly observations, 3,268 weather stations, and 58,361 terrain path profiles. **Critical bias: 99.5% of contacts are Aug-Sep** — all atmospheric correlations are effectively summer-only findings.
+**QSO data:** 58,367 total QSOs across 13 bands (ARRL Microwave Contest, 1992-2024). All tropospheric (distance < 3,000 km). Enriched with 41,071,398 HRRR model profiles (37,925 matched to contacts, 65% coverage), ERA5 reanalysis for the remaining 26,002 pre-HRRR contacts (0.25° hourly, 1940-present), 15,092 IEMRE gridded hourly observations, 10,935 RAOB soundings, 3,268 weather stations, and 58,361 terrain path profiles. RTMA (2.5 km, 15-min) supplements real-time surface conditions between HRRR hours. **Critical bias: 99.5% of contacts are Aug-Sep** — all atmospheric correlations are effectively summer-only findings.
 
 **Link data:** 7 commercial links near DFW (Princeton TX area) at 11/24/68 GHz, polled via SNMP at 5-minute intervals. All links use KTKI ASOS for weather correlation. Live polling is active; historical dataset from March 14-29 2026 (18,540 samples) was used for initial algorithm validation.
 
@@ -82,6 +82,42 @@ Primary atmospheric data source: NOAA HRRR v4 (3 km, hourly, 18h forecast cycle)
 
 **Vertical resolution limitation:** The 25 hPa pressure level spacing resolves features ≥100m thick. Thin surface ducts (50–100m) detectable by RAOB at ~10m resolution may be missed. Comparative statistics: HRRR gradients cluster −40 to −130 N/km (median −70); collocated RAOB gradients extend to −500+ N/km. Scoring thresholds are calibrated to HRRR-derived gradient distributions, with RAOB duct detections used as supplementary data where available (3,901 profiles from 112 stations).
 
+### Path-Integrated HRRR Scoring
+
+Contact scoring uses HRRR profiles at all path points (pos1, midpoint, pos2) rather than just the transmitter location. The aggregation strategy reflects physical reality:
+
+- **Best along path** for beneficial factors (refractivity gradient, surface pressure): a duct or frontal boundary at any point along the path can enable propagation.
+- **Worst along path** for harmful factors (rain, wind): a rain cell or turbulence at any segment degrades the entire link.
+- **Average along path** for neutral factors (temperature, dewpoint, PWAT, BL depth): path-integrated moisture and stability represent the bulk atmospheric state.
+
+Time-of-day, season, and sky cover are taken from the first profile (uniform along the path at these scales).
+
+### ERA5 Reanalysis (Historical Backfill)
+
+Secondary atmospheric source for contacts where HRRR is unavailable — primarily pre-2014 QSOs (26,002 contacts). ECMWF ERA5 provides global hourly reanalysis from 1940 to present at 0.25° resolution.
+
+**Access:** Copernicus Climate Data Store (CDS) API v1. Async job-based: submit request → poll for completion → download GRIB2. Typical latency 30s–5min per request.
+
+**Extracted fields:**
+- Single-level: T₂ₘ, Td₂ₘ, Psfc, U₁₀ₘ/V₁₀ₘ, TCWV (total column water vapour), BLH (boundary layer height)
+- Pressure levels: T, Td, Z at 1000–700 hPa (12 levels: 1000, 975, 950, 925, 900, 875, 850, 825, 800, 775, 750, 700)
+
+**Derived products:** Same as HRRR — N(h) profile, dN/dh minimum, M(h) duct detection, surface refractivity, via `SoundingParams.derive/1`. ERA5 profiles are stored in `era5_profiles` with the same schema as `hrrr_profiles` for interoperability.
+
+**Resolution trade-off:** ERA5's 0.25° (~28 km) horizontal resolution is 10x coarser than HRRR's 3 km, but its 12 pressure levels in the boundary layer provide comparable vertical resolution. For historical contacts, ERA5 is far better than no atmospheric data.
+
+**Unified lookup:** `Weather.best_profile_for_contact/1` tries HRRR first, falls back to ERA5. `Weather.profiles_along_path/1` does the same for path-integrated scoring.
+
+### RTMA (Real-Time Mesoscale Analysis)
+
+Supplementary surface data source: NOAA RTMA at 2.5 km resolution with 15-minute analysis cycles. Available on AWS S3 at `s3://noaa-rtma-pds/`.
+
+**What RTMA adds:** 4x temporal resolution over HRRR for surface conditions. Captures rapidly evolving mesoscale events (outflow boundaries, sea breeze fronts, convective gust fronts) between HRRR hourly cycles.
+
+**Fields:** T₂ₘ, Td₂ₘ, Psfc, U₁₀ₘ/V₁₀ₘ, visibility. No vertical profiles, HPBL, PWAT, or refractivity gradient — these still come from HRRR or ERA5.
+
+**Access:** GRIB2 via byte-range HTTP requests from S3, same pattern as HRRR. No authentication required.
+
 ### Surface Observations (ASOS)
 
 ASOS provides in-situ validation and additional parameters not in HRRR output: