docs(algo): align algo.md with Richardson-gated scoring implementation

Three small corrections now that the Richardson gate is actually
wired into Scorer.score_refractivity/5:

- Fix arity reference /4 → /5 in Part 2c.
- Reconcile the backtest section's "bulk_richardson is dead" finding
  with the new gate wiring — they're compatible claims. Richardson
  is dead as a standalone ML discriminator (stable and unstable
  inversions both produce ducts), but useful as a gate on the
  native-duct boost (a duct with high Ri is likely to mix out
  before the signal gets through).
- Note that nil Richardson is treated as "no info" so older native
  profiles without the column still receive the unconditional boost.
This commit is contained in:
Graham McIntire 2026-04-18 12:42:20 -05:00
parent d3b0420457
commit b5decefa8e
No known key found for this signature in database
GPG key ID: F4ABF488E6029E59

View file

@ -700,7 +700,7 @@ The continuous-vs-binary signal is sharp: ducting soundings have avg gradient **
**Only 13.5% of native-resolution profiles contain a duct at all**, and of those, **92.7% support only sub-5 GHz frequencies**. Microwave-supporting ducts (15+ GHz) are 14 / 11,472 = **0.12%** of the population. This quantifies why microwave tropospheric ducting is so much rarer than the VHF-tropo experience suggests — the sounding-derived "ducting %" includes a lot of weak ducts that don't support 10+ GHz at all. `best_duct_band_ghz` is the per-cell upper bound the refractivity scorer consults for the native-duct boost (below). **Only 13.5% of native-resolution profiles contain a duct at all**, and of those, **92.7% support only sub-5 GHz frequencies**. Microwave-supporting ducts (15+ GHz) are 14 / 11,472 = **0.12%** of the population. This quantifies why microwave tropospheric ducting is so much rarer than the VHF-tropo experience suggests — the sounding-derived "ducting %" includes a lot of weak ducts that don't support 10+ GHz at all. `best_duct_band_ghz` is the per-cell upper bound the refractivity scorer consults for the native-duct boost (below).
**Bulk Richardson number** is systematically lower in duct cells (8.718.4) than non-duct cells (38.3), reflecting the dynamic-stability requirement for thin trapping layers. `Scorer.score_refractivity/4` uses this as a gating condition on the native-duct boost: a low `best_duct_band_ghz` reading with a high Richardson number is likely noise (a duct that would get shredded by mechanical mixing), so the 1.15× boost only applies when Richardson is in the stable regime (< 25). **Bulk Richardson number** is systematically lower in duct cells (8.718.4) than non-duct cells (38.3), reflecting the dynamic-stability requirement for thin trapping layers. `Scorer.score_refractivity/5` uses this as a gating condition on the native-duct boost: a `best_duct_band_ghz` reading with a high Richardson number is likely noise (a duct that would get shredded by mechanical mixing), so the 1.15× boost only applies when Richardson is in the stable regime (< 25). A nil Richardson is treated as "no information" rather than "turbulent", so older profiles without the column still receive the unconditional boost.
### Per-band signal highlights ### Per-band signal highlights
@ -2323,7 +2323,7 @@ Features are grouped by data source and physical quantity:
| ~~duct_usable_10ghz~~ | hrrr_native_profiles | DEAD — no discrimination (always 1.0 for both QSO and baseline) | | ~~duct_usable_10ghz~~ | hrrr_native_profiles | DEAD — no discrimination (always 1.0 for both QSO and baseline) |
| ~~duct_usable_24ghz~~ | hrrr_native_profiles | DEAD — no discrimination (always 1.0) | | ~~duct_usable_24ghz~~ | hrrr_native_profiles | DEAD — no discrimination (always 1.0) |
| ~~duct_usable_47ghz~~ | hrrr_native_profiles | DEAD — no discrimination (always 1.0) | | ~~duct_usable_47ghz~~ | hrrr_native_profiles | DEAD — no discrimination (always 1.0) |
| ~~bulk_richardson~~ | hrrr_native_profiles | DEAD — near-identical means (24.8 vs 23.4), no lift | | bulk_richardson | hrrr_native_profiles | Dead as a standalone ML feature (near-identical means — see backtest), but wired as a **gate** on the native-duct boost in `Scorer.score_refractivity/5`: a duct reading with Ri ≥ 25 is suppressed because mechanical mixing would likely destroy the trapping layer before it carried a signal. |
**Climatology and remote sensing:** **Climatology and remote sensing:**
@ -2364,7 +2364,7 @@ Sample: 5,000 QSOs, 11,431 native profiles across 499 HRRR hours (2019-2024).
- **best_duct_freq** confirms that QSO-producing ducts are physically stronger (trap lower frequencies) - **best_duct_freq** confirms that QSO-producing ducts are physically stronger (trap lower frequencies)
- **duct_thickness** being inverted makes physical sense: shallow surface ducts (100-200m) produce the strongest trapping for microwave bands, while thick ducts (>200m) are weaker elevated features - **duct_thickness** being inverted makes physical sense: shallow surface ducts (100-200m) produce the strongest trapping for microwave bands, while thick ducts (>200m) are weaker elevated features
- **duct_usable_*ghz** features are dead because nearly all detected ducts are thick enough to trap 10-47 GHz — the threshold is too low to discriminate - **duct_usable_*ghz** features are dead because nearly all detected ducts are thick enough to trap 10-47 GHz — the threshold is too low to discriminate
- **bulk_richardson** shows no signal, likely because both stable (high Ri) and unstable (low Ri) conditions can produce inversions; Ri alone doesn't predict duct quality - **bulk_richardson** shows no signal as a standalone ML feature because both stable (high Ri) and unstable (low Ri) conditions can produce inversions; Ri alone doesn't predict duct quality. It does work as a **gate** on the native-duct boost though — see `Scorer.score_refractivity/5` — because the question "is this duct dynamically stable enough to survive" is different from "does this cell have a duct at all"
**Implications for real-time scoring:** The native features cannot currently be used in real-time propagation scoring because the native HRRR product (~530 MB/hour) is too expensive to fetch for the full CONUS grid. However, the findings validate the existing scorer's physics: humidity, td_depression, and refractivity factors capture the same mechanisms (moisture-driven refractivity, inversion strength) that the native features measure more directly. Future work could incorporate native data for specific paths or high-interest regions. **Implications for real-time scoring:** The native features cannot currently be used in real-time propagation scoring because the native HRRR product (~530 MB/hour) is too expensive to fetch for the full CONUS grid. However, the findings validate the existing scorer's physics: humidity, td_depression, and refractivity factors capture the same mechanisms (moisture-driven refractivity, inversion strength) that the native features measure more directly. Future work could incorporate native data for specific paths or high-interest regions.