Graham McIntire
5af4142e28
Three new analysis passes in scripts/recalibrate_algo.py that match the
scoring factors landed in commit 86364f4:
* NEXRAD composite reflectivity vs distance per band — checks that the
Marshall-Palmer rain rate in Scorer.dbz_to_rain_rate_mmhr/1 moves the
needle in the expected direction at 24+ GHz.
* hrrr_native_profiles.best_duct_band_ghz vs distance per band —
validates the 1.15× boost in Scorer.score_refractivity/4. Current
corpus shows essentially zero contacts with a native duct supporting
≥5 GHz at the target band, consistent with the 0.12% rate in Part 2c;
the boost is physics-correct but almost never fires today.
* Commercial-link rx_power degradation vs contemporaneous DFW contacts —
pearson + binned distance. Empty today because commercial_samples only
covers Mar 30–Apr 13 and contests don't overlap. Scaffolding is in
place so the moment contest-season overlap exists, the signal shows.
Report rebuilt at docs/algo-reports/2026-04-13-recalibration.md.
180 lines
7.3 KiB
Markdown
180 lines
7.3 KiB
Markdown
# Recalibration Analysis Report — 2026-04-13
|
||
|
||
> Auto-generated by `scripts/recalibrate_algo.py`. This report is the
|
||
> input for `algo.md` updates — diff against the prior dated section to
|
||
> see what's actually moved.
|
||
|
||
Connection: `postgres://prop:***@10.0.15.24:5432/prop`
|
||
Statement timeout: 20min
|
||
|
||
|
||
## Row counts and date ranges
|
||
|
||
| tbl | n | lo | hi |
|
||
|:---------------------|---------:|:-----------|:-----------|
|
||
| contacts | 58560 | 1991-05-04 | 2026-03-07 |
|
||
| era5_profiles | 0 | | |
|
||
| hrrr_native_profiles | 11472 | 2019-03-09 | 2025-09-21 |
|
||
| hrrr_profiles | 76151193 | 2016-09-17 | 2026-04-13 |
|
||
| iemre_observations | 15745 | | |
|
||
| nexrad_observations | 7286 | 2019-08-17 | 2024-09-22 |
|
||
| propagation_scores | 10543632 | 2026-04-13 | 2026-04-13 |
|
||
| rtma_observations | 0 | | |
|
||
| soundings | 16864 | 1990-06-07 | 2026-03-07 |
|
||
| surface_observations | 392444 | 1991-05-03 | 2026-04-13 |
|
||
| terrain_profiles | 58554 | | |
|
||
|
||
|
||
## Data gaps (these should NOT be zero in a healthy prod)
|
||
|
||
- **era5_rows**: 0
|
||
- **hrrr_climatology_rows**: 0
|
||
- **rtma_rows**: 0
|
||
- **metar5_rows**: 0
|
||
- **hrrr_complete_contacts**: 44147
|
||
- **hrrr_pending_contacts**: 14413
|
||
|
||
|
||
## Contacts by band (≥902 MHz)
|
||
|
||
| band_mhz | contacts | avg_km | max_km |
|
||
|-----------:|-----------:|---------:|---------:|
|
||
| 902 | 36 | 172.000 | 915.000 |
|
||
| 1296 | 74 | 259.000 | 1497.000 |
|
||
| 2304 | 15 | 91.000 | 348.000 |
|
||
| 3456 | 5 | 79.000 | 157.000 |
|
||
| 5760 | 4 | 97.000 | 148.000 |
|
||
| 10000 | 53698 | 213.000 | 2393.000 |
|
||
| 24000 | 3786 | 96.000 | 710.000 |
|
||
| 47000 | 764 | 66.000 | 343.000 |
|
||
| 75000 | 108 | 62.000 | 289.000 |
|
||
| 122000 | 35 | 27.000 | 139.000 |
|
||
| 134000 | 5 | 65.000 | 157.000 |
|
||
| 142000 | 4 | 47.000 | 79.700 |
|
||
| 145000 | 2 | 40.000 | 79.600 |
|
||
| 241000 | 15 | 29.000 | 114.400 |
|
||
| 288000 | 1 | 1.000 | 1.246 |
|
||
| 322000 | 1 | 1.000 | 1.400 |
|
||
| 403000 | 1 | 1.000 | 1.400 |
|
||
| 411000 | 1 | 0.000 | 0.050 |
|
||
|
||
|
||
## Monthly sounding ducting
|
||
|
||
| month | soundings | ducting_pct | avg_min_grad | avg_pwat |
|
||
|--------:|------------:|--------------:|---------------:|-----------:|
|
||
| 1 | 95 | 28.400 | -171.400 | 10.900 |
|
||
| 2 | 131 | 29.000 | -163.100 | 7.900 |
|
||
| 3 | 81 | 17.300 | -148.100 | 9.400 |
|
||
| 4 | 107 | 38.300 | -190.100 | 15.700 |
|
||
| 5 | 311 | 51.800 | -286.400 | 25.600 |
|
||
| 6 | 380 | 69.500 | -355.900 | 32.000 |
|
||
| 7 | 292 | 67.800 | -296.000 | 29.300 |
|
||
| 8 | 2572 | 54.900 | -261.200 | 34.600 |
|
||
| 9 | 2335 | 56.200 | -279.800 | 27.200 |
|
||
| 10 | 155 | 57.400 | -307.100 | 19.300 |
|
||
| 11 | 158 | 50.000 | -256.700 | 12.000 |
|
||
| 12 | 140 | 25.700 | -185.200 | 10.800 |
|
||
|
||
|
||
## HRRR native-profile duct distribution (best supportable band)
|
||
|
||
| band_bin | profiles | avg_inv_top_m | avg_theta_e_jump | avg_richardson |
|
||
|:-----------|-----------:|----------------:|-------------------:|-----------------:|
|
||
| 15-30 GHz | 10 | 2091.000 | 5.900 | 8.690 |
|
||
| 30-75 GHz | 4 | 3860.000 | 5.000 | 12.120 |
|
||
| 5-15 GHz | 98 | 4408.000 | 25.700 | 12.550 |
|
||
| <5 GHz | 1432 | 2097.000 | 5.400 | 18.420 |
|
||
| none | 9928 | 11497.000 | 80.500 | 38.270 |
|
||
|
||
|
||
## Contact ↔ HRRR per-band Pearson correlations (matched n=1020)
|
||
|
||
| band_mhz | n | rho_pr | rho_dpc | rho_pwat | rho_sref | rho_grad | rho_tc | rho_hpbl |
|
||
|-----------:|--------:|---------:|----------:|-----------:|-----------:|-----------:|---------:|-----------:|
|
||
| 1296.000 | 63.000 | -0.446 | 0.150 | -0.004 | 0.321 | -0.308 | -0.313 | -0.383 |
|
||
| 10000.000 | 680.000 | -0.109 | 0.044 | 0.002 | 0.182 | -0.127 | -0.107 | -0.198 |
|
||
| 24000.000 | 147.000 | -0.290 | -0.179 | -0.219 | 0.096 | -0.036 | -0.300 | -0.223 |
|
||
| 47000.000 | 75.000 | -0.111 | 0.016 | 0.007 | 0.217 | -0.175 | 0.089 | -0.154 |
|
||
|
||
|
||
## 10 GHz: HPBL bin distance distribution
|
||
|
||
| bin | n | avg_km | p50_km |
|
||
|:----------|----:|---------:|---------:|
|
||
| <200 | 54 | 217.600 | 174.000 |
|
||
| 200-500 | 177 | 184.800 | 160.000 |
|
||
| 500-1000 | 177 | 163.800 | 126.000 |
|
||
| 1000-1500 | 104 | 158.800 | 136.000 |
|
||
| 1500-2000 | 94 | 142.800 | 136.000 |
|
||
| >=2000 | 74 | 100.300 | 90.000 |
|
||
|
||
|
||
## 10 GHz: pressure bin distance distribution
|
||
|
||
| bin | n | avg_km | p50_km |
|
||
|:----------|----:|---------:|---------:|
|
||
| <990 | 103 | 214.200 | 136.000 |
|
||
| 990-1000 | 536 | 154.800 | 128.000 |
|
||
| 1000-1010 | 28 | 164.600 | 115.000 |
|
||
| 1010-1020 | 2 | 23.000 | 23.000 |
|
||
| >=1020 | 11 | 103.100 | 23.000 |
|
||
|
||
|
||
## NEXRAD composite reflectivity vs distance (matched n=102)
|
||
|
||
Sanity check for the rain-attenuation direction: at rain-sensitive bands higher `max_dbz` should map to shorter contacts. At 10 GHz the effect should be barely detectable.
|
||
|
||
|
||
**10 GHz:**
|
||
|
||
| bin | n | avg_km | p50_km |
|
||
|:-----------------|----:|---------:|---------:|
|
||
| none (<5) | 59 | 155.700 | 143.000 |
|
||
| drizzle (5-20) | 7 | 56.400 | 49.000 |
|
||
| light (20-30) | 13 | 95.200 | 40.000 |
|
||
| moderate (30-45) | 4 | 97.200 | 92.000 |
|
||
| heavy (>=45) | 1 | 213.000 | 213.000 |
|
||
|
||
|
||
**24 GHz:**
|
||
|
||
| bin | n | avg_km | p50_km |
|
||
|:-----------------|----:|---------:|---------:|
|
||
| none (<5) | 5 | 29.800 | 23.000 |
|
||
| drizzle (5-20) | 1 | 49.000 | 49.000 |
|
||
| light (20-30) | 5 | 34.000 | 40.000 |
|
||
| moderate (30-45) | 1 | 60.000 | 60.000 |
|
||
|
||
|
||
## hrrr_native_profiles.best_duct_band_ghz vs distance (matched n=619)
|
||
|
||
Validates the 1.15× boost in `Scorer.score_refractivity/4`: contacts where the native duct supports the target band should run longer than those where it does not.
|
||
|
||
|
||
**10 GHz:**
|
||
|
||
| bin | n | avg_km | p50_km |
|
||
|:-------|--------:|---------:|---------:|
|
||
| none | 447.000 | 140.800 | 128.000 |
|
||
| <5 GHz | 38.000 | 150.000 | 114.500 |
|
||
|
||
|
||
**24 GHz:**
|
||
|
||
| bin | n | avg_km | p50_km |
|
||
|:-------|-------:|---------:|---------:|
|
||
| none | 97.000 | 47.600 | 32.000 |
|
||
| <5 GHz | 3.000 | 56.300 | 60.000 |
|
||
|
||
|
||
**47 GHz:**
|
||
|
||
| bin | n | avg_km | p50_km |
|
||
|:------|-------:|---------:|---------:|
|
||
| none | 16.000 | 54.700 | 42.000 |
|
||
|
||
|
||
## Commercial-link rx_power degradation vs contemporaneous DFW contacts (matched n=0)
|
||
|
||
_No DFW-zone contacts fall inside the commercial_samples date window yet. Expected to stay empty until the Aug/Sep contest season produces contacts that overlap with live SNMP polling._
|