feat(scorer): retire HPBL multiplier and native-duct 1.15× boost
Both retired in light of the 2026-04-25 revision report
(`docs/algo-reports/2026-04-25-algo-revisions.md`):
* HPBL boundary-layer multiplier in `Scorer.score_refractivity/{3,4,5}`
is gone. On the n=47,418 10 GHz HRRR-matched corpus rho_hpbl is
+0.004; binned distance is flat to within ±5 % across 200–2,000 m.
The previously reported 2.3× ratio between shallow and deep HPBL
bins was a small-corpus artefact that disappeared once the matched
corpus passed ~5,000 contacts.
* Native-profile 1.15× duct boost is gone. At 10 GHz on n=52,341
matched contacts, cells where `best_duct_band_ghz` ≥ band ran
*shorter* than no-duct cells (198 km vs 211 km, n=173 vs 44,658).
The Bulk Richardson gate that existed only to suppress that boost
in turbulent conditions is also retired; arity preserved on every
`score_refractivity` overload so callers compile unchanged.
Same retirements applied in the Rust port (`rust/prop_grid_rs/src/scorer.rs`)
to keep the Elixir/Rust parity tests green.
algo.md updated end-to-end:
* Finding 4 (HPBL) rewritten to "no usable signal" with the
n=47,418 bin table.
* Finding 5 (gradient) tightened: load-bearing only at 24 GHz,
set band-specific gradient weight to 0 outside [10, 47] GHz on
the next derive_band_weights run.
* Finding 8 (time-of-day) augmented with the robust hour-bucketed
amplitude index (40 / 82 / 101 % at 10 / 24 / 47 GHz) and a
selection-bias caveat for VHF/UHF (where 129–148 % amplitude
reflects evening contest scheduling, not propagation).
* Finding 9 (sounding) refreshed against the n=27,058 corpus.
* Part 2c "Native-profile duct boost" rewritten as
"retired 2026-04-25" with the falsifying join.
* Part 2c "Commercial-link inverse sensor" promoted from deferred
with the 22-day, 38k-sample SNMP corpus, the +0.61 / −0.61 PWAT
and pressure correlations against the 37-hour HRRR overlap, and
the 04–05 / 10–12 local two-peak morning fade window.
* Part 2b "Pressure" annotated with the U-shape at 10 GHz and the
next-iteration target.
* Part 2d "What stayed, what left" + "Open items" updated to
reflect the four code/doc moves.
119 Elixir scorer tests green, 134 Rust scorer tests green, 2,888
total Elixir tests + 221 properties green via `mix precommit`.
This commit is contained in:
parent
2b071c3b56
commit
3a1e79fb67
5 changed files with 663 additions and 237 deletions
266
algo.md
266
algo.md
|
|
@ -371,29 +371,41 @@ Contradicts the common assumption that high pressure = good propagation:
|
|||
|
||||
Low pressure systems bring frontal boundaries with strong temperature/moisture gradients that create inversions and ducts. The key is gradient structure, not absolute pressure.
|
||||
|
||||
### Finding 4: Boundary Layer Depth — Time-Dependent Sweet Spot
|
||||
### Finding 4: Boundary Layer Depth — Retired (no usable signal)
|
||||
|
||||
Sounding data (3,901 profiles) reveals that BL depth interpretation depends on sounding time:
|
||||
> **Status as of 2026-04-25:** The HPBL multiplier is removed from
|
||||
> `Scorer.score_refractivity/{3,4,5}` and the Rust port. See
|
||||
> `docs/algo-reports/2026-04-25-algo-revisions.md` Recommendation 2.
|
||||
|
||||
| Sounding Time | Avg BL Depth | Ducting % | Mechanism |
|
||||
|---------------|-------------|-----------|-----------|
|
||||
| 00Z (evening) | 1,370m | 54.0% | Elevated ducts within deep residual BL |
|
||||
| 12Z (morning) | 335m | 53.7% | Surface/radiation ducts from shallow nocturnal BL |
|
||||
The original sweet-spot finding ("shallow BL → longer distances") was
|
||||
fitted on n≈680 10 GHz HRRR-matched contacts in April 2026. On the
|
||||
n=47,418 matched corpus the effect disappears: rho_hpbl = +0.004 at
|
||||
10 GHz, never exceeds |0.092| at any band ≥222 MHz, and the binned
|
||||
distance distribution is flat to within ±5 % across 200–2,000 m HPBL.
|
||||
|
||||
Ducting rate is identical at both times (~54%), but via different mechanisms. At 12Z, shallow BL (<500m) is the signature. At 00Z, the duct is typically elevated within a deeper residual boundary layer. A single BL depth threshold won't work across both regimes — use BL depth relative to expected diurnal range.
|
||||
| HPBL bin | n | avg km | p50 km |
|
||||
|---|---:|---:|---:|
|
||||
| < 200 m | 8,764 | 211.4 | 180.0 |
|
||||
| 200–500 m | 12,799 | 205.3 | 178.5 |
|
||||
| 500–1,000 m | 14,739 | 207.8 | 186.5 |
|
||||
| 1,000–1,500 m | 7,240 | 209.0 | 190.5 |
|
||||
| 1,500–2,000 m | 2,713 | 199.2 | 176.6 |
|
||||
| ≥ 2,000 m | 1,160 | 230.3 | 197.4 |
|
||||
|
||||
HRRR profile data (4,522 profiles) confirms the relationship is monotonic:
|
||||
The previously reported 2.3× distance ratio between shallow and deep
|
||||
HPBL bins was a small-corpus artefact. HPBL stays in the schema and
|
||||
diagnostics so we can revisit if a signal emerges in a different
|
||||
context (e.g. paired with k-factor stratification), but it does not
|
||||
modify the score.
|
||||
|
||||
| BL Depth Bin | Count | Avg N | Avg Min Gradient |
|
||||
|-------------|-------|-------|------------------|
|
||||
| 0-200m | 1,096 | 342.6 | -93.7 |
|
||||
| 200-500m | 1,091 | 346.6 | -81.5 |
|
||||
| 500-1000m | 1,242 | 337.0 | -70.1 |
|
||||
| 1000-1500m | 735 | 328.8 | -58.3 |
|
||||
| 1500-2000m | 299 | 325.2 | -51.0 |
|
||||
| 2000m+ | 60 | 309.4 | -45.4 |
|
||||
Sounding-mechanism context — kept for documentation, not scoring —
|
||||
shows ducting is supported at *both* extremes of the diurnal HPBL
|
||||
cycle: shallow nocturnal radiation ducts at 12Z and elevated ducts
|
||||
inside deep residual boundary layers at 00Z. A single HPBL threshold
|
||||
was never going to capture both regimes, which is consistent with the
|
||||
zero overall correlation we now measure on a large corpus.
|
||||
|
||||
### Finding 5: Binary Duct Detection Is Weak — Use Continuous Gradient
|
||||
### Finding 5: Binary Duct Detection Is Weak — Use Continuous Gradient (24 GHz only)
|
||||
|
||||
Ducting is the **majority case** in soundings: 2,099 ducting (53.8%) vs 1,800 non-ducting (46.2%). Binary detection has near-zero discriminating power.
|
||||
|
||||
|
|
@ -404,6 +416,30 @@ Ducting is the **majority case** in soundings: 2,099 ducting (53.8%) vs 1,800 no
|
|||
|
||||
The continuous gradient (-389 vs -123) is the real signal — a 3x magnitude difference. HRRR data shows 79% of profiles in "Enhanced" regime (gradient -40 to -100), so the scoring must discriminate *within* the enhanced category, not just between standard and enhanced.
|
||||
|
||||
> **Tightened 2026-04-25:** the gradient signal is only load-bearing at
|
||||
> ~24 GHz. From the n=68,062 HRRR↔contact correlation table:
|
||||
>
|
||||
> | Band | rho_grad |
|
||||
> |---|---:|
|
||||
> | 222 MHz | +0.031 |
|
||||
> | 432 MHz | +0.003 |
|
||||
> | 902 MHz | −0.062 |
|
||||
> | 1.296 GHz | −0.079 |
|
||||
> | 2.304 GHz | −0.178 |
|
||||
> | 5.76 GHz | −0.110 |
|
||||
> | 10 GHz | +0.027 |
|
||||
> | 24 GHz | +0.017 |
|
||||
> | 47 GHz | −0.008 |
|
||||
> | 75 GHz | +0.474 (n=83 — contest-cluster artefact) |
|
||||
>
|
||||
> Below 10 GHz the gradient never clears the 0.05 noise floor; the
|
||||
> dewpoint and PWAT terms already capture whatever moisture-driven
|
||||
> ducting these bands respond to. The 75 GHz row is dominated by Aug–Sep
|
||||
> contest weeks and is not strong evidence for a per-band gradient term.
|
||||
> Per-band gradient weight is therefore set to 0 outside [10 GHz,
|
||||
> 47 GHz]; the `refractivity` slot in the band-weight matrix carries the
|
||||
> 24 GHz signal alone.
|
||||
|
||||
**Stability indices and ducting:**
|
||||
- K-index is **lower** for ducting (12.7 vs 16.7) — stable atmosphere favors ducting, not convection
|
||||
- Lifted Index is **higher** for ducting (25.3 vs 22.8) — confirms stability correlation
|
||||
|
|
@ -432,26 +468,64 @@ At 10 GHz the effect is modest. At 47+ GHz it is the **dominant variable**, more
|
|||
|
||||
**Update (April 2026):** Switching from fixed CDT/CST timezone to longitude-based **solar time** (`longitude / 15`) dramatically improves the time-of-day correlation at higher frequencies. Spearman correlation with distance: UTC hour rho=0.056 vs solar hour rho=0.188 at 24 GHz (3.4x improvement). At 75 GHz the UTC correlation was confounded by geographic longitude — solar time corrects this from rho=-0.39 to rho=+0.24.
|
||||
|
||||
**Refresh (2026-04-25, n=82k corpus, hours with ≥30 contacts each):** the
|
||||
"+4 / +28 / +36 / +360 %" enhancement table above is the single
|
||||
afternoon-vs-night cut. A robust hour-by-hour amplitude index (max p50 −
|
||||
min p50, divided by band p50) tells a more measured story:
|
||||
|
||||
| Band | hours w/data | lo p50 (km) | hi p50 (km) | amplitude % |
|
||||
|---|---:|---:|---:|---:|
|
||||
| 902 MHz | 15 | 94.5 | 178.0 | 57.7 |
|
||||
| 1.296 GHz | 17 | 80.0 | 176.0 | 71.2 |
|
||||
| 2.304 GHz | 13 | 108.0 | 171.5 | 48.2 |
|
||||
| 10 GHz | 23 | 141.2 | 217.1 | 40.4 |
|
||||
| 24 GHz | 17 | 57.3 | 129.8 | 82.0 |
|
||||
| 47 GHz | 14 | 31.8 | 91.3 | 101.1 |
|
||||
|
||||
So 10 → 24 → 47 GHz amplitude does double at each step (40 → 82 →
|
||||
101 %), but it never reaches the 360 % figure quoted from the small
|
||||
75 GHz subsample. The frequency-scaling direction in the time-of-day
|
||||
weights stands; the *magnitude* projection toward 75 GHz+ is unsupported
|
||||
on present data and will be revisited if/when the 75/122 GHz corpus
|
||||
breaks past n=200 with hour coverage.
|
||||
|
||||
> **Selection-bias warning (added 2026-04-25).** Diurnal amplitude
|
||||
> derived from QSO timestamps reflects *both* propagation diurnal cycles
|
||||
> *and* operator-scheduling diurnal cycles. At VHF/UHF the second term
|
||||
> dominates: the same robust amplitude table run for 222/432 MHz lands at
|
||||
> 129 % / 148 %, well above any microwave band — physics doesn't predict
|
||||
> that, contest scheduling does (VHF contests run evenings/weekends,
|
||||
> microwave contests run mornings as rovers chase grids). Time-of-day
|
||||
> weights for 222/432 MHz must therefore *not* be fitted to the QSO
|
||||
> diurnal curve; they keep the global default until a clean atmospheric
|
||||
> signal exists (e.g. continuous beacon monitoring).
|
||||
|
||||
### Finding 9: Ducting Peaks June-July, Not August
|
||||
|
||||
Monthly ducting probability from 3,901 soundings:
|
||||
Monthly ducting probability from the 27,058-sounding corpus
|
||||
(2026-04-25 refresh):
|
||||
|
||||
| Month | Soundings | Ducting % | Avg N | Avg Min Gradient | Avg PWAT (mm) |
|
||||
|-------|-----------|-----------|-------|------------------|---------------|
|
||||
| Jan | 32 | 21.9% | 306.2 | -133.4 | 9.5 |
|
||||
| Feb | 58 | 17.2% | 304.4 | -132.7 | 6.6 |
|
||||
| Mar | 37 | **10.8%** | 293.3 | -112.5 | 6.6 |
|
||||
| Apr | 64 | 37.5% | 306.0 | -152.0 | 14.0 |
|
||||
| May | 68 | 48.5% | 323.3 | -233.1 | 20.5 |
|
||||
| Jun | 134 | **68.7%** | 335.0 | -322.7 | 28.6 |
|
||||
| Jul | 85 | **76.5%** | 339.3 | -301.4 | 27.6 |
|
||||
| Aug | 1,700 | 53.9% | 341.9 | -260.7 | 33.3 |
|
||||
| Sep | 1,564 | 56.4% | 332.4 | -286.5 | 26.2 |
|
||||
| Oct | 48 | 60.4% | 321.5 | -314.3 | 17.9 |
|
||||
| Nov | 53 | 56.6% | 308.4 | -264.3 | 10.6 |
|
||||
| Dec | 58 | 12.1% | 305.0 | -139.8 | 10.5 |
|
||||
| Month | Soundings | Ducting % | Avg dN/dh | Avg PWAT (mm) |
|
||||
|-------|----------:|----------:|----------:|--------------:|
|
||||
| Jan | 95 | 28.4 % | −171 | 10.9 |
|
||||
| Feb | 131 | 29.0 % | −163 | 7.9 |
|
||||
| Mar | 81 | **17.3 %** | −148 | 9.4 |
|
||||
| Apr | 200 | 25.5 % | −161 | 12.7 |
|
||||
| May | 311 | 51.8 % | −286 | 25.6 |
|
||||
| Jun | 380 | **69.5 %** | −356 | 32.0 |
|
||||
| Jul | 302 | 67.5 % | −294 | 29.7 |
|
||||
| Aug | 5,013 | 55.2 % | −255 | 35.1 |
|
||||
| Sep | 2,343 | 56.3 % | −280 | 27.2 |
|
||||
| Oct | 155 | 57.4 % | −307 | 19.3 |
|
||||
| Nov | 158 | 50.0 % | −257 | 12.0 |
|
||||
| Dec | 140 | 25.7 % | −185 | 10.8 |
|
||||
|
||||
**March is the worst month** (10.8%), not winter. December-February averages ~17%. The sharp ramp April→July is nonlinear. Contest data (Aug-Sep) slightly undersamples the true ducting peak (Jun-Jul at 69-77%). Note: Aug/Sep have disproportionate sounding counts because data is enriched from QSO time windows during contest months.
|
||||
**March is still the worst month** (17.3 %, up from 10.8 % on the 3.9k
|
||||
corpus) and **June–July still the peak** (67–70 %). With 6.9× more
|
||||
soundings the shoulder-month numbers stabilised but the seasonal shape
|
||||
is unchanged. Aug/Sep dominate the row counts because the sounding
|
||||
backfill is QSO-driven during contest months; that's a sampling
|
||||
artefact, not a meteorological one.
|
||||
|
||||
### Finding 10: Mode Matters — CW Advantage Scales with Frequency
|
||||
|
||||
|
|
@ -706,21 +780,34 @@ The continuous-vs-binary signal is sharp: ducting soundings have avg gradient **
|
|||
|
||||
The full correlation matrix across every band with ≥200 matched contacts is in Part 2d. Three signals are worth flagging at the physics level:
|
||||
|
||||
- **HPBL is negative at every band tested** (r −0.20 to −0.38). Shallower BL → longer paths. Folded into `Scorer.score_refractivity/4` as a multiplier on the gradient score: 1.10× at <200 m, 1.00× at 200-1000 m, 0.92× at 1000-2000 m, 0.78× at ≥2000 m.
|
||||
|
||||
| HPBL bin (m) | avg km | p50 km |
|
||||
|---|---|---|
|
||||
| <200 | 229.8 | 176.0 |
|
||||
| 200–500 | 186.2 | 154.5 |
|
||||
| 500–1000 | 158.9 | 126.0 |
|
||||
| 1000–1500 | 162.0 | 136.0 |
|
||||
| 1500–2000 | 136.5 | 124.5 |
|
||||
| ≥2000 | 100.3 | 90.0 |
|
||||
- **HPBL — retired (no usable signal).** The earlier "−0.20 to −0.38"
|
||||
HPBL correlations were small-corpus artefacts; the n=47,418 10 GHz
|
||||
match shows rho_hpbl = +0.004 and the bin tables are flat to within
|
||||
±5 % across 200–2,000 m. Multiplier removed from the scorer 2026-04-25.
|
||||
See Finding 4 for details.
|
||||
|
||||
- **Surface refractivity is consistently positive** (+0.08 to +0.18 across VHF/UHF bands). Higher N bends rays further under the same gradient; independent of dN/dh. Contributes to the `refractivity` factor via the additive surface-N term in `score_refractivity/4`.
|
||||
|
||||
- **Pressure is negative at every band** (r −0.04 at 902 MHz up to −0.42 at 47 GHz in magnitude). Low pressure brings the frontal boundaries and moisture gradients that build ducts — `Scorer.score_pressure/2` scores <980 mb highest and >1020 mb lowest.
|
||||
|
||||
**2026-04-25 caveat — bimodality at 10 GHz.** The pressure-bin
|
||||
distribution at 10 GHz is U-shaped, not monotonic:
|
||||
|
||||
| Pressure bin (mb) | n | avg km | p50 km |
|
||||
|---|---:|---:|---:|
|
||||
| < 990 | 23,536 | 219.4 | 206.0 |
|
||||
| 990–1000 | 12,221 | 182.7 | 159.6 |
|
||||
| 1000–1010 | 6,246 | 204.5 | 170.6 |
|
||||
| 1010–1020 | 3,996 | 214.7 | 173.6 |
|
||||
| ≥ 1020 | 1,419 | **235.7** | **208.9** |
|
||||
|
||||
Two physically distinct regimes are stacked into one signal: low
|
||||
pressure → frontal lift / advection ducts; high pressure → subsidence
|
||||
inversion ducts. The current linear scoring captures the < 990 mb
|
||||
effect but under-weights the ≥ 1020 mb ridge. Listed as a next-
|
||||
iteration target — current weights stay linear pending a piecewise
|
||||
rewrite of `Scorer.score_pressure/2`.
|
||||
|
||||
### Sub-mm band coverage
|
||||
|
||||
The contact corpus contains sub-mm contacts at 142, 145, 241, 288, 322, 403, and 411 GHz. `BandConfig` has entries for all of them; ITU-R P.676/838 coefficients are interpolated from the 134/241 GHz entries, and ranges are scaled from observed contact distances.
|
||||
|
|
@ -741,11 +828,32 @@ R (mm/hr) = (Z / 200)^(1/1.6) where Z = 10^(dBZ/10)
|
|||
|
||||
with a 5 dBZ noise floor (ground clutter / clear air) and a 150 mm/hr ceiling (hail contamination). Only active for `forecast_hour == 0` — the worker skips NEXRAD merge on f01+ because we have no future radar image.
|
||||
|
||||
### Native-profile duct boost
|
||||
### Native-profile duct boost — retired 2026-04-25
|
||||
|
||||
The base refractivity score uses HRRR's 13 pressure levels, which systematically under-read thin trapping layers (see Part 2c — only 0.12% of native-resolution cells support ≥15 GHz, but the pressure-level data looks the same above and below that threshold). When `hrrr_native_profiles.best_duct_band_ghz` is present and its value is ≥ the target band's frequency, `Scorer.score_refractivity/4` multiplies the base score by 1.15×. A duct that only supports sub-band frequencies does *not* boost — it's evidence that the gradient we have is all there is at the target band.
|
||||
The 1.15× boost on `Scorer.score_refractivity/{4,5}` for cells where
|
||||
`hrrr_native_profiles.best_duct_band_ghz` ≥ target frequency was
|
||||
removed on 2026-04-25 after the n=56,837 native-profile join falsified
|
||||
its premise. At 10 GHz, contacts where the native duct supports the
|
||||
band ran *shorter* than no-duct contacts:
|
||||
|
||||
This composes with the HPBL multiplier: a thin shallow-BL cell with a 24-GHz-supporting native duct gets the full 1.05–1.10× HPBL bonus *and* the 1.15× native-duct bonus, clamped at 100. The boost only applies when the cell's `bulk_richardson` is in the stable regime (< 25) — a low duct-band reading under turbulent conditions (high Richardson) is likely a duct that would be broken up by mechanical mixing.
|
||||
| Band | n_total | no duct (km) | duct supports band (km) | duct below band (km) |
|
||||
|---|---:|---:|---:|---:|
|
||||
| 10 GHz | 52,341 | **211.3** (n=44 658) | 197.9 (n=173) | 199.8 (n=7,510) |
|
||||
| 24 GHz | 3,700 | 94.8 (n=3,182) | 131.5 (n=6) | 98.3 (n=512) |
|
||||
| 47 GHz | 689 | 64.3 (n=552) | – (n=0) | 58.7 (n=137) |
|
||||
|
||||
The 10 GHz cell with 173 supports-band samples is large enough to
|
||||
say the boost was not just absent — it was *opposite* the truth in
|
||||
our matched corpus. At 24/47 GHz the supports-band cell is too small
|
||||
to draw any conclusion either way. The Bulk Richardson gate that
|
||||
existed only to suppress this boost is also retired; the function
|
||||
arity stays put so existing call sites compile unchanged. Refractivity
|
||||
scoring now relies on `min_refractivity_gradient` alone.
|
||||
|
||||
`hrrr_native_profiles` continues to be ingested — the schema, worker,
|
||||
and the per-cell duct-band column are useful diagnostically and may
|
||||
yet earn back a scoring role under a different statistical model. We
|
||||
just don't multiply the refractivity score by it.
|
||||
|
||||
### Commercial-link inverse sensor
|
||||
|
||||
|
|
@ -763,6 +871,59 @@ Seven af11x / af60 commercial microwave links around Princeton TX (33.2°N, 96.5
|
|||
|
||||
This is the first *measured* signal in the algorithm — every other factor is a model-derived proxy. It only helps a ~150 km radius around DFW, but in that zone it's the strongest single indicator we have of actual refractivity anomalies happening right now. Out-of-zone cells see no change.
|
||||
|
||||
**2026-04-25 calibration update.** `commercial_samples` now has 38,443
|
||||
rows over 22 days (2026-03-30 → 2026-04-20) across the seven links.
|
||||
Per-link rx_power_0 standard deviation is **1.0–1.7 dB** on the af11x
|
||||
links and **2.5–2.9 dB** on the af60 links — the 60 GHz pair is
|
||||
intermittent (only 150–200 up-samples each in the window) which is
|
||||
itself the strongest signal for "60 GHz weather sensitivity" the
|
||||
project has ever recorded.
|
||||
|
||||
QSO overlap with the sample window was zero (DFW microwave activity
|
||||
is contest-driven, Aug/Sep), so direct contact-distance vs fade
|
||||
correlation is still pending. But against the 37-hour HRRR overlap
|
||||
that *does* exist in the window, hourly mean rx-degradation correlates
|
||||
with HRRR fields at magnitudes 4–5× anything the QSO-vs-HRRR table
|
||||
produces at 24 GHz:
|
||||
|
||||
| corr(fade_db, …) | value |
|
||||
|---|---:|
|
||||
| min_refractivity_gradient | −0.161 |
|
||||
| surface_dewpoint_c | **+0.487** |
|
||||
| pwat_mm | **+0.609** |
|
||||
| t-td depression | +0.147 |
|
||||
| surface_pressure_mb | **−0.605** |
|
||||
| HPBL | +0.033 |
|
||||
|
||||
Direction matches the QSO-derived 24 GHz weights exactly: wet column
|
||||
→ more fade, low pressure → more fade. The sensor is wired and
|
||||
producing usable signal; QSO co-occurrence is the missing piece, not
|
||||
the sensor design.
|
||||
|
||||
**Morning fade window.** Binning the 27,820 af11x up-samples by local
|
||||
hour, the rate of fades ≥ 3 dB below per-link mean is two-peaked:
|
||||
|
||||
| Local hour | n | n_fade≥3dB | pct |
|
||||
|---|---:|---:|---:|
|
||||
| 03 | 1,139 | 7 | 0.61 |
|
||||
| 04 | 1,135 | 16 | **1.41** |
|
||||
| 05 | 1,135 | 18 | **1.59** |
|
||||
| 06 | 1,135 | 7 | 0.62 |
|
||||
| 09 | 1,130 | 13 | 1.15 |
|
||||
| 10 | 1,134 | 25 | **2.20** |
|
||||
| 11 | 1,140 | 16 | 1.40 |
|
||||
| 12 | 1,117 | 19 | 1.70 |
|
||||
| 13–22 | flat | 3–8 | 0.18–0.65 |
|
||||
| 00–02 | quiet | 0–2 | 0.00–0.18 |
|
||||
|
||||
Two physical mechanisms: 04–05 local is the breakdown of overnight
|
||||
radiation-fog / nocturnal-inversion ducting; 10–12 local is the onset
|
||||
of boundary-layer mixing. This is the qualitative "morning fade
|
||||
window" 11 GHz operators have always experienced — quantified for the
|
||||
first time. Suggests the diurnal time-of-day curve at 10–24 GHz
|
||||
should be biased toward morning hours rather than the symmetric
|
||||
"night peak" that sub-mm-band physics alone would suggest.
|
||||
|
||||
---
|
||||
|
||||
## Part 2d: Full-Corpus Per-Band Recalibration
|
||||
|
|
@ -876,17 +1037,20 @@ Monthly ducting probability from the expanded sounding corpus. Compared to the P
|
|||
### What stayed, what left
|
||||
|
||||
- **Humidity direction flip at ~15 GHz** — preserved. Data shows it clearly (signs flip going from 5.76 GHz to 10 GHz and again from 10 GHz to 24 GHz).
|
||||
- **Pressure is the strongest 10 GHz correlator** — preserved, but magnitude is now −0.066 (vs −0.180 reported in Part 2b); the newer matched corpus has cleaner spatial joins and less contest-seasonality bias.
|
||||
- **HPBL is negative at every band we can fit** — preserved. Already folded into refractivity as a multiplier.
|
||||
- **Pressure is the strongest 10 GHz correlator** — preserved, but magnitude is now −0.066 (vs −0.180 reported in Part 2b); the newer matched corpus has cleaner spatial joins and less contest-seasonality bias. The bin distribution at 10 GHz is U-shaped — see Part 2c "Pressure" for the bimodality and the next-iteration target.
|
||||
- **HPBL boundary-layer multiplier — retired 2026-04-25.** rho_hpbl ≈ 0 at every band ≥ 222 MHz on the n=68k matched corpus; the previously reported "shallow-BL → longer-distance" effect was a small-corpus artefact. Multiplier removed from `Scorer.score_refractivity/{3,4,5}` and the Rust port; the column stays in the schema and diagnostics.
|
||||
- **Native-profile 1.15× duct boost — retired 2026-04-25.** At 10 GHz on n=52,341 matched contacts, cells where the native duct supports the band ran *shorter* than no-duct cells (198 vs 211 km, n=173 vs 44,658). Removed from both the Elixir scorer and the Rust port; arity preserved so callers don't need a coordinated rewrite. `hrrr_native_profiles` continues to be ingested for diagnostics.
|
||||
- **Per-band recalibration is statistically defensible** — the Part 2c moratorium lifts. 9 bands have n ≥ 200 matched contacts.
|
||||
- **NARR historical calibration — deferred.** 103 pre-2014 matches at 10 GHz isn't enough to validate the weights against the earlier atmosphere; the NARR backfill queue is still draining. Re-run when `narr_profiles` crosses ~10K rows.
|
||||
- **Commercial-link sensor calibration — deferred.** Zero DFW-zone contact overlap with the 20-day commercial_samples window. Re-run after the next contest season.
|
||||
- **Commercial-link inverse sensor — promoted from deferred.** 22-day, 38k-sample SNMP corpus now exists. 37-hour HRRR overlap shows rho(fade, PWAT) = +0.61, rho(fade, P) = −0.61. Sensor is detecting the right physics; QSO co-occurrence is the missing piece, not the sensor design. See Part 2c "Commercial-link inverse sensor."
|
||||
- **NARR historical calibration — still deferred.** 103 pre-2014 matches at 10 GHz isn't enough to validate the weights against the earlier atmosphere; the NARR backfill is still at 358 rows. Re-run when `narr_profiles` crosses ~10K rows.
|
||||
|
||||
### Open items
|
||||
|
||||
1. **50 MHz and 144 MHz are unmodeled (0 contacts in corpus).** Both bands carry default weights but ranges and seasonal tables are pure physics priors. Any VHF calibration needs an import of 6 m and 2 m contacts from external logs before we can say anything.
|
||||
2. **47+ GHz inherit defaults.** 47 GHz has n=53, below the 200-contact floor; 75 GHz has n=106 in the DB but only a handful HRRR-match. Re-fit when contest-season 47/75 GHz logs accumulate.
|
||||
3. **Refractivity gradient signal is load-bearing only at 24 GHz.** At every other band, the gradient correlation rides the noise floor. Moving to native-resolution HRRR (`hrrr_native_profiles.best_duct_band_ghz`) is how this gets better; the data in this table shows 10–15× cleaner gradients when we have a duct-supporting cell.
|
||||
3. **Refractivity gradient signal is load-bearing only at 24 GHz.** At every other band the gradient correlation rides the noise floor. Per-band gradient weight set to 0 outside [10 GHz, 47 GHz] (Recommendation 7 in `docs/algo-reports/2026-04-25-algo-revisions.md`). This will be reflected in the band-weight matrix on the next `derive_band_weights.py` run.
|
||||
4. **Pressure scoring is linear, distance is U-shaped at 10 GHz.** The < 990 mb tail and the ≥ 1020 mb tail both run longer than the 990–1010 mb middle. `Scorer.score_pressure/2` should switch from linear to a piecewise score with a peak near the 990–1010 mb minimum and rising to both tails.
|
||||
5. **VHF/UHF time-of-day weights are contest-schedule-contaminated.** Running the diurnal-amplitude fit on 222/432 MHz lands at 129 % / 148 %, which physics doesn't predict; that's evening-contest scheduling, not the atmosphere. VHF/UHF time-of-day weights stay at the global default until a clean atmospheric signal exists.
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
346
docs/algo-reports/2026-04-25-algo-revisions.md
Normal file
346
docs/algo-reports/2026-04-25-algo-revisions.md
Normal file
|
|
@ -0,0 +1,346 @@
|
|||
# Proposed `algo.md` Revisions — 2026-04-25
|
||||
|
||||
> Scope: re-reads the freshest contact corpus (`contacts` n=81,994, HRRR-matched
|
||||
> n=68,062) and the new commercial_samples corpus (n=38,443 over 22 days,
|
||||
> 2026-03-30 → 2026-04-20) and turns the deltas into concrete `algo.md` edits.
|
||||
> Companion to the auto-generated `2026-04-25-recalibration.md`.
|
||||
>
|
||||
> Local dev DB (`prop_dev` mirrored from prod 10.0.15.24). Queries below
|
||||
> can be re-run verbatim against either.
|
||||
|
||||
## Summary of recommended `algo.md` edits
|
||||
|
||||
| # | Section | Change | Confidence |
|
||||
|---|---|---|---|
|
||||
| 1 | Part 2c "Native-profile duct boost" | **Drop the 1.15× boost.** Data shows duct-supports-band contacts run *shorter* than no-duct contacts at 10 GHz. | High |
|
||||
| 2 | Finding 4 "HPBL — Time-Dependent Sweet Spot" | **Demote to "no usable signal."** rho_hpbl ≈ 0 at every band ≥222 MHz; the bin tables are within sampling noise. | High |
|
||||
| 3 | Part 2b "Pressure" + Part 2d weights row 10 GHz | **Note the U-shape.** Distance is longest at *both* tails (<990 mb and ≥1020 mb) at 10 GHz. Linear scoring underweights the high-pressure ridge. | Medium |
|
||||
| 4 | Part 2c "Commercial-link inverse sensor" | **Promote from deferred → calibrated.** 22-day SNMP corpus exists; 37 HRRR-overlap hours show rho(fade, PWAT) = +0.61, rho(fade, P) = −0.61. The sensor is detecting the right physics; QSO overlap is the missing piece, not the sensor. | High |
|
||||
| 5 | Finding 8 "Time-of-Day Scales with Frequency" | **Revise magnitudes.** Diurnal amplitude (median IQR / band median, hours with ≥30 samples) is 40 % at 10 GHz, 82 % at 24 GHz, 101 % at 47 GHz — not the 4 % / 28 % / 36 % currently quoted. | Medium |
|
||||
| 6 | Finding 8 / Part 2b | **Add contest-schedule caveat.** 222/432 MHz diurnal amplitude is 129–148 %; that signal is operator behavior (evening contests), not propagation. Document explicitly so future weight-fitters don't propagate it. | High |
|
||||
| 7 | Finding 5 "Use Continuous Gradient" | **Affirm and tighten.** Gradient correlation only clears noise at 24 GHz (rho ≈ 0.017 — still tiny) and 75 GHz (0.474, n=83). At every other band it's <0.1. Recommend dropping the gradient term from VHF/UHF weight vectors entirely. | Medium |
|
||||
| 8 | Part 2c "Sounding ducting probability" | **Refresh table.** Numbers in the current `algo.md` Part 2c table differ from the latest sounding pull (e.g. April 25.5 % vs 31.3 %). Use the values from `2026-04-25-recalibration.md`. | Low (mechanical) |
|
||||
| 9 | Finding 11 / Part 2d "Open items" | **Add the diurnal-fade pattern of commercial links.** 11 GHz fade ≥3 dB peaks 04 :00–12 :00 local (1.4–2.2 % of samples) vs 0.2–0.7 % afternoon/evening — consistent with morning radiation-fog ducting + boundary-layer mixing onset. | Medium |
|
||||
|
||||
---
|
||||
|
||||
## Evidence per recommendation
|
||||
|
||||
### 1. Native-duct boost is wrong-signed at 10 GHz
|
||||
|
||||
`Scorer.score_refractivity/4` multiplies the refractivity component by 1.15 when
|
||||
`hrrr_native_profiles.best_duct_band_ghz` ≥ contact band. The validation
|
||||
matrix:
|
||||
|
||||
```sql
|
||||
WITH joined AS (
|
||||
SELECT DISTINCT ON (c.id)
|
||||
c.id, c.band::int AS band, c.distance_km::float AS dist,
|
||||
n.best_duct_band_ghz AS duct_ghz
|
||||
FROM contacts c
|
||||
JOIN hrrr_native_profiles n
|
||||
ON n.lat BETWEEN (c.pos1->>'lat')::float - 0.07 AND (c.pos1->>'lat')::float + 0.07
|
||||
AND n.lon BETWEEN (c.pos1->>'lon')::float - 0.07 AND (c.pos1->>'lon')::float + 0.07
|
||||
AND n.valid_time BETWEEN c.qso_timestamp - INTERVAL '1 hour' AND c.qso_timestamp + INTERVAL '1 hour'
|
||||
WHERE c.pos1 IS NOT NULL AND c.distance_km < 3000 AND c.flagged_invalid = false
|
||||
ORDER BY c.id, ABS(EXTRACT(EPOCH FROM n.valid_time - c.qso_timestamp))
|
||||
)
|
||||
SELECT band,
|
||||
AVG(dist) FILTER (WHERE duct_ghz IS NULL) AS no_duct,
|
||||
AVG(dist) FILTER (WHERE duct_ghz IS NOT NULL AND duct_ghz*1000 >= band) AS supports,
|
||||
AVG(dist) FILTER (WHERE duct_ghz IS NOT NULL AND duct_ghz*1000 < band) AS below
|
||||
FROM joined GROUP BY band HAVING count(*) >= 100;
|
||||
```
|
||||
|
||||
| Band | n_total | no duct (km) | duct supports band (km) | duct below band (km) |
|
||||
|---|---:|---:|---:|---:|
|
||||
| 10 GHz | 52,341 | **211.3** (n=44 658) | 197.9 (n=173) | 199.8 (n=7,510) |
|
||||
| 24 GHz | 3,700 | 94.8 (n=3,182) | 131.5 (n=6) | 98.3 (n=512) |
|
||||
| 47 GHz | 689 | 64.3 (n=552) | – (n=0) | 58.7 (n=137) |
|
||||
|
||||
The 10 GHz row has 173 supports-band samples — small but big enough to say the
|
||||
1.15× boost is not just absent, it's *opposite* the truth in our matched
|
||||
corpus. At 24 GHz the supports-band cell is too small (n=6) to draw any
|
||||
conclusion. At 47 GHz it's empty.
|
||||
|
||||
**Edit:** in Part 2c "Native-profile duct boost" replace the last paragraph
|
||||
with: *"Validation against the 56,837-contact native-profile join shows
|
||||
duct-supports-band contacts run* slightly shorter *than no-duct contacts at
|
||||
10 GHz (198 vs 211 km, n=173 vs 44,658). The 1.15× multiplier in
|
||||
`Scorer.score_refractivity/4` is unsupported and removed; refractivity
|
||||
scoring now relies on the continuous `min_refractivity_gradient` term only."*
|
||||
|
||||
### 2. HPBL has no signal at any matched band
|
||||
|
||||
From `2026-04-25-recalibration.md` correlations (rho_hpbl column):
|
||||
|
||||
| Band | n | rho_hpbl |
|
||||
|---|---:|---:|
|
||||
| 222 MHz | 5,578 | −0.031 |
|
||||
| 432 MHz | 6,782 | −0.063 |
|
||||
| 902 MHz | 1,168 | −0.045 |
|
||||
| 1.296 GHz | 2,111 | −0.064 |
|
||||
| 2.304 GHz | 405 | −0.085 |
|
||||
| 10 GHz | 47,418 | **+0.004** |
|
||||
| 24 GHz | 3,429 | −0.031 |
|
||||
| 47 GHz | 679 | −0.092 |
|
||||
|
||||
Magnitude never exceeds 0.092; at 10 GHz where n is largest it's three orders
|
||||
of magnitude below "noise floor" (0.05 in the existing `s_band,factor`
|
||||
formula). The bin tables in the recalibration report tell the same story —
|
||||
all six HPBL bins agree within ±5 % at every band where n ≥ 200.
|
||||
|
||||
**Edit:** Finding 4 currently leans on the time-dependent sweet-spot
|
||||
hypothesis ("HPBL 200–800 m daytime ↔ longer contacts"). Replace with: *"On
|
||||
the n=47,418 10 GHz matched corpus, distance is statistically independent of
|
||||
HPBL across 200–2,000 m. The previously reported sweet-spot is below the
|
||||
joint sampling noise of HRRR's own HPBL field (~150 m RMS) and matched-pair
|
||||
spatial tolerance. HPBL retained in the schema for diagnostics; weight set
|
||||
to 0 in band weights."*
|
||||
|
||||
### 3. 10 GHz pressure is U-shaped, not monotonic
|
||||
|
||||
From the recalibration pressure-bin table at 10 GHz:
|
||||
|
||||
| Pressure bin | n | avg_km | p50_km |
|
||||
|---|---:|---:|---:|
|
||||
| < 990 mb | 23,536 | 219.4 | 206.0 |
|
||||
| 990–1000 | 12,221 | 182.7 | 159.6 |
|
||||
| 1000–1010 | 6,246 | 204.5 | 170.6 |
|
||||
| 1010–1020 | 3,996 | 214.7 | 173.6 |
|
||||
| ≥ 1020 | 1,419 | **235.7** | **208.9** |
|
||||
|
||||
The minimum is 990–1000 mb, *both* tails are longer. The current scorer
|
||||
treats pressure linearly (lower = better), which captures the <990 effect but
|
||||
under-weights the ≥1020 ridge. Two physically distinct regimes are stacked
|
||||
into one signal: low-pressure → frontal lift / advection ducts;
|
||||
high-pressure → subsidence inversion ducts.
|
||||
|
||||
**Edit:** Part 2b "Finding 3" — note the bimodality. Recommend in Part 2d
|
||||
that pressure scoring switch from linear to a piecewise score with a peak at
|
||||
the 990–1000 mb minimum and rising toward both tails. (Keep the current
|
||||
linear weight in code for now; flag as next-iteration target.)
|
||||
|
||||
### 4. Commercial-link sensor: lift the deferral
|
||||
|
||||
`commercial_samples` now has 38,443 rows over 22 days (2026-03-30 → 04-20)
|
||||
across 7 links — 5× af11x at 11 GHz, 2× af60 at 60 GHz. Per-link baseline:
|
||||
|
||||
| Link | radio | n_up | avg_rx0 (dBm) | sd_rx0 | n_fade≥3dB | n_fade≥8dB |
|
||||
|---|---|---:|---:|---:|---:|---:|
|
||||
| climax-to-core | af11x | 5,488 | −63.86 | 1.06 | 7 | 5 |
|
||||
| core-new-hope | af11x | 5,491 | −50.83 | 1.65 | 62 | 2 |
|
||||
| core-to-climax | af11x | 5,491 | −61.10 | 0.91 | 7 | 5 |
|
||||
| new-hope-to-core | af11x | 5,493 | −51.50 | 1.36 | 33 | 3 |
|
||||
| verona-to-climax | af11x | 5,501 | −41.97 | 1.04 | 80 | 12 |
|
||||
| 380_982_60LR | af60 | 150 | −52.49 | 2.93 | 8 | 4 |
|
||||
| 982_380_60LR | af60 | 206 | −52.14 | 2.48 | 8 | 3 |
|
||||
|
||||
The 60 GHz pair is intermittent (only 150–200 up-samples each — rain knocks
|
||||
them down hard) which is itself the strongest signal for "60 GHz weather
|
||||
sensitivity" we've ever had.
|
||||
|
||||
QSO overlap: zero contacts in the DFW box during the 22-day window. **But**
|
||||
the sensor is correlating with HRRR cleanly. Hourly mean degradation
|
||||
vs nearest-grid HRRR profile (n=37 hours of overlap — limited because the
|
||||
local `hrrr_profiles` mirror only has 3 days in the window):
|
||||
|
||||
| corr(fade_db, …) | value |
|
||||
|---|---:|
|
||||
| min_refractivity_gradient | −0.161 |
|
||||
| surface_dewpoint_c | **+0.487** |
|
||||
| pwat_mm | **+0.609** |
|
||||
| t-td depression | +0.147 |
|
||||
| surface_pressure_mb | **−0.605** |
|
||||
| HPBL | +0.033 |
|
||||
|
||||
Magnitudes are 4–5× anything the contact-vs-HRRR table produces at 24 GHz
|
||||
because the sensor is sampled every 30 s instead of once per QSO.
|
||||
|
||||
**Edit:** in Part 2c "Commercial-link inverse sensor" replace "deferred until
|
||||
contest season" with: *"The 7-link cluster at 33.2N/−96.5W now has 22 days
|
||||
of 30-second polling. Per-link rx_power_0 SD is 1.0–1.7 dB on af11x and
|
||||
2.5–2.9 dB on af60. Hourly mean rx-degradation correlates with HRRR PWAT at
|
||||
+0.61 and surface pressure at −0.61 — direction matches the QSO-derived
|
||||
24 GHz weights. The sensor is wired and producing usable signal; QSO overlap
|
||||
is gated on contest-season activity in the DFW zone."*
|
||||
|
||||
### 5. Diurnal amplitude scales with frequency, but the magnitudes in algo.md are off
|
||||
|
||||
Restricting to local hours with ≥30 contacts per band (so 0300-local doesn't
|
||||
get a single 79 km outlier) produces a defensible amplitude index:
|
||||
|
||||
```sql
|
||||
WITH local_q AS (
|
||||
SELECT band::int AS band, distance_km::float AS dist,
|
||||
EXTRACT(HOUR FROM qso_timestamp AT TIME ZONE 'America/Chicago')::int AS local_hr
|
||||
FROM contacts
|
||||
WHERE pos1 IS NOT NULL AND distance_km < 3000 AND flagged_invalid = false
|
||||
AND band >= 222 AND qso_timestamp >= '2014-10-02'
|
||||
),
|
||||
hourly AS (
|
||||
SELECT band, local_hr, count(*) AS n,
|
||||
PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY dist) AS med
|
||||
FROM local_q GROUP BY band, local_hr HAVING count(*) >= 30
|
||||
)
|
||||
SELECT band, count(*) hours,
|
||||
MIN(med) lo, MAX(med) hi,
|
||||
100.0 * (MAX(med) - MIN(med)) / AVG(med) AS pct_amp
|
||||
FROM hourly GROUP BY band;
|
||||
```
|
||||
|
||||
| Band | hours | lo p50 (km) | hi p50 (km) | amplitude % |
|
||||
|---|---:|---:|---:|---:|
|
||||
| 222 MHz | 19 | 112.0 | 379.5 | **129.2** |
|
||||
| 432 MHz | 19 | 95.0 | 355.0 | **148.4** |
|
||||
| 902 MHz | 15 | 94.5 | 178.0 | 57.7 |
|
||||
| 1.296 GHz | 17 | 80.0 | 176.0 | 71.2 |
|
||||
| 2.304 GHz | 13 | 108.0 | 171.5 | 48.2 |
|
||||
| 10 GHz | 23 | 141.2 | 217.1 | 40.4 |
|
||||
| 24 GHz | 17 | 57.3 | 129.8 | 82.0 |
|
||||
| 47 GHz | 14 | 31.8 | 91.3 | 101.1 |
|
||||
|
||||
**Microwave bands behave as advertised** (10 → 24 → 47 GHz amplitude doubles
|
||||
roughly each step). **VHF/UHF bands do not** — and this is the big gotcha:
|
||||
|
||||
### 6. VHF/UHF diurnal "signal" is contest-schedule artifact
|
||||
|
||||
The 222 MHz and 432 MHz amplitudes (129 %, 148 %) blow past every microwave
|
||||
band. Physics doesn't predict this — at VHF the optical-path-loss and
|
||||
diffraction terms are not strongly time-of-day-dependent. What is happening
|
||||
is operator behavior: VHF contests run primarily evenings and weekends,
|
||||
microwave contests run primarily mornings (rovers chase grids with
|
||||
predictable schedules). The QSO database conflates "what propagation looks
|
||||
like at 21 :00 local" with "what 21 :00-local QSOs look like, which is
|
||||
overwhelmingly evening contest exchanges."
|
||||
|
||||
**Edit:** Add a new caveat box to Finding 8:
|
||||
|
||||
> **Selection bias warning.** Diurnal amplitude derived from QSO timestamps
|
||||
> reflects *both* propagation diurnal cycles *and* operator-scheduling
|
||||
> diurnal cycles. At VHF/UHF the second term dominates: the 130–150 %
|
||||
> amplitude at 222/432 MHz is contest scheduling, not the atmosphere. Time-
|
||||
> of-day weights for these bands should be **smaller** than what the QSO
|
||||
> diurnal fit suggests, not larger. Until a clean atmospheric-only signal
|
||||
> exists (e.g. continuous beacon monitoring, see project plan
|
||||
> `beacon_monitoring`), keep VHF/UHF time-of-day weights at the global
|
||||
> default, not at a per-band-fit value.
|
||||
|
||||
### 7. Refractivity gradient is below noise at most bands
|
||||
|
||||
Reproduced from `2026-04-25-recalibration.md`:
|
||||
|
||||
| Band | rho_grad |
|
||||
|---|---:|
|
||||
| 222 MHz | +0.031 |
|
||||
| 432 MHz | +0.003 |
|
||||
| 902 MHz | −0.062 |
|
||||
| 1.296 GHz | −0.079 |
|
||||
| 2.304 GHz | −0.178 |
|
||||
| 3.4 GHz | 0.000 |
|
||||
| 5.76 GHz | −0.110 |
|
||||
| 10 GHz | +0.027 |
|
||||
| 24 GHz | +0.017 |
|
||||
| 47 GHz | −0.008 |
|
||||
| 75 GHz | +0.474 (n=83) |
|
||||
|
||||
The 75 GHz row is n=83, dominated by Aug/Sep contest weeks where the gradient
|
||||
just happens to track the same boundary-layer evolution that drives 75 GHz
|
||||
contacts. It is not strong evidence for a per-band gradient term.
|
||||
|
||||
**Edit:** Finding 5 already says "use continuous gradient." Tighten to: *"…
|
||||
use continuous gradient* at 24 GHz only*; below 10 GHz the gradient signal
|
||||
is at or below the 0.05 noise floor and adds nothing the dewpoint and PWAT
|
||||
terms don't already capture. Set band-specific gradient weight to 0 outside
|
||||
[10 GHz, 47 GHz]."*
|
||||
|
||||
### 8. Mechanical updates
|
||||
|
||||
The Part 2c sounding-ducting table was last refreshed at n=6,757. The
|
||||
2026-04-25 recalibration ran on n=27,058. April ducting % is 25.5 %, not
|
||||
31.3 % as currently shown. April PWAT is 12.7 mm, not 13.5 mm. Replace
|
||||
verbatim from the recalibration report.
|
||||
|
||||
### 9. Diurnal fade pattern of 11 GHz commercial links
|
||||
|
||||
Bonus observation from the same 22-day SNMP corpus (af11x links, 27,820
|
||||
up-samples binned by local hour, fade ≥3 dB below per-link mean):
|
||||
|
||||
| Local hour | n | n_fade≥3dB | pct |
|
||||
|---|---:|---:|---:|
|
||||
| 00 | 1,100 | 0 | 0.00 |
|
||||
| 01 | 1,135 | 2 | 0.18 |
|
||||
| 02 | 1,140 | 0 | 0.00 |
|
||||
| 03 | 1,139 | 7 | 0.61 |
|
||||
| 04 | 1,135 | 16 | **1.41** |
|
||||
| 05 | 1,135 | 18 | **1.59** |
|
||||
| 06 | 1,135 | 7 | 0.62 |
|
||||
| 07 | 1,140 | 4 | 0.35 |
|
||||
| 08 | 1,140 | 10 | 0.88 |
|
||||
| 09 | 1,130 | 13 | 1.15 |
|
||||
| 10 | 1,134 | 25 | **2.20** |
|
||||
| 11 | 1,140 | 16 | 1.40 |
|
||||
| 12 | 1,117 | 19 | 1.70 |
|
||||
| 13–22 | flat | 3–8 | 0.18–0.65 |
|
||||
|
||||
Two-peak structure: 04–05 local (radiation-fog / nocturnal-inversion ducting
|
||||
breaks down) and 10–12 local (boundary-layer mixing onset). This is the
|
||||
"morning fade window" that 11 GHz operators have always experienced
|
||||
qualitatively; we now have a quantitative number for it.
|
||||
|
||||
**Edit:** Add a short subsection under Part 2c "Commercial-link inverse
|
||||
sensor" titled *Morning fade window* documenting the 4–12 local two-peak
|
||||
pattern, with the suggestion that the diurnal scoring curve at 10–24 GHz be
|
||||
*biased toward morning hours* rather than the symmetric "night peak" that
|
||||
sub-mm-band physics suggests.
|
||||
|
||||
---
|
||||
|
||||
## What did NOT change
|
||||
|
||||
| Finding | Status |
|
||||
|---|---|
|
||||
| F1: humidity reverses at ~15 GHz | preserved — rho signs flip 5.76 GHz → 10 GHz → 24 GHz |
|
||||
| F3: low pressure correlates with longer distances | preserved (rho 10 GHz −0.075, 24 GHz −0.20, 47 GHz −0.20) |
|
||||
| F9: ducting peaks Jun–Jul | preserved (sounding ducting % Jun 69.5, Jul 67.5, Aug 55.2) |
|
||||
| F10: CW advantage scales with freq | not re-tested in this run; corpus and methodology unchanged |
|
||||
| F11: regional performance unmodeled | unchanged |
|
||||
| 2014-10-02 NARR/HRRR cutover | unchanged; NARR pre-2014 corpus still 358 rows, deferred |
|
||||
| 50/144 MHz unmodeled | unchanged; need external imports |
|
||||
|
||||
## Carry-overs into ingestion / pipeline
|
||||
|
||||
These are not algo.md edits but worth flagging from the same query session:
|
||||
|
||||
1. **`hrrr_climatology` is empty in dev mirror** — `mix hrrr_climatology` has
|
||||
not been run (or its output is not synced). Doesn't affect scoring today
|
||||
but blocks the climatology-anomaly factor on the algo.md roadmap.
|
||||
2. **`rtma_observations` is empty** — `RtmaFetchWorker` not enabled in dev.
|
||||
Currently only relevant as fallback for contacts whose HRRR enrichment
|
||||
misses the ±1 h window; per the recalibration report HRRR matches 68,062
|
||||
of ~82,000 contacts so RTMA fallback would close ~14k of the 36k pending
|
||||
gap.
|
||||
3. **`metar_5min_observations` is empty** — schema exists but no ingestor.
|
||||
Either wire an IEM 5-minute ASOS fetcher or drop the schema.
|
||||
4. **HRRR backlog: 36,276 contacts pending enrichment, 45,718 complete
|
||||
(55.8 %).** Year breakdown via the new section in
|
||||
`scripts/recalibrate_algo.py` will identify whether this is stalled
|
||||
historical backfill or stuck live queue.
|
||||
|
||||
## Reproducibility
|
||||
|
||||
Every query in this report is self-contained SQL against `prop_dev` (which
|
||||
mirrors `10.0.15.24:5432/prop`). The schema-touch points are:
|
||||
|
||||
```
|
||||
contacts (n=81,994)
|
||||
hrrr_profiles (n=81,293,173)
|
||||
hrrr_native_profiles (n=11,472)
|
||||
soundings (n=27,058)
|
||||
nexrad_observations (n=7,286)
|
||||
narr_profiles (n=358 — known low)
|
||||
commercial_links (n=7)
|
||||
commercial_samples (n=38,443; 2026-03-30 → 2026-04-20)
|
||||
```
|
||||
|
||||
Re-running the auto-generated companion is `python3 scripts/recalibrate_algo.py`.
|
||||
|
|
@ -183,10 +183,15 @@ defmodule Microwaveprop.Propagation.Scorer do
|
|||
amplify trapping (surface inversion → steep gradient → ducting), deep
|
||||
boundary layers indicate convective mixing that disrupts ducts.
|
||||
|
||||
Empirical basis (Apr 13 2026 refresh, n=680 10 GHz contacts joined to
|
||||
HRRR profiles): 230 km avg distance for HPBL <200 m vs 100 km for HPBL
|
||||
≥2000 m — a 2.3× difference monotonic across bins. The multiplier is
|
||||
applied to *both* the threshold-matched score and the default fallback.
|
||||
As of the 2026-04-25 algo revisions the HPBL boundary-layer multiplier
|
||||
is retired — `bl_depth_m` is accepted (and the schema still stores it
|
||||
for diagnostics) but it does not modify the score. See
|
||||
`docs/algo-reports/2026-04-25-algo-revisions.md` Recommendation 2: on
|
||||
the n=47,418 10 GHz HRRR-matched corpus rho_hpbl was +0.004, well below
|
||||
the 0.05 noise floor; the bin tables agreed within ±5 % across
|
||||
200–2,000 m. The previously reported "shallow-BL → longer-distance"
|
||||
effect was a small-sample artefact and disappeared once the matched
|
||||
corpus grew past ~5,000 contacts.
|
||||
"""
|
||||
@spec score_refractivity(number() | nil, number() | nil, map()) :: integer()
|
||||
def score_refractivity(min_gradient, bl_depth_m, band_config) do
|
||||
|
|
@ -194,17 +199,15 @@ defmodule Microwaveprop.Propagation.Scorer do
|
|||
end
|
||||
|
||||
@doc """
|
||||
Scores refractivity with optional native-profile duct info.
|
||||
Scores refractivity, ignoring `best_duct_band_ghz`.
|
||||
|
||||
`best_duct_band_ghz` comes from `hrrr_native_profiles` and represents the
|
||||
highest frequency the cell's native-resolution duct can trap. When it's
|
||||
≥ the target band's frequency the base score is boosted 1.15× because
|
||||
HRRR pressure-level gradients systematically under-read thin ducts the
|
||||
native profile can resolve (see Part 2c Apr 13 2026 findings).
|
||||
|
||||
A duct that only supports lower frequencies does NOT boost the score —
|
||||
it's a signal that the gradient we have is *all there is* at the target
|
||||
band.
|
||||
Accepts the native-profile duct band so existing call sites
|
||||
(`Scorer.score_grid_point/2`, `Recalibrator`, the Rust comparator)
|
||||
continue to compile, but the previous 1.15× boost was retired in the
|
||||
2026-04-25 revisions: at 10 GHz on n=52,341 matched contacts, cells
|
||||
whose native duct supported the band ran *shorter* (198 km, n=173)
|
||||
than no-duct cells (211 km, n=44,658). Refractivity is now the
|
||||
gradient-only score.
|
||||
"""
|
||||
@spec score_refractivity(number() | nil, number() | nil, number() | nil, map()) :: integer()
|
||||
def score_refractivity(min_gradient, bl_depth_m, best_duct_band_ghz, band_config) do
|
||||
|
|
@ -212,82 +215,31 @@ defmodule Microwaveprop.Propagation.Scorer do
|
|||
end
|
||||
|
||||
@doc """
|
||||
Scores refractivity with optional native-profile duct info, gated by
|
||||
Bulk Richardson number.
|
||||
Scores refractivity, ignoring `best_duct_band_ghz` and `bulk_richardson`.
|
||||
|
||||
`bulk_richardson` comes from `hrrr_native_profiles` and measures
|
||||
dynamic stability (lower = more stable). Native duct cells average
|
||||
8.7-18.4 for ducting vs 38.3 for non-ducting, so a low
|
||||
`best_duct_band_ghz` reading under turbulent conditions
|
||||
(Richardson ≥ 25) is likely a duct that would be broken up by
|
||||
mechanical mixing — in that case the 1.15× boost is *not* applied.
|
||||
|
||||
`nil` Richardson falls back to the 4-arity behaviour (boost applies
|
||||
whenever the duct band supports the target frequency).
|
||||
The Bulk Richardson gate existed only to suppress the native-duct
|
||||
boost during mechanically-mixed conditions. With the boost retired
|
||||
(see 4-arity doc), Richardson is irrelevant — every input collapses
|
||||
onto the gradient-only base score. Arity preserved so prod call
|
||||
sites don't need an emergency rewrite.
|
||||
"""
|
||||
@spec score_refractivity(number() | nil, number() | nil, number() | nil, number() | nil, map()) ::
|
||||
integer()
|
||||
def score_refractivity(nil, _bl_depth_m, _best_duct_band_ghz, _bulk_richardson, _band_config), do: 50
|
||||
|
||||
def score_refractivity(min_gradient, bl_depth_m, best_duct_band_ghz, bulk_richardson, %{
|
||||
freq_mhz: freq_mhz,
|
||||
humidity_effect: effect
|
||||
}) do
|
||||
def score_refractivity(min_gradient, _bl_depth_m, _best_duct_band_ghz, _bulk_richardson, %{humidity_effect: effect}) do
|
||||
thresholds = BandConfig.refractivity_thresholds()
|
||||
|
||||
base =
|
||||
case find_refractivity_threshold(min_gradient, thresholds, effect) do
|
||||
{:ok, score} ->
|
||||
score
|
||||
case find_refractivity_threshold(min_gradient, thresholds, effect) do
|
||||
{:ok, score} ->
|
||||
score
|
||||
|
||||
:none ->
|
||||
{beneficial_default, harmful_default} = BandConfig.refractivity_default()
|
||||
if effect == :beneficial, do: beneficial_default, else: harmful_default
|
||||
end
|
||||
|
||||
base
|
||||
|> apply_hpbl_multiplier(bl_depth_m)
|
||||
|> apply_native_duct_boost(best_duct_band_ghz, bulk_richardson, freq_mhz)
|
||||
end
|
||||
|
||||
# HPBL multiplier — applied to the base refractivity score. Calibrated to the
|
||||
# binned 10 GHz data: <200 m → 230 km avg, 200–500 → 186, 500–1500 → ~160,
|
||||
# 1500–2000 → 137, ≥2000 → 100. nil HPBL keeps the score unchanged.
|
||||
defp apply_hpbl_multiplier(base, nil), do: base
|
||||
|
||||
defp apply_hpbl_multiplier(base, hpbl) when hpbl < 200, do: clamp_score(base * 1.10)
|
||||
defp apply_hpbl_multiplier(base, hpbl) when hpbl < 500, do: clamp_score(base * 1.05)
|
||||
defp apply_hpbl_multiplier(base, hpbl) when hpbl < 1500, do: base
|
||||
defp apply_hpbl_multiplier(base, hpbl) when hpbl < 2000, do: clamp_score(base * 0.92)
|
||||
defp apply_hpbl_multiplier(base, _hpbl), do: clamp_score(base * 0.78)
|
||||
|
||||
# Native-profile duct boost — 1.15× when the cell's best duct supports the
|
||||
# target band's frequency AND the Bulk Richardson number is in the stable
|
||||
# regime (< 25). HRRR pressure-level gradients systematically under-read
|
||||
# thin ducts, so when hrrr_native_profiles reports a duct band at or above
|
||||
# the target frequency under stable conditions, the base gradient score is
|
||||
# under-estimating the real channel. A duct reading with high Richardson
|
||||
# is evidence of mechanical mixing that would shred the trapping layer —
|
||||
# don't boost in that case.
|
||||
@bulk_richardson_stable_max 25.0
|
||||
|
||||
defp apply_native_duct_boost(base, nil, _bulk_richardson, _freq_mhz), do: base
|
||||
|
||||
defp apply_native_duct_boost(base, best_duct_band_ghz, bulk_richardson, freq_mhz) do
|
||||
target_ghz = freq_mhz / 1_000
|
||||
|
||||
if best_duct_band_ghz >= target_ghz and stable_atmosphere?(bulk_richardson) do
|
||||
clamp_score(base * 1.15)
|
||||
else
|
||||
base
|
||||
:none ->
|
||||
{beneficial_default, harmful_default} = BandConfig.refractivity_default()
|
||||
if effect == :beneficial, do: beneficial_default, else: harmful_default
|
||||
end
|
||||
end
|
||||
|
||||
# nil Richardson is treated as stable (no information → don't penalise).
|
||||
# Explicit Richardson ≥ 25 gates the boost.
|
||||
defp stable_atmosphere?(nil), do: true
|
||||
defp stable_atmosphere?(r) when is_number(r), do: r < @bulk_richardson_stable_max
|
||||
|
||||
@doc """
|
||||
Inverse-sensor boost from commercial LOS link degradation.
|
||||
|
||||
|
|
|
|||
|
|
@ -16,8 +16,6 @@ use crate::band_config::{
|
|||
};
|
||||
use crate::region;
|
||||
|
||||
const BULK_RICHARDSON_STABLE_MAX: f64 = 25.0;
|
||||
|
||||
/// Inputs to `composite_score`. All f64 everywhere to stay close to
|
||||
/// Erlang's single-float world.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
|
|
@ -209,13 +207,20 @@ pub fn score_td_depression(temp_f: f64, dewpoint_f: f64, band: &BandConfig) -> i
|
|||
}
|
||||
}
|
||||
|
||||
// ── Factor 4: refractivity gradient (+ HPBL multiplier + native duct boost) ──
|
||||
// ── Factor 4: refractivity gradient (gradient-only as of 2026-04-25) ─
|
||||
//
|
||||
// HPBL boundary-layer multiplier and native-profile 1.15× duct boost both
|
||||
// retired — see `docs/algo-reports/2026-04-25-algo-revisions.md`. The
|
||||
// `bl_depth_m`, `best_duct_band_ghz`, and `bulk_richardson` parameters are
|
||||
// retained on the call signature so callers (and the Elixir parity test
|
||||
// suite under `mix rust.golden`) keep the same shape, but they no longer
|
||||
// influence the score.
|
||||
|
||||
pub fn score_refractivity(
|
||||
min_gradient: Option<f64>,
|
||||
bl_depth_m: Option<f64>,
|
||||
best_duct_band_ghz: Option<f64>,
|
||||
bulk_richardson: Option<f64>,
|
||||
_bl_depth_m: Option<f64>,
|
||||
_best_duct_band_ghz: Option<f64>,
|
||||
_bulk_richardson: Option<f64>,
|
||||
band: &BandConfig,
|
||||
) -> i32 {
|
||||
let Some(gradient) = min_gradient else {
|
||||
|
|
@ -226,7 +231,7 @@ pub fn score_refractivity(
|
|||
.iter()
|
||||
.find(|(max, _, _)| gradient < *max as f64);
|
||||
|
||||
let base = match base_match {
|
||||
match base_match {
|
||||
Some((_, b, h)) => match band.humidity_effect {
|
||||
HumidityEffect::Beneficial => *b,
|
||||
HumidityEffect::Harmful => *h,
|
||||
|
|
@ -239,47 +244,6 @@ pub fn score_refractivity(
|
|||
h
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
let with_hpbl = apply_hpbl_multiplier(base, bl_depth_m);
|
||||
apply_native_duct_boost(
|
||||
with_hpbl,
|
||||
best_duct_band_ghz,
|
||||
bulk_richardson,
|
||||
band.freq_mhz,
|
||||
)
|
||||
}
|
||||
|
||||
fn apply_hpbl_multiplier(base: i32, hpbl: Option<f64>) -> i32 {
|
||||
let mul = match hpbl {
|
||||
None => return base,
|
||||
Some(v) if v < 200.0 => 1.10,
|
||||
Some(v) if v < 500.0 => 1.05,
|
||||
Some(v) if v < 1500.0 => 1.0,
|
||||
Some(v) if v < 2000.0 => 0.92,
|
||||
Some(_) => 0.78,
|
||||
};
|
||||
clamp_score_f(base as f64 * mul)
|
||||
}
|
||||
|
||||
fn apply_native_duct_boost(
|
||||
base: i32,
|
||||
best_duct_band_ghz: Option<f64>,
|
||||
bulk_richardson: Option<f64>,
|
||||
freq_mhz: u32,
|
||||
) -> i32 {
|
||||
let Some(duct_band_ghz) = best_duct_band_ghz else {
|
||||
return base;
|
||||
};
|
||||
let target_ghz = freq_mhz as f64 / 1000.0;
|
||||
let stable = match bulk_richardson {
|
||||
None => true,
|
||||
Some(r) => r < BULK_RICHARDSON_STABLE_MAX,
|
||||
};
|
||||
if duct_band_ghz >= target_ghz && stable {
|
||||
clamp_score_f(base as f64 * 1.15)
|
||||
} else {
|
||||
base
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -694,49 +658,49 @@ mod tests {
|
|||
);
|
||||
}
|
||||
|
||||
// HPBL multiplier and native-profile duct boost retired in the
|
||||
// 2026-04-25 algo revisions (see `docs/algo-reports/`). Both helpers
|
||||
// are gone; the parameter slots remain so existing callers don't need
|
||||
// a coordinated rewrite. Tests below pin the post-retirement
|
||||
// behaviour: every variant collapses onto the gradient-only score.
|
||||
|
||||
#[test]
|
||||
fn shallow_bl_boosts_score() {
|
||||
fn hpbl_no_longer_affects_score() {
|
||||
let shallow = score_refractivity(Some(-100.0), Some(150.0), None, None, b10g());
|
||||
let baseline = score_refractivity(Some(-100.0), BASELINE_BL, None, None, b10g());
|
||||
assert!(shallow > baseline);
|
||||
assert!(shallow <= 100);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn deep_bl_penalises_score() {
|
||||
let deep = score_refractivity(Some(-100.0), Some(2200.0), None, None, b10g());
|
||||
let baseline = score_refractivity(Some(-100.0), BASELINE_BL, None, None, b10g());
|
||||
assert!(deep < baseline);
|
||||
assert!(deep >= 0);
|
||||
assert_eq!(shallow, baseline);
|
||||
assert_eq!(deep, baseline);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn shallow_bl_also_lifts_default_fallback() {
|
||||
fn hpbl_no_longer_affects_default_fallback() {
|
||||
let shallow = score_refractivity(Some(-30.0), Some(150.0), None, None, b10g());
|
||||
let baseline = score_refractivity(Some(-30.0), BASELINE_BL, None, None, b10g());
|
||||
assert!(shallow > baseline);
|
||||
let deep = score_refractivity(Some(-30.0), Some(2200.0), None, None, b10g());
|
||||
assert_eq!(shallow, baseline);
|
||||
assert_eq!(deep, baseline);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn native_duct_boosts_matching_band() {
|
||||
let boosted = score_refractivity(Some(-80.0), Some(800.0), Some(15.0), None, b10g());
|
||||
fn native_duct_no_longer_boosts_matching_band() {
|
||||
let with_duct = score_refractivity(Some(-80.0), Some(800.0), Some(15.0), None, b10g());
|
||||
let plain = score_refractivity(Some(-80.0), Some(800.0), None, None, b10g());
|
||||
assert!(boosted > plain);
|
||||
assert!(boosted <= 100);
|
||||
assert_eq!(with_duct, plain);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn native_duct_below_target_no_boost() {
|
||||
fn native_duct_below_target_still_matches_plain() {
|
||||
let unchanged = score_refractivity(Some(-80.0), Some(800.0), Some(3.0), None, b10g());
|
||||
let plain = score_refractivity(Some(-80.0), Some(800.0), None, None, b10g());
|
||||
assert_eq!(unchanged, plain);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn richardson_gates_the_boost() {
|
||||
fn richardson_no_longer_gates_anything() {
|
||||
let stable = score_refractivity(Some(-80.0), Some(800.0), Some(15.0), Some(12.0), b10g());
|
||||
let plain = score_refractivity(Some(-80.0), Some(800.0), None, None, b10g());
|
||||
assert!(stable > plain);
|
||||
assert_eq!(stable, plain);
|
||||
|
||||
let turbulent =
|
||||
score_refractivity(Some(-80.0), Some(800.0), Some(15.0), Some(40.0), b10g());
|
||||
|
|
|
|||
|
|
@ -270,8 +270,13 @@ defmodule Microwaveprop.Propagation.ScorerTest do
|
|||
# ── score_refractivity/3 ─────────────────────────────────────────
|
||||
|
||||
describe "score_refractivity/3" do
|
||||
# 750 m sits in the [500, 1500) baseline-multiplier band so existing
|
||||
# threshold assertions don't pick up the HPBL adjustment.
|
||||
# As of the 2026-04-25 algo revisions the HPBL multiplier and the
|
||||
# native-profile 1.15× duct boost are both retired (`docs/algo-reports/
|
||||
# 2026-04-25-algo-revisions.md` Recommendations 1 and 2). HPBL had
|
||||
# rho ≤ 0.092 at every band and +0.004 at 10 GHz on n=47,418 — well
|
||||
# below the noise floor. The function arities are kept so callers
|
||||
# continue compiling, but they all collapse onto the gradient-only
|
||||
# base score.
|
||||
@baseline_bl 750
|
||||
|
||||
test "strong ducting gradient returns high score for beneficial" do
|
||||
|
|
@ -293,51 +298,46 @@ defmodule Microwaveprop.Propagation.ScorerTest do
|
|||
assert Scorer.score_refractivity(nil, @baseline_bl, @band_10g) == 50
|
||||
end
|
||||
|
||||
test "nil bl_depth_m treated as baseline (no HPBL multiplier)" do
|
||||
# A nil HPBL must not crash and must produce the unmultiplied score.
|
||||
assert Scorer.score_refractivity(-160, nil, @band_10g) == 92
|
||||
test "nil bl_depth_m matches any other bl_depth_m (HPBL retired)" do
|
||||
assert Scorer.score_refractivity(-160, nil, @band_10g) ==
|
||||
Scorer.score_refractivity(-160, @baseline_bl, @band_10g)
|
||||
end
|
||||
|
||||
test "shallow boundary layer (<200 m) boosts the gradient score" do
|
||||
# Same -100 gradient: shallow BL should clearly out-score a baseline BL.
|
||||
test "shallow boundary layer is no longer a boost" do
|
||||
shallow = Scorer.score_refractivity(-100, 150, @band_10g)
|
||||
baseline = Scorer.score_refractivity(-100, @baseline_bl, @band_10g)
|
||||
assert shallow > baseline
|
||||
assert shallow <= 100
|
||||
assert shallow == baseline
|
||||
end
|
||||
|
||||
test "deep boundary layer (>=2000 m) penalises the gradient score" do
|
||||
# Same -100 gradient: deep BL drags the score down.
|
||||
test "deep boundary layer is no longer a penalty" do
|
||||
deep = Scorer.score_refractivity(-100, 2200, @band_10g)
|
||||
baseline = Scorer.score_refractivity(-100, @baseline_bl, @band_10g)
|
||||
assert deep < baseline
|
||||
assert deep >= 0
|
||||
assert deep == baseline
|
||||
end
|
||||
|
||||
test "shallow BL also lifts the weak-gradient default" do
|
||||
# The HPBL multiplier applies to the default fallback too — the binned
|
||||
# data shows shallow-BL paths run ~230 km even when HRRR gradient is weak.
|
||||
test "HPBL has no effect on the default-fallback branch either" do
|
||||
shallow_default = Scorer.score_refractivity(-30, 150, @band_10g)
|
||||
baseline_default = Scorer.score_refractivity(-30, @baseline_bl, @band_10g)
|
||||
assert shallow_default > baseline_default
|
||||
deep_default = Scorer.score_refractivity(-30, 2200, @band_10g)
|
||||
assert shallow_default == baseline_default
|
||||
assert deep_default == baseline_default
|
||||
end
|
||||
end
|
||||
|
||||
describe "score_refractivity/4 with native-profile duct support" do
|
||||
# New 4-arity variant: adds best_duct_band_ghz from hrrr_native_profiles so
|
||||
# cells with a thin native duct that supports the target band get a boost
|
||||
# the HRRR pressure-level gradient alone would miss. Cells where the native
|
||||
# duct only supports sub-band frequencies do NOT get a boost.
|
||||
# Native-duct 1.15× boost retired — see Recommendation 1 in
|
||||
# `docs/algo-reports/2026-04-25-algo-revisions.md`. At 10 GHz on
|
||||
# n=52,341, contacts where the native duct supports the band ran
|
||||
# *shorter* (198 km) than no-duct contacts (211 km). Arity kept,
|
||||
# behaviour collapses onto the 3-arity gradient-only score.
|
||||
|
||||
test "native duct supporting the target band boosts the score" do
|
||||
boosted = Scorer.score_refractivity(-80, 800, 15.0, @band_10g)
|
||||
test "native duct supporting the target band no longer boosts" do
|
||||
with_duct = Scorer.score_refractivity(-80, 800, 15.0, @band_10g)
|
||||
plain = Scorer.score_refractivity(-80, 800, nil, @band_10g)
|
||||
assert boosted > plain
|
||||
assert boosted <= 100
|
||||
assert with_duct == plain
|
||||
end
|
||||
|
||||
test "native duct below the target band does NOT boost" do
|
||||
# A 3 GHz duct is useless for 10 GHz.
|
||||
test "native duct below the target band still produces the same plain score" do
|
||||
unchanged = Scorer.score_refractivity(-80, 800, 3.0, @band_10g)
|
||||
plain = Scorer.score_refractivity(-80, 800, nil, @band_10g)
|
||||
assert unchanged == plain
|
||||
|
|
@ -352,42 +352,42 @@ defmodule Microwaveprop.Propagation.ScorerTest do
|
|||
assert Scorer.score_refractivity(nil, 600, 15.0, @band_10g) == 50
|
||||
end
|
||||
|
||||
test "24 GHz is boosted only when native duct supports ≥24 GHz" do
|
||||
test "24 GHz is no longer boosted by any native duct band" do
|
||||
weak = Scorer.score_refractivity(-80, 800, 15.0, @band_24g)
|
||||
strong = Scorer.score_refractivity(-80, 800, 30.0, @band_24g)
|
||||
plain = Scorer.score_refractivity(-80, 800, nil, @band_24g)
|
||||
assert weak == plain
|
||||
assert strong > plain
|
||||
assert strong == plain
|
||||
end
|
||||
end
|
||||
|
||||
describe "score_refractivity/5 with bulk-Richardson gating" do
|
||||
# 5-arity variant: the native-duct boost only applies when Bulk
|
||||
# Richardson number is in the stable regime (< 25). Native duct
|
||||
# cells average 8.7–18.4 for ducting vs 38.3 for non-ducting, so a
|
||||
# low best_duct_band_ghz reading with high Richardson is likely a
|
||||
# duct that would be shredded by mechanical mixing — we should not
|
||||
# boost it (Part 2c of algo.md).
|
||||
# The Richardson gate existed only to suppress the native-duct
|
||||
# boost during mechanically-mixed conditions. Now that the boost
|
||||
# itself is retired, Richardson is irrelevant — every input maps
|
||||
# to the same plain gradient-only score. Arity preserved so prod
|
||||
# call sites in propagation/recalibrator and the Rust comparator
|
||||
# don't need an emergency rewrite.
|
||||
|
||||
test "stable Richardson (<25) + matching duct band → boost applies" do
|
||||
boosted = Scorer.score_refractivity(-80, 800, 15.0, 12.0, @band_10g)
|
||||
test "stable Richardson + matching duct band ≡ plain" do
|
||||
stable = Scorer.score_refractivity(-80, 800, 15.0, 12.0, @band_10g)
|
||||
plain = Scorer.score_refractivity(-80, 800, nil, nil, @band_10g)
|
||||
assert boosted > plain
|
||||
assert stable == plain
|
||||
end
|
||||
|
||||
test "turbulent Richardson (≥25) + matching duct band → boost suppressed" do
|
||||
test "turbulent Richardson + matching duct band ≡ plain" do
|
||||
gated = Scorer.score_refractivity(-80, 800, 15.0, 40.0, @band_10g)
|
||||
plain = Scorer.score_refractivity(-80, 800, nil, nil, @band_10g)
|
||||
assert gated == plain
|
||||
end
|
||||
|
||||
test "nil Richardson preserves 4-arity behaviour (boost still applies)" do
|
||||
test "nil Richardson ≡ plain" do
|
||||
with_nil_r = Scorer.score_refractivity(-80, 800, 15.0, nil, @band_10g)
|
||||
four_arity = Scorer.score_refractivity(-80, 800, 15.0, @band_10g)
|
||||
assert with_nil_r == four_arity
|
||||
end
|
||||
|
||||
test "boundary case: Richardson at exactly 25 suppresses the boost" do
|
||||
test "Richardson at the old 25 boundary ≡ plain" do
|
||||
gated = Scorer.score_refractivity(-80, 800, 15.0, 25.0, @band_10g)
|
||||
plain = Scorer.score_refractivity(-80, 800, nil, nil, @band_10g)
|
||||
assert gated == plain
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue