Commit graph

1 commit

Author SHA1 Message Date
fc3eb58910
Refresh propagation algo against expanded prod corpus (2026-04-13)
Direct queries against prod (75M HRRR profiles, 16,864 soundings, 392k
surface obs, new hrrr_native_profiles table) drive these changes:

* Reinstate shallow-BL bonus as an HPBL multiplier on the refractivity
  score (1.10× <200m → 0.78× ≥2000m). The Apr 11 "shallow BL bonus
  removed" conclusion was an artifact of the prior matching strategy;
  with the cleaner contact↔HRRR join (n=680) the binned data goes 230 km
  avg at HPBL <200m vs 100 km at ≥2000m, monotonic across all bins.
* Add 6 missing bands (142, 145, 288, 322, 403, 411 GHz) so contacts in
  those bands stop being silently dropped. Coefficients extrapolate
  ITU-R P.676/P.838 trends from the existing 134/241 GHz entries.
* Bump ERA5 poll timeout 10 min → 1 hour and Era5MonthBatchWorker
  max_attempts 3 → 5 so CDS slowness stops discarding tiles. Also dump
  the full failure body when CDS omits the message field — the prior
  "ERA5 job failed: nil" was masking real error reasons in oban_jobs.
* Add scripts/recalibrate_algo.py so this analysis can be re-run any
  time new data lands without manual SQL. Reads PROP_PROD_DB_URL from
  .envrc, drops a Markdown report into docs/algo-reports/.
* Append a dated Part 2c to algo.md documenting the corpus expansion,
  the honest accounting of historical HRRR coverage (only ~1,020 of 58k
  contacts are precision-matchable), and the empty era5/rtma/climatology
  tables. Update the gaseous-absorption and rain-attenuation tables to
  include the new bands.

Test suite: 1,335 tests, 0 failures.
2026-04-13 10:59:01 -05:00