Commit graph

2 commits

Author SHA1 Message Date
Graham McInitre
b3ad3bdbe6 fix: sweep bugs across Python MQTT listener, Elixir PSKR/propagation, and Rust
Python pskr_mqtt_listen.py:
- Guard against malformed CONNACK/SUBACK/PUBLISH packets
- Fix _s() truthiness: is not None instead of if v (zero SNR was lost)
- EINTR-safe select loop, OOB data handling, VBInt overflow check
- Proper socket cleanup with try/finally + DISCONNECT on shutdown

Elixir:
- Log dropped spot errors in aggregator (was silently discarded)
- Wire retain_scores_window into NotifyListener chain completion
- Recurse sweep_tmp_dir into band/weather_scalars subdirectories
- Rescue recalibrator.run/0 to write failed status row on crash
- Handle nil in fmt_snr/fmt_callsigns (no more nil dB crashes)
- Replace Stream.run with Enum.reduce logging exits in poll_worker
- Handle File.stat race in ms_footprints prune, grid_center nil log
- Fix unused variables, stale comments, missing get_path!/1

Rust:
- Graceful JoinError handling in fetcher/hrdps_fetcher (no more panics)
- round_to_5min returns Option (leap-second safe)
- Acquire/Release ordering on shutdown flags (was Relaxed)
- Parameterized NOTIFY in db.rs, defensive scalar sweep, NaN guard
- OsString::push for tmp naming, clippy fixes
2026-07-20 19:48:45 -05:00
48000b0dca
feat(pskr): weekly recalibration analysis on the calibration corpus
`Pskr.Recalibrator.run/0` reads `pskr_calibration_samples`, bins
each sample per (band × feature), and writes spot-density stats to
`pskr_feature_bins` so an operator can read whether a feature
actually discriminates propagation at the threshold granularity
the scorer uses.

Bins, not regression: `BandConfig` already encodes scoring as
discrete thresholds, so the bin output matches that shape and an
operator can copy adjusted thresholds directly without translating
from regression coefficients.

Self-healing: corpus too thin ⇒ run row written with status
`skipped_insufficient_data` and the analysis is a no-op until next
fire. `min_total_samples = 1000` (≈ 4-5 days of CONUS PSKR
activity); per-band threshold is 100. Both surface in the run row's
`notes`.

Auto-applies nothing. Weight changes still go through human review
of `BandConfig.@band_configs` and a code commit. The recalibrator
is a read-only analyst that stays out of the production scoring
path.

Features binned (matching the scorer's discriminating fields):
  * pwat_mm — humidity U-shape candidate
  * hpbl_m — boundary layer (mechanism vs scoring re-eval)
  * min_refractivity_gradient — refractivity threshold validation
  * surface_pressure_mb — pressure-front proxy
  * kp_index — aurora boost magnitude tuning

Schema: two tables.
  * `pskr_recalibration_runs` — one row per fire with corpus
    stats, status, notes
  * `pskr_feature_bins` — one row per (run, band, feature, bin)
    with sample_count, spot_count_total/avg/p50/p90

Cron: `0 4 * * 0` (Sundays 04:00 UTC, off-peak, post-climatology).
Manual reruns enqueue with no args.

Tests cover the empty-corpus skip path, sub-threshold totals,
per-band threshold gating, the actual bin emission, nil-feature
handling, spot-count averaging, and the always-records-a-run
audit invariant. 8 new tests, 3282 total passing.

Backfill pipeline untouched.
2026-05-04 16:18:17 -05:00