prop/lib/microwaveprop
Graham McIntire 48000b0dca
feat(pskr): weekly recalibration analysis on the calibration corpus
`Pskr.Recalibrator.run/0` reads `pskr_calibration_samples`, bins
each sample per (band × feature), and writes spot-density stats to
`pskr_feature_bins` so an operator can read whether a feature
actually discriminates propagation at the threshold granularity
the scorer uses.

Bins, not regression: `BandConfig` already encodes scoring as
discrete thresholds, so the bin output matches that shape and an
operator can copy adjusted thresholds directly without translating
from regression coefficients.

Self-healing: corpus too thin ⇒ run row written with status
`skipped_insufficient_data` and the analysis is a no-op until next
fire. `min_total_samples = 1000` (≈ 4-5 days of CONUS PSKR
activity); per-band threshold is 100. Both surface in the run row's
`notes`.

Auto-applies nothing. Weight changes still go through human review
of `BandConfig.@band_configs` and a code commit. The recalibrator
is a read-only analyst that stays out of the production scoring
path.

Features binned (matching the scorer's discriminating fields):
  * pwat_mm — humidity U-shape candidate
  * hpbl_m — boundary layer (mechanism vs scoring re-eval)
  * min_refractivity_gradient — refractivity threshold validation
  * surface_pressure_mb — pressure-front proxy
  * kp_index — aurora boost magnitude tuning

Schema: two tables.
  * `pskr_recalibration_runs` — one row per fire with corpus
    stats, status, notes
  * `pskr_feature_bins` — one row per (run, band, feature, bin)
    with sample_count, spot_count_total/avg/p50/p90

Cron: `0 4 * * 0` (Sundays 04:00 UTC, off-peak, post-climatology).
Manual reruns enqueue with no args.

Tests cover the empty-corpus skip path, sub-threshold totals,
per-band threshold gating, the actual bin emission, nil-feature
handling, spot-count averaging, and the always-records-a-run
audit invariant. 8 new tests, 3282 total passing.

Backfill pipeline untouched.
2026-05-04 16:18:17 -05:00
..
accounts feat(accounts): derive 10-char home_grid from explicit lat/lon 2026-04-25 17:04:45 -05:00
aprs polish(aprs): consolidate callsign regex, document raise, note negative-sample v1 limit 2026-05-01 13:10:57 -05:00
backtest fix(dialyzer): clear 125+ warnings under strict flags 2026-04-21 10:30:06 -05:00
beacon_monitors Add dialyzer specs and types across the codebase 2026-04-12 08:55:04 -05:00
beacons Short-circuit RangeEstimate below 5.76 GHz to stop browser lockup 2026-04-14 16:56:24 -05:00
buildings fix(buildings): use explicit step in descending range in encode_quadkey 2026-04-26 10:54:44 -05:00
callsign_location Drop gridmap.org dependency, resolve callsigns locally 2026-04-12 17:43:29 -05:00
canopy feat(rover): add canopy clutter penalty + Canopy.BulkFetch downloader 2026-04-26 12:57:50 -05:00
commercial test: push line coverage from 71.7% → 78.67% 2026-04-23 18:43:18 -05:00
ionosphere fix(ionosphere): disable TLS verify for GIRO to unbreak ionosonde fetch 2026-04-22 09:49:03 -05:00
prom_ex perf(prop): cap ScoreCache LRU + drop eager warmers to fix hot-pod OOMs 2026-04-24 18:49:36 -05:00
propagation feat(scoring): aurora boost on VHF/low-UHF during geomagnetic storms 2026-05-04 16:02:49 -05:00
pskr feat(pskr): weekly recalibration analysis on the calibration corpus 2026-05-04 16:18:17 -05:00
qrz feat(telemetry): broaden Instrument span coverage 2026-04-18 17:25:33 -05:00
radio feat(contacts): record + show flagger attribution; admin delete 2026-05-04 10:15:13 -05:00
rover refactor(rover-locations): rename status atoms ideal→good, off_limits→bad 2026-05-03 11:13:18 -05:00
rover_planning feat(rover-planning): worker pre-computes full /path output, PathLive renders cached 2026-05-03 14:58:16 -05:00
space_weather fix(ingest): dedupe ASOS duplicates + shrink SWPC decode errors 2026-04-21 16:39:17 -05:00
terrain refactor: deduplicate helpers and simplify Scorer 2026-04-28 14:14:34 -05:00
weather chore: delete dead RTMA + METAR5 code paths 2026-05-04 15:09:19 -05:00
workers feat(pskr): weekly recalibration analysis on the calibration corpus 2026-05-04 16:18:17 -05:00
accounts.ex feat(accounts): auto-prefill home QTH from QRZ on register + backfill on boot 2026-04-25 17:03:01 -05:00
application.ex feat(pskr): ingest PSK Reporter MQTT firehose for weather correlation 2026-05-04 09:24:20 -05:00
aprs.ex polish(aprs): consolidate callsign regex, document raise, note negative-sample v1 limit 2026-05-01 13:10:57 -05:00
aprs_repo.ex feat(aprs): add read-only AprsRepo for aprs.me database access 2026-05-01 12:25:33 -05:00
backtest.ex chore(specs): add @spec to public API in accounts/release/backtest 2026-04-21 13:32:42 -05:00
beacon_monitors.ex Add dialyzer specs and types across the codebase 2026-04-12 08:55:04 -05:00
beacons.ex fix(dialyzer): actually fix LiveTable warnings instead of hiding them 2026-04-21 12:58:10 -05:00
cache.ex fix(cache): add periodic TTL sweeper to bound ETS growth 2026-04-21 13:19:53 -05:00
callsign_location.ex Drop gridmap.org dependency, resolve callsigns locally 2026-04-12 17:43:29 -05:00
canopy.ex feat(canopy): add tree-canopy height layer for path obstruction analysis 2026-04-26 12:51:48 -05:00
commercial.ex fix: April 2026 codebase review — address 13 bugs across propagation chain 2026-04-25 10:52:51 -05:00
format.ex refactor(format): consolidate number/bytes formatters + handle flaky IEM transports 2026-04-21 17:09:41 -05:00
geo.ex refactor: deduplicate helpers and simplify Scorer 2026-04-28 14:14:34 -05:00
geocoder.ex feat(telemetry): broaden Instrument span coverage 2026-04-18 17:25:33 -05:00
instrument.ex feat(telemetry): wide instrumentation + bump hrrr to 2 per pod 2026-04-18 16:33:34 -05:00
ionosphere.ex refactor: deduplicate helpers and simplify Scorer 2026-04-28 14:14:34 -05:00
mailer.ex Send mail as prop@w5isp.com with reply-to graham@mcintire.me 2026-04-12 14:00:38 -05:00
markdown.ex Add dialyzer specs and types across the codebase 2026-04-12 08:55:04 -05:00
moon.ex feat(eme): add /eme Earth-Moon-Earth calculator 2026-04-23 16:18:20 -05:00
oban_error_reporter.ex feat(observability): attach Oban exception telemetry tap 2026-04-21 14:13:53 -05:00
prom_ex.ex fix: stabilize flaky tests + silence PromEx Oban poller in test 2026-04-21 13:49:07 -05:00
propagation.ex feat(scoring): aurora boost on VHF/low-UHF during geomagnetic storms 2026-05-04 16:02:49 -05:00
pskr.ex feat(pskr): preserve full locator precision from MQTT spots 2026-05-04 12:42:46 -05:00
qrz.ex Drop gridmap.org dependency, resolve callsigns locally 2026-04-12 17:43:29 -05:00
radio.ex feat(contacts): record + show flagger attribution; admin delete 2026-05-04 10:15:13 -05:00
release.ex chore(specs): add @spec to public API in accounts/release/backtest 2026-04-21 13:32:42 -05:00
repo.ex initial 2026-03-28 11:28:47 -05:00
repo_listener.ex chore(specs): add @spec to public API in accounts/release/backtest 2026-04-21 13:32:42 -05:00
rover.ex feat(rover): admins can edit/delete any rover location 2026-05-02 17:04:52 -05:00
rover_planning.ex feat(rover-planning): backfill resets stale :complete paths to repopulate term blob 2026-05-03 15:16:42 -05:00
space_weather.ex refactor(db): use explicit on_conflict set clauses over :replace_all_except 2026-04-21 14:22:15 -05:00
terrain.ex Add dialyzer specs and types across the codebase 2026-04-12 08:55:04 -05:00
valkey.ex feat(weather): GridCache backend pluggable, Valkey for cross-pod cache 2026-05-03 17:00:51 -05:00
weather.ex chore: delete dead RTMA + METAR5 code paths 2026-05-04 15:09:19 -05:00