Graham McIntire
48000b0dca
`Pskr.Recalibrator.run/0` reads `pskr_calibration_samples`, bins
each sample per (band × feature), and writes spot-density stats to
`pskr_feature_bins` so an operator can read whether a feature
actually discriminates propagation at the threshold granularity
the scorer uses.
Bins, not regression: `BandConfig` already encodes scoring as
discrete thresholds, so the bin output matches that shape and an
operator can copy adjusted thresholds directly without translating
from regression coefficients.
Self-healing: corpus too thin ⇒ run row written with status
`skipped_insufficient_data` and the analysis is a no-op until next
fire. `min_total_samples = 1000` (≈ 4-5 days of CONUS PSKR
activity); per-band threshold is 100. Both surface in the run row's
`notes`.
Auto-applies nothing. Weight changes still go through human review
of `BandConfig.@band_configs` and a code commit. The recalibrator
is a read-only analyst that stays out of the production scoring
path.
Features binned (matching the scorer's discriminating fields):
* pwat_mm — humidity U-shape candidate
* hpbl_m — boundary layer (mechanism vs scoring re-eval)
* min_refractivity_gradient — refractivity threshold validation
* surface_pressure_mb — pressure-front proxy
* kp_index — aurora boost magnitude tuning
Schema: two tables.
* `pskr_recalibration_runs` — one row per fire with corpus
stats, status, notes
* `pskr_feature_bins` — one row per (run, band, feature, bin)
with sample_count, spot_count_total/avg/p50/p90
Cron: `0 4 * * 0` (Sundays 04:00 UTC, off-peak, post-climatology).
Manual reruns enqueue with no args.
Tests cover the empty-corpus skip path, sub-threshold totals,
per-band threshold gating, the actual bin emission, nil-feature
handling, spot-count averaging, and the always-records-a-run
audit invariant. 8 new tests, 3282 total passing.
Backfill pipeline untouched.
35 lines
1.1 KiB
Elixir
35 lines
1.1 KiB
Elixir
defmodule Microwaveprop.Workers.PskrRecalibrationWorker do
|
|
@moduledoc """
|
|
Weekly cron that runs `Pskr.Recalibrator` against the
|
|
PSKR calibration corpus.
|
|
|
|
Scheduled for Sunday 04:00 UTC (off-peak, after the climatology
|
|
rebuild and before the Monday-morning operator review window). At
|
|
fire time the recalibrator decides whether the corpus is dense
|
|
enough to bin — when it isn't, the run still records a row with
|
|
status `"skipped_insufficient_data"` so the audit trail captures
|
|
every fire.
|
|
|
|
Manual reruns: enqueue this worker with no args. It always
|
|
produces a fresh run record; previous runs stay queryable for
|
|
trend comparison.
|
|
"""
|
|
use Oban.Worker,
|
|
queue: :backfill_enqueue,
|
|
max_attempts: 3,
|
|
unique: [
|
|
period: 86_400,
|
|
states: [:available, :scheduled, :executing, :retryable]
|
|
]
|
|
|
|
alias Microwaveprop.Pskr.Recalibrator
|
|
|
|
require Logger
|
|
|
|
@impl Oban.Worker
|
|
def perform(%Oban.Job{}) do
|
|
run = Recalibrator.run()
|
|
Logger.info("PskrRecalibrationWorker: status=#{run.status} sample_count=#{run.sample_count}")
|
|
:ok
|
|
end
|
|
end
|