Graham McIntire
48000b0dca
`Pskr.Recalibrator.run/0` reads `pskr_calibration_samples`, bins
each sample per (band × feature), and writes spot-density stats to
`pskr_feature_bins` so an operator can read whether a feature
actually discriminates propagation at the threshold granularity
the scorer uses.
Bins, not regression: `BandConfig` already encodes scoring as
discrete thresholds, so the bin output matches that shape and an
operator can copy adjusted thresholds directly without translating
from regression coefficients.
Self-healing: corpus too thin ⇒ run row written with status
`skipped_insufficient_data` and the analysis is a no-op until next
fire. `min_total_samples = 1000` (≈ 4-5 days of CONUS PSKR
activity); per-band threshold is 100. Both surface in the run row's
`notes`.
Auto-applies nothing. Weight changes still go through human review
of `BandConfig.@band_configs` and a code commit. The recalibrator
is a read-only analyst that stays out of the production scoring
path.
Features binned (matching the scorer's discriminating fields):
* pwat_mm — humidity U-shape candidate
* hpbl_m — boundary layer (mechanism vs scoring re-eval)
* min_refractivity_gradient — refractivity threshold validation
* surface_pressure_mb — pressure-front proxy
* kp_index — aurora boost magnitude tuning
Schema: two tables.
* `pskr_recalibration_runs` — one row per fire with corpus
stats, status, notes
* `pskr_feature_bins` — one row per (run, band, feature, bin)
with sample_count, spot_count_total/avg/p50/p90
Cron: `0 4 * * 0` (Sundays 04:00 UTC, off-peak, post-climatology).
Manual reruns enqueue with no args.
Tests cover the empty-corpus skip path, sub-threshold totals,
per-band threshold gating, the actual bin emission, nil-feature
handling, spot-count averaging, and the always-records-a-run
audit invariant. 8 new tests, 3282 total passing.
Backfill pipeline untouched.
61 lines
2.3 KiB
Elixir
61 lines
2.3 KiB
Elixir
defmodule Microwaveprop.Repo.Migrations.CreatePskrRecalibrationTables do
|
|
use Ecto.Migration
|
|
|
|
# Two-table layout for the PSKR-driven recalibration analysis:
|
|
#
|
|
# * `pskr_recalibration_runs` — one row per scheduled run with
|
|
# corpus stats and run status. Cheap to scan, fronts the
|
|
# analysis tables.
|
|
# * `pskr_feature_bins` — one row per (run, band, feature, bin)
|
|
# with spot-density stats. The recalibrator generates these by
|
|
# bucketing the corpus on each scoring feature so an operator
|
|
# can read directly: "at PWAT 40-55 mm, 10 GHz cells averaged
|
|
# N spots vs M at 15-25 mm — is that real or sampling noise?"
|
|
#
|
|
# Bins beat continuous regression for the operator workflow:
|
|
# the algorithm's per-band weights are also discrete thresholds
|
|
# (`@band_configs`), so matching the shape lets the operator copy
|
|
# adjusted thresholds back into `BandConfig` without translation.
|
|
|
|
def change do
|
|
create table(:pskr_recalibration_runs, primary_key: false) do
|
|
add :id, :binary_id, primary_key: true
|
|
add :run_at, :utc_datetime, null: false
|
|
add :status, :string, null: false
|
|
add :sample_count, :integer, null: false, default: 0
|
|
add :band_count, :integer, null: false, default: 0
|
|
add :earliest_sample, :utc_datetime
|
|
add :latest_sample, :utc_datetime
|
|
add :avg_spots_per_sample, :float
|
|
add :notes, :text
|
|
|
|
timestamps(type: :utc_datetime)
|
|
end
|
|
|
|
create index(:pskr_recalibration_runs, [:run_at])
|
|
|
|
create table(:pskr_feature_bins, primary_key: false) do
|
|
add :id, :binary_id, primary_key: true
|
|
|
|
add :run_id,
|
|
references(:pskr_recalibration_runs, type: :binary_id, on_delete: :delete_all),
|
|
null: false
|
|
|
|
add :band, :string, null: false
|
|
add :feature, :string, null: false
|
|
add :bin_label, :string, null: false
|
|
add :bin_min, :float
|
|
add :bin_max, :float
|
|
add :sample_count, :integer, null: false, default: 0
|
|
add :spot_count_total, :integer, null: false, default: 0
|
|
add :spot_count_avg, :float
|
|
add :spot_count_p50, :float
|
|
add :spot_count_p90, :float
|
|
|
|
timestamps(type: :utc_datetime)
|
|
end
|
|
|
|
create unique_index(:pskr_feature_bins, [:run_id, :band, :feature, :bin_label])
|
|
create index(:pskr_feature_bins, [:run_id])
|
|
end
|
|
end
|