No description
Graham McIntire
7a2b1f292c
PSK Reporter is now an always-on data feed, so the calibration
corpus that recalibration will eventually train against can grow
hourly from when the firehose started. One row per (hour, band,
0.125° midpoint cell) joins three sources:
* `pskr_spots_hourly` — observed spot density (truth signal)
* `hrrr_profiles` nearest match at the cell midpoint at hour
boundary (atmospheric features: T, Td, PWAT, P, dN/dh, HPBL,
ducting flag)
* `geomagnetic_observations` latest Kp at the hour (space weather)
Predicted scores are intentionally NOT stored — they're a function
of the algorithm version under evaluation. Storing only features
keeps the corpus stable across every weight refit; the recalibrator
computes predictions on demand.
Components:
* Migration `20260504210756_create_pskr_calibration_samples` —
new table + unique index on (hour, band, midpoint_lat, lon),
plus a midpoint spatial index on `pskr_spots_hourly` to keep
the join cheap.
* `Pskr.CalibrationSample` — schema mirror of the table with
the same `(hour, band, midpoint_lat, lon)` unique constraint.
* `Pskr.CalibrationSampler.build_for_hour/1` — pulls all spots
for the hour, snaps midpoints to the propagation grid (0.125°),
builds an in-memory HRRR index over a ±0.07°/±60 min window,
and bulk-upserts samples. Idempotent — re-runs upsert.
* `Workers.PskrCalibrationWorker` — Oban cron entry on the
`:backfill_enqueue` queue. Default args target the previous
full hour; explicit `hour_utc` arg reruns any hour.
* Cron `25 * * * *` — fires past HRRR analysis publish window
(~HH:50→HH:05) and PSKR aggregator's 60s flush.
Forward-only: PSKR has no historical archive, so the corpus only
grows from when the feed started. Recalibration weight refits
should wait for ~30 days / ~10k samples to cover diurnal and
synoptic variability.
Tests cover cell-snapping, HRRR feature joining, Kp stamping,
median-distance aggregation, mode dedup, idempotent reruns, and
the worker's default-hour and explicit-hour args (12 new tests,
all passing).
Backfill pipeline untouched — none of these changes feed into
contact enrichment.
|
||
|---|---|---|
| .forgejo/workflows | ||
| .github | ||
| assets | ||
| bin | ||
| config | ||
| docs | ||
| k8s | ||
| lib | ||
| lib_ml | ||
| notebooks | ||
| priv | ||
| rel | ||
| rust/prop_grid_rs | ||
| scripts | ||
| test | ||
| vendor | ||
| .credo.exs | ||
| .dockerignore | ||
| .formatter.exs | ||
| .gitignore | ||
| .tool-versions | ||
| AGENTS.md | ||
| algo.md | ||
| app.json | ||
| bugs.md | ||
| CLAUDE.md | ||
| Dockerfile | ||
| Dockerfile.base | ||
| elevation.md | ||
| LICENSE | ||
| mix.exs | ||
| mix.lock | ||
| prediction.md | ||
| README.md | ||
| tail_logs.sh | ||
| updates.md | ||
Microwaveprop
To start your Phoenix server:
- Run
mix setupto install and setup dependencies - Start Phoenix endpoint with
mix phx.serveror inside IEx withiex -S mix phx.server
Now you can visit localhost:4000 from your browser.
Ready to run in production? Please check our deployment guides.
Learn more
- Official website: https://www.phoenixframework.org/
- Guides: https://hexdocs.pm/phoenix/overview.html
- Docs: https://hexdocs.pm/phoenix
- Forum: https://elixirforum.com/c/phoenix-forum
- Source: https://github.com/phoenixframework/phoenix