graham/prop - Forgejo: Beyond coding. We forge.

1986 commits 2 branches 0 tags 10 MiB

Author	SHA1	Message	Date
Graham McIntire	2fd88a94ea	fix(propagation): 7 pipeline bugs + validation harnesses + model improvements Some checks failed Build and Push / Build and Push Docker Image (push) Waiting to run Details Build prop-grid-rs / Test, build, push (push) Failing after 3m41s Details Phase A — 7 pipeline bugs: - retain_scores_window: fix timeline self-deletion (NOTIFY payload run_time\|valid_time) - Rust/Elixir weight divergence: Rust loads band_weights.json at startup - Aurora boost ported to Rust (Kp query + per-cell boost for bands <= 432 MHz) - Commercial-link boost applied in Rust per-cell scoring - f00 native gradient preferred over pressure-level gradient - HRDPS files: greedy regex fixed, now visible to timeline/prune/retain - GEFS/HRRR collision: GEFS namespace as .gefs.prop, merge at read Phase B — Validation harness: - scripts/validate_algo.py: out-of-sample Spearman rho(score,distance) + baselines - docs/algo-reports/validation-2026-08-01.{json,md} Phase C — Forecast-skill evaluation: - scripts/validate_forecast.py: skill degradation by lead time (0h-24h) - docs/algo-reports/forecast-2026-08-01.{json,md} Phase D — Calibration + model improvements: - recalibrate.py: validation gate before weight deploy - Recalibrator: Nx.max(0.0) replaces Nx.abs(), L2 regularization, val-set integrity - ML train/serve defaults unified; prop_compare null-handling skew fixed - Latitude-aware sunrise: solar-declination sunrise_hour(lat,month) (Elixir + Rust) - Path scoring: wind/sky/rain from HRRR (was ~30% of composite weight silent) - Region multiplier documented as unvalidated; PWAT/refractivity doc-vs-code noted - algo.md: synced scoring sections, marked retired features, D5/D6 changelog - Credo: path_compute cyclomatic complexity bumped (6->10 fields) — informational only	2026-08-01 19:28:41 -05:00
Graham McIntire	a6b89961f6	feat(ml): retrain propagation model and add per-prediction explainability Retrained on 60k month-balanced HRRR profiles through 2026-04-05 (test RMSE 1.7 score points, R² 0.97). Added explain_prediction/4 returning ranked feature contributions via batched finite-difference attribution, so the UI can show users which weather factors drove each prediction.	2026-04-28 14:14:22 -05:00
Graham McIntire	2c16fd979d	Move ML deps to dev/test only, exclude from production build - Nx, Axon, EXLA, Polaris deps restricted to only: [:dev, :test] - model.ex and training mix tasks moved to lib_ml/ (compiled via elixirc_paths in dev/test only) - load_ml_model uses Code.ensure_loaded? + apply/3 to avoid compile-time references to ML modules in production - Verified: MIX_ENV=prod compiles clean with no ML warnings	2026-04-01 12:10:31 -05:00

Author

SHA1

Message

Date

Graham McIntire

2fd88a94ea

fix(propagation): 7 pipeline bugs + validation harnesses + model improvements

Build and Push / Build and Push Docker Image (push) Waiting to run

Details

Build prop-grid-rs / Test, build, push (push) Failing after 3m41s

Details

Phase A — 7 pipeline bugs:
- retain_scores_window: fix timeline self-deletion (NOTIFY payload run_time|valid_time)
- Rust/Elixir weight divergence: Rust loads band_weights.json at startup
- Aurora boost ported to Rust (Kp query + per-cell boost for bands <= 432 MHz)
- Commercial-link boost applied in Rust per-cell scoring
- f00 native gradient preferred over pressure-level gradient
- HRDPS files: greedy regex fixed, now visible to timeline/prune/retain
- GEFS/HRRR collision: GEFS namespace as .gefs.prop, merge at read

Phase B — Validation harness:
- scripts/validate_algo.py: out-of-sample Spearman rho(score,distance) + baselines
- docs/algo-reports/validation-2026-08-01.{json,md}

Phase C — Forecast-skill evaluation:
- scripts/validate_forecast.py: skill degradation by lead time (0h-24h)
- docs/algo-reports/forecast-2026-08-01.{json,md}

Phase D — Calibration + model improvements:
- recalibrate.py: validation gate before weight deploy
- Recalibrator: Nx.max(0.0) replaces Nx.abs(), L2 regularization, val-set integrity
- ML train/serve defaults unified; prop_compare null-handling skew fixed
- Latitude-aware sunrise: solar-declination sunrise_hour(lat,month) (Elixir + Rust)
- Path scoring: wind/sky/rain from HRRR (was ~30% of composite weight silent)
- Region multiplier documented as unvalidated; PWAT/refractivity doc-vs-code noted
- algo.md: synced scoring sections, marked retired features, D5/D6 changelog
- Credo: path_compute cyclomatic complexity bumped (6->10 fields) — informational only

2026-08-01 19:28:41 -05:00

Graham McIntire

a6b89961f6

feat(ml): retrain propagation model and add per-prediction explainability

Retrained on 60k month-balanced HRRR profiles through 2026-04-05
(test RMSE 1.7 score points, R² 0.97). Added explain_prediction/4
returning ranked feature contributions via batched finite-difference
attribution, so the UI can show users which weather factors drove
each prediction.

2026-04-28 14:14:22 -05:00

Graham McIntire

2c16fd979d

Move ML deps to dev/test only, exclude from production build

- Nx, Axon, EXLA, Polaris deps restricted to only: [:dev, :test]
- model.ex and training mix tasks moved to lib_ml/ (compiled via
  elixirc_paths in dev/test only)
- load_ml_model uses Code.ensure_loaded? + apply/3 to avoid
  compile-time references to ML modules in production
- Verified: MIX_ENV=prod compiles clean with no ML warnings

2026-04-01 12:10:31 -05:00

Renamed from lib/microwaveprop/propagation/model.ex (Browse further)

3 commits