Graham McIntire
ab04cb9168
Add Microwaveprop.Backtest: a feature-evaluation framework that runs a (lat, lon, valid_time) -> float function over the historical QSO corpus and a matched random-time baseline, reporting distribution statistics, distance-binned lift, and band-stratified lift. Adds four baseline feature wrappers around the current scorer inputs (naive_gradient, td_depression, time_of_day, pressure), a mix backtest CLI, and the first set of baseline reports under priv/backtest_reports so downstream phases have a frozen reference point to compare against.
31 lines
1.2 KiB
Markdown
31 lines
1.2 KiB
Markdown
# Backtest: Microwaveprop.Backtest.Features.td_depression
|
|
|
|
QSO sample: 5000
|
|
Baseline sample: 5000
|
|
|
|
## Matched distribution
|
|
|
|
| Set | N | Mean | Stddev | p50 | p90 | Min | Max |
|
|
|---|---|---|---|---|---|---|---|
|
|
| QSO times | 5000 | 6.569 | 5.365 | 5.573 | 12.969 | -0.693 | 26.814 |
|
|
| Random baseline | 161 | 4.719 | 4.367 | 3.534 | 10.096 | -1.258 | 21.762 |
|
|
|
|
## Lift by distance (km)
|
|
|
|
| Set | N | Mean | Stddev | p50 | p90 | Min | Max |
|
|
|---|---|---|---|---|---|---|---|
|
|
| 0-100 | 1249 | 5.324 | 4.270 | 4.151 | 10.907 | -0.693 | 25.733 |
|
|
| 100-250 | 2295 | 6.941 | 5.789 | 5.589 | 14.564 | -0.630 | 26.814 |
|
|
| 250-500 | 1288 | 7.007 | 5.510 | 6.054 | 14.450 | -0.568 | 25.733 |
|
|
| 500-1000 | 167 | 7.390 | 3.911 | 7.422 | 12.637 | -0.142 | 15.938 |
|
|
| 1000+ | 1 | 7.722 | 0.000 | 7.722 | 7.722 | 7.722 | 7.722 |
|
|
|
|
## Lift by band
|
|
|
|
| Set | N | Mean | Stddev | p50 | p90 | Min | Max |
|
|
|---|---|---|---|---|---|---|---|
|
|
| 10000 MHz | 4444 | 6.578 | 5.300 | 5.573 | 12.879 | -0.693 | 26.814 |
|
|
| 24000 MHz | 465 | 6.431 | 5.895 | 5.127 | 14.450 | -0.693 | 26.814 |
|
|
| 47000 MHz | 77 | 7.232 | 6.062 | 5.804 | 16.799 | 0.355 | 25.762 |
|
|
| 75000 MHz | 14 | 4.824 | 2.365 | 3.339 | 7.317 | 2.483 | 7.317 |
|
|
|