Graham McIntire
ab04cb9168
Add Microwaveprop.Backtest: a feature-evaluation framework that runs a (lat, lon, valid_time) -> float function over the historical QSO corpus and a matched random-time baseline, reporting distribution statistics, distance-binned lift, and band-stratified lift. Adds four baseline feature wrappers around the current scorer inputs (naive_gradient, td_depression, time_of_day, pressure), a mix backtest CLI, and the first set of baseline reports under priv/backtest_reports so downstream phases have a frozen reference point to compare against.
31 lines
1.3 KiB
Markdown
31 lines
1.3 KiB
Markdown
# Backtest: Microwaveprop.Backtest.Features.naive_gradient
|
|
|
|
QSO sample: 5000
|
|
Baseline sample: 5000
|
|
|
|
## Matched distribution
|
|
|
|
| Set | N | Mean | Stddev | p50 | p90 | Min | Max |
|
|
|---|---|---|---|---|---|---|---|
|
|
| QSO times | 5000 | -113.971 | 42.207 | -105.154 | -69.043 | -331.397 | -35.521 |
|
|
| Random baseline | 151 | -118.002 | 43.405 | -107.528 | -72.860 | -268.079 | -41.047 |
|
|
|
|
## Lift by distance (km)
|
|
|
|
| Set | N | Mean | Stddev | p50 | p90 | Min | Max |
|
|
|---|---|---|---|---|---|---|---|
|
|
| 0-100 | 1249 | -114.766 | 43.617 | -107.614 | -66.251 | -321.312 | -35.521 |
|
|
| 100-250 | 2295 | -114.555 | 42.595 | -103.392 | -71.537 | -331.397 | -44.027 |
|
|
| 250-500 | 1288 | -112.728 | 41.507 | -105.238 | -65.613 | -321.312 | -39.224 |
|
|
| 500-1000 | 167 | -109.723 | 29.535 | -115.466 | -69.348 | -206.669 | -45.098 |
|
|
| 1000+ | 1 | -90.610 | 0.000 | -90.610 | -90.610 | -90.610 | -90.610 |
|
|
|
|
## Lift by band
|
|
|
|
| Set | N | Mean | Stddev | p50 | p90 | Min | Max |
|
|
|---|---|---|---|---|---|---|---|
|
|
| 10000 MHz | 4444 | -114.037 | 42.734 | -105.238 | -68.535 | -321.312 | -37.154 |
|
|
| 24000 MHz | 465 | -116.766 | 37.728 | -107.614 | -82.149 | -331.397 | -64.361 |
|
|
| 47000 MHz | 77 | -98.886 | 35.255 | -93.461 | -68.535 | -227.800 | -35.521 |
|
|
| 75000 MHz | 14 | -83.273 | 13.340 | -93.461 | -68.535 | -99.652 | -68.535 |
|
|
|