Add SFI, Kp max (solar), K-index, lifted index (sounding stability),
and ducting_detected (HRRR) as model features. Training now joins to
solar_indices and nearest sounding (within 6 hours) for both phases.
Model can learn solar/geomagnetic effects if they exist in the data.
- 15 features: add surface_refractivity and latitude
- Bigger network: 128→64→32 (3 hidden layers)
- Phase 1: pretrain on 500K stratified algorithm scores (all seasons/locations)
- Phase 2: fine-tune on 57K real QSO-HRRR matched data (percentile target)
- Lower LR (0.0003) for fine-tuning to preserve pretrained knowledge
- Model.train accepts :initial_state option for transfer learning
- 3 hidden layers instead of 2 for better feature interaction learning
- Target is within-band distance percentile (0-1) instead of raw
normalized distance — reduces noise from operator/equipment variation
Features: HRRR conditions averaged at both QSO endpoints + solar time
Target: distance_km normalized per-band (distance / p99_range, capped at 1.0)
This trains on actual propagation outcomes from 57K+ QSOs, not the
hand-tuned algorithm output.
Raw features had vastly different scales (pressure ~1013, sin/cos ~[-1,1])
causing gradient explosion. Normalize all atmospheric features to ~[0,1]
using known physical bounds. Add Polaris dep for optimizer.
Solar time (longitude/15) replaces fixed CDT/CST offset for time-of-day
scoring. Correlation analysis shows dramatic improvement at higher
frequencies: 24 GHz rho jumps from 0.056 (UTC) to 0.188 (solar), and
75 GHz corrects from spurious -0.39 to physically correct +0.24.
Score time-of-day per grid point using longitude/15 solar offset instead of
hardcoded CST/CDT. Add PWAT as 10th scoring factor. Refine pressure thresholds.
Update ML model and training pipeline to use local solar time.
- HrrrClient.hrrr_url accepts forecast_hour param (wrfsfcfHH.grib2)
- PropagationGridWorker fetches all 19 forecast hours per run
- Propagation.scores_at/3 queries scores at specific valid_time
- Propagation.available_valid_times/1 returns all forecast times for timeline
- Pruning keeps scores with valid_time >= now - 2h (forecast-aware)
- MapLive: select_time event, timeline data pushed to JS
- JS: forecast timeline bar at bottom of map with clickable hour buttons
- PubSub broadcast sends list of valid_times instead of single time
Enqueue worker now gathers atmospheric data at pos1, midpoint, and pos2
along each QSO path instead of only pos1. Existing has_* guards prevent
duplicate fetches at each grid point.
- Add Radio.qso_path_points/1 for path point extraction
- Update hrrr_job_for_qso, iemre_job_for_qso, jobs_for_qso to iterate path points
- Refactor Weather into find_nearest_hrrr/3 and find_nearest_iemre/3
- Add hrrr_profiles_for_path/1 and iemre_for_path/1 query functions
- Add mix reset_enrichment task to trigger re-processing
