graham/prop
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1704 commits 1 branch 0 tags 8.9 MiB
Commit graph

5 commits

Author SHA1 Message Date
Graham McIntire
d61fbd346e
fix(dialyzer): clear 125+ warnings under strict flags 
Enabled :error_handling, :unknown, :unmatched_returns, :extra_return,
:missing_return in an earlier commit and landed a 129-warning baseline.
Four parallel agents each fixed a directory slice:

- Core contexts (29): Radio, Release, Weather, Beacons, Cache,
  Backtest.Features, Terrain.Srtm, Ionosphere.GiroClient,
  Propagation.RunTiming, Accounts.Scope, RepoListener. Fixes were
  (a) prefix side-effect calls (Task.start, Phoenix.PubSub,
  Logger, :ets.new) with _ = ; (b) tighten/widen specs that didn't
  match actual returns; (c) add missing @type t declarations;
  (d) drop dead parse_int(nil) clause.

- Propagation + weather subdirs (15): FreshnessMonitor, NotifyListener,
  ScoreCache, ScoreCacheReconciler, Weather.FrontalAnalysis,
  Weather.Grib2.Extractor, Weather.Grib2.Wgrib2, GridCache,
  HrrrPointEnqueuer, NexradCache. Same patterns — mostly _ = on
  PubSub / :ets / Repo.insert_all; widened two specs (float ->
  number) where integer returns were reachable.

- Workers (35): BackfillEnqueue, CanadianSoundingFetch,
  ContactImport, ContactWeatherEnqueue, GefsFetch, IemreFetch,
  NarrFetch, SolarIndex, TerrainProfile, WeatherFetch. Prefixed
  Repo.update_all / Radio.set_enrichment_status! / Weather.upsert_*
  side-effect calls. Fixed one :pattern_match in
  CanadianSoundingFetch.most_recent_sounding_time/1 where a
  tautological cond guard generated unreachable code.

- Web + Mix tasks + lib_ml (46 of 50): controllers, LiveViews,
  UserAuth, and 11 mix tasks. Same prefix strategy. 4 remaining
  warnings originate in LiveTable.LiveResource dep macro expansion
  and can't be fixed without forking the dep — added .dialyzer_ignore.exs
  to suppress just those specific file:line pairs.

Also wired ignore_warnings in mix.exs dialyzer config.

mix dialyzer --format short | grep ^lib/ | wc -l -> 0
mix test: 2163 tests, 3 pre-existing flakes, 0 regressions.
 2026-04-21 10:30:06 -05:00
Graham McIntire
e7a7ae073d Phase 9.3, 9.4, and Phase 3 NEXRAD pipeline 
Task 9.3 - Weight recalibration via gradient descent:
- Recalibrator module fits logistic regression weights using Nx
- Trains on QSO positives vs random baseline negatives
- Cross-validates by month, normalizes weights to sum to 1.0
- Mix task: mix recalibrate_scorer --sample 5000 --epochs 2000

Task 9.4 - Side-by-side scorer comparison:
- ScorerDiff.compare/3 re-scores grid with old vs new weights
- Reports mean diff, regressions, improvements, per-band breakdown
- Mix task: mix scorer_diff --new-weights '{...}'

Phase 3 - NEXRAD ingestion pipeline:
- NexradClient fetches IEM n0q composite PNGs, extracts per-point
  box statistics (mean/max dBZ, texture variance)
- NexradObservation schema with unique (lat, lon, observed_at)
- NexradWorker on :nexrad queue for background processing
- nexrad_texture backtest feature in Features module
- mix nexrad_backfill --limit 200

All tasks added to AdminTaskWorker and Release for production use.
1116 tests, 0 failures.
 2026-04-10 12:48:36 -05:00
Graham McIntire
6d92973853 Phase 9.1: Consolidated backtest report and contest log import 
- Add `mix backtest --all` for consolidated pass/fail table across all features
- Add Backtest.consolidated_report/2 and to_consolidated_markdown/1
- Add Features.all_features/0 to auto-discover backtestable features
- Add `mix import_contest_logs` for bulk ARRL contest CSV import with dedup
- Fix hrrr_climatology to batch by (month, hour) to avoid query timeout
- Fix Repo.query! result pattern (Postgrex.Result, not tuple)
- Backtest reports for all Phase 1-6 features
 2026-04-10 11:57:15 -05:00
Graham McIntire
65e3159a85 Pause Oban queues in all mix tasks 
Mix tasks that call app.start were also booting Oban's cron scheduler,
causing PropagationGridWorker and other cron jobs to fire during
backfills. Add Oban.pause_all_queues(Oban) immediately after app.start
in every mix task that only needs Repo access.
 2026-04-10 09:13:30 -05:00
Graham McIntire
0f0e5e8d43 Phase 3 spike (parked) + Phase 6: temperature anomaly 
Phase 3 NEXRAD spike: IEM n0q composite available at 5-min cadence
back to 2022+. Compression-ratio proxy shows afternoon images have
13-81% more texture than dawn (directionally correct), but the n0q
product thresholds out the faint clear-air returns needed for BL
stability detection. Parked until MRMS or Level III products can be
investigated. See docs/research/nexrad_spike.md.

Phase 6: hrrr_climatology table aggregating surface_temp_c by
(lat, lon, month, hour) from the 42M+ hrrr_profiles grid-point
rows. mix hrrr_climatology builds it via a single SQL GROUP BY +
upsert. Backtest.Features.temperature_anomaly computes current_temp
minus climatological mean — the meteorologist's "temperature
deviation above normal" predictor for summer afternoon enhancement.
 2026-04-10 08:47:11 -05:00