Commit graph

8 commits

Author SHA1 Message Date
9db2cd20f5
fix: resolve 7 pre-existing test failures and ScoreCache DateTime bug
- CacheTest: fix sweep timing race by using negative TTL (-1)
  instead of positive TTL (1) for already-expired entries
- ScoreCache: replace ETS match-spec DateTime comparisons with
  :ets.foldl + DateTime.compare — DateTime structs are maps in
  Elixir >= 1.15 and ETS can't compare maps with :< / :> guards
- Accounts: drop unsupported returning: true on delete_all,
  return [] for expired tokens list
- Backtest: catch ArgumentError from String.to_existing_atom
  for unknown feature names, preserving the helpful Mix.Error
- ContactLive IndexTest: invalidate monthly_bars cache before
  assertion so test data is visible
- RoverLocationsLive MapTest: invalidate cached points before
  assertion
- StatusLiveTest: add DB cleanup in setup to reduce test
  interference; parameterize NARR candidate coordinates
2026-06-10 13:50:24 -05:00
316fb2fbc7
Fix low-severity bugs and re-enable Credo checks
- Bug #12: Lower rate limit to 15/min
- Bug #13: Wrap model loading in Task.start
- Bug #14: Fix classify_time_period guard gap at -3.0
- Bug #15: Not applicable (Elixir has no ?? operator)
- Bug #16: Remove fallback Repo.get for station preload
- Bug #17: Extract cache_key() in ContactMapController
- Bug #18: Add has_many :contacts and :beacons to User schema
- A&D #4: Move serve_markdown_if_requested after secure headers
- Config #1: Move signing_salt to runtime.exs env var
- Config #2: Use --check-unused instead of --unused
- Config #3: Re-enable UnsafeToAtom Credo check
- Config #5: Re-enable LeakyEnvironment Credo check
- Add @spec annotations to fix re-enabled Specs violations
- Replace String.to_atom with to_existing_atom where guarded
2026-05-29 17:29:22 -05:00
d61fbd346e
fix(dialyzer): clear 125+ warnings under strict flags
Enabled :error_handling, :unknown, :unmatched_returns, :extra_return,
:missing_return in an earlier commit and landed a 129-warning baseline.
Four parallel agents each fixed a directory slice:

- Core contexts (29): Radio, Release, Weather, Beacons, Cache,
  Backtest.Features, Terrain.Srtm, Ionosphere.GiroClient,
  Propagation.RunTiming, Accounts.Scope, RepoListener. Fixes were
  (a) prefix side-effect calls (Task.start, Phoenix.PubSub,
  Logger, :ets.new) with _ = ; (b) tighten/widen specs that didn't
  match actual returns; (c) add missing @type t declarations;
  (d) drop dead parse_int(nil) clause.

- Propagation + weather subdirs (15): FreshnessMonitor, NotifyListener,
  ScoreCache, ScoreCacheReconciler, Weather.FrontalAnalysis,
  Weather.Grib2.Extractor, Weather.Grib2.Wgrib2, GridCache,
  HrrrPointEnqueuer, NexradCache. Same patterns — mostly _ = on
  PubSub / :ets / Repo.insert_all; widened two specs (float ->
  number) where integer returns were reachable.

- Workers (35): BackfillEnqueue, CanadianSoundingFetch,
  ContactImport, ContactWeatherEnqueue, GefsFetch, IemreFetch,
  NarrFetch, SolarIndex, TerrainProfile, WeatherFetch. Prefixed
  Repo.update_all / Radio.set_enrichment_status! / Weather.upsert_*
  side-effect calls. Fixed one :pattern_match in
  CanadianSoundingFetch.most_recent_sounding_time/1 where a
  tautological cond guard generated unreachable code.

- Web + Mix tasks + lib_ml (46 of 50): controllers, LiveViews,
  UserAuth, and 11 mix tasks. Same prefix strategy. 4 remaining
  warnings originate in LiveTable.LiveResource dep macro expansion
  and can't be fixed without forking the dep — added .dialyzer_ignore.exs
  to suppress just those specific file:line pairs.

Also wired ignore_warnings in mix.exs dialyzer config.

mix dialyzer --format short | grep ^lib/ | wc -l -> 0
mix test: 2163 tests, 3 pre-existing flakes, 0 regressions.
2026-04-21 10:30:06 -05:00
e7a7ae073d Phase 9.3, 9.4, and Phase 3 NEXRAD pipeline
Task 9.3 - Weight recalibration via gradient descent:
- Recalibrator module fits logistic regression weights using Nx
- Trains on QSO positives vs random baseline negatives
- Cross-validates by month, normalizes weights to sum to 1.0
- Mix task: mix recalibrate_scorer --sample 5000 --epochs 2000

Task 9.4 - Side-by-side scorer comparison:
- ScorerDiff.compare/3 re-scores grid with old vs new weights
- Reports mean diff, regressions, improvements, per-band breakdown
- Mix task: mix scorer_diff --new-weights '{...}'

Phase 3 - NEXRAD ingestion pipeline:
- NexradClient fetches IEM n0q composite PNGs, extracts per-point
  box statistics (mean/max dBZ, texture variance)
- NexradObservation schema with unique (lat, lon, observed_at)
- NexradWorker on :nexrad queue for background processing
- nexrad_texture backtest feature in Features module
- mix nexrad_backfill --limit 200

All tasks added to AdminTaskWorker and Release for production use.
1116 tests, 0 failures.
2026-04-10 12:48:36 -05:00
6d92973853 Phase 9.1: Consolidated backtest report and contest log import
- Add `mix backtest --all` for consolidated pass/fail table across all features
- Add Backtest.consolidated_report/2 and to_consolidated_markdown/1
- Add Features.all_features/0 to auto-discover backtestable features
- Add `mix import_contest_logs` for bulk ARRL contest CSV import with dedup
- Fix hrrr_climatology to batch by (month, hour) to avoid query timeout
- Fix Repo.query! result pattern (Postgrex.Result, not tuple)
- Backtest reports for all Phase 1-6 features
2026-04-10 11:57:15 -05:00
65e3159a85 Pause Oban queues in all mix tasks
Mix tasks that call app.start were also booting Oban's cron scheduler,
causing PropagationGridWorker and other cron jobs to fire during
backfills. Add Oban.pause_all_queues(Oban) immediately after app.start
in every mix task that only needs Repo access.
2026-04-10 09:13:30 -05:00
7630934bcb Accept CamelCase feature names in mix backtest
mix backtest --feature NaiveGradient now resolves to the same function
as --feature naive_gradient. Names are normalized via Macro.underscore,
so any casing works. Also prints the available functions when an
unknown feature is requested so typos don't produce an opaque
UndefinedFunctionError deep in the Enum.map stack.
2026-04-09 16:15:01 -05:00
ab04cb9168 Phase 0: backtest harness
Add Microwaveprop.Backtest: a feature-evaluation framework that runs
a (lat, lon, valid_time) -> float function over the historical QSO
corpus and a matched random-time baseline, reporting distribution
statistics, distance-binned lift, and band-stratified lift.

Adds four baseline feature wrappers around the current scorer inputs
(naive_gradient, td_depression, time_of_day, pressure), a mix backtest
CLI, and the first set of baseline reports under priv/backtest_reports
so downstream phases have a frozen reference point to compare against.
2026-04-09 16:10:54 -05:00