Commit graph

9 commits

Author SHA1 Message Date
Graham McInitre
b65d3227cd feat: clickable band filters on PSK Reporter tab, fix all test & credo issues
PSK Reporter tab:
- Fix handle_info(:refresh_spots) to use start_async instead of blocking
- Add require Logger to fix compiler warning
- Make band counts clickable: clicking a band filters table to that band's
  last 100 spots; clicking again clears the filter
- Highlight active filter badge, show Clear filter link when filtered
- Update subtitle and empty state dynamically based on filter
- Filter persists across auto-refresh (60s)
- Fix empty state condition to render immediately without async guard

Credo fixes (mix credo --strict now passes):
- weather_map_component.ex: replace @doc false with @impl true (missing spec)
- contact_weather_enqueue_worker.ex: extract hrrr_placeholder_for_contact/3
  to reduce nesting depth
- profile_lookup.ex: replace MapSet+then+reject pattern with Enum.filter
  to fix both nesting depth and cyclomatic complexity

Test infrastructure fixes:
- Fix Release.migrate/0 to handle repos without migration directories
  (AprsRepo in test has no migrations)
- Fix insert_spot helper: add missing 19th param for inserted_at/updated_at
- Fix String.index/2 -> :binary.match for Elixir 1.20 compat
- Fix DateTime.add!/3 -> DateTime.add/3 for Elixir 1.20 compat
- Fix substring matching in limit test (GRID1 matched GRID10)
- Add Process.sleep+render calls for async band_counts in new tests
- Replace impossible 'empty state for filtered band' test with
  'switching band filter updates table' test
- All 19 tests pass, mix credo --strict clean
2026-07-20 13:01:00 -05:00
7a7b30f7bf
chore(specs): add @spec to public API in accounts/release/backtest 2026-04-21 13:32:42 -05:00
d61fbd346e
fix(dialyzer): clear 125+ warnings under strict flags
Enabled :error_handling, :unknown, :unmatched_returns, :extra_return,
:missing_return in an earlier commit and landed a 129-warning baseline.
Four parallel agents each fixed a directory slice:

- Core contexts (29): Radio, Release, Weather, Beacons, Cache,
  Backtest.Features, Terrain.Srtm, Ionosphere.GiroClient,
  Propagation.RunTiming, Accounts.Scope, RepoListener. Fixes were
  (a) prefix side-effect calls (Task.start, Phoenix.PubSub,
  Logger, :ets.new) with _ = ; (b) tighten/widen specs that didn't
  match actual returns; (c) add missing @type t declarations;
  (d) drop dead parse_int(nil) clause.

- Propagation + weather subdirs (15): FreshnessMonitor, NotifyListener,
  ScoreCache, ScoreCacheReconciler, Weather.FrontalAnalysis,
  Weather.Grib2.Extractor, Weather.Grib2.Wgrib2, GridCache,
  HrrrPointEnqueuer, NexradCache. Same patterns — mostly _ = on
  PubSub / :ets / Repo.insert_all; widened two specs (float ->
  number) where integer returns were reachable.

- Workers (35): BackfillEnqueue, CanadianSoundingFetch,
  ContactImport, ContactWeatherEnqueue, GefsFetch, IemreFetch,
  NarrFetch, SolarIndex, TerrainProfile, WeatherFetch. Prefixed
  Repo.update_all / Radio.set_enrichment_status! / Weather.upsert_*
  side-effect calls. Fixed one :pattern_match in
  CanadianSoundingFetch.most_recent_sounding_time/1 where a
  tautological cond guard generated unreachable code.

- Web + Mix tasks + lib_ml (46 of 50): controllers, LiveViews,
  UserAuth, and 11 mix tasks. Same prefix strategy. 4 remaining
  warnings originate in LiveTable.LiveResource dep macro expansion
  and can't be fixed without forking the dep — added .dialyzer_ignore.exs
  to suppress just those specific file:line pairs.

Also wired ignore_warnings in mix.exs dialyzer config.

mix dialyzer --format short | grep ^lib/ | wc -l -> 0
mix test: 2163 tests, 3 pre-existing flakes, 0 regressions.
2026-04-21 10:30:06 -05:00
811251dd15
Drop propagation_scores table, ScoresFile is the only store
Full cutover: the propagation_scores Postgres table is gone, and
the binary files under /data/scores are the sole source of truth
for the map render path. Three stacked changes:

1. New migration drops the propagation_scores table and its indexes
   (the earlier tuning migrations for it were already applied and
   are now no-ops against a missing table, which is fine — Ecto
   just runs them on fresh environments).

2. Propagation context is gutted of every GridScore reference.
   replace_scores/2 writes files only. upsert_scores/2 is deleted.
   load_scores_from_db, available_valid_times_from_db,
   point_detail_from_db, point_forecast_from_db, fetch_factors,
   coalesce_factors, the Postgres side of prune_old_scores, and
   the Postgres fallbacks in latest/earliest_valid_time are all
   removed. point_detail always returns an empty factors map now
   since factor breakdowns were retired with the table.

3. Deleted modules:
   - Propagation.GridScore (the schema)
   - Propagation.ScorerDiff (read factors from the table)
   - Propagation.AsosNudge (helper for AsosAdjustmentWorker)
   - Workers.AsosAdjustmentWorker (its cron was already disabled)
   - Mix.Tasks.ScorerDiff (wrapper around the deleted module)
   And their tests. AdminTaskWorker's scorer_diff task is a
   logging no-op so any queued Oban rows drain cleanly.
   Release.scorer_diff stays as a stub that tells the operator.

propagation_prune_worker_test rewritten to exercise ScoresFile
pruning. propagation_test.exs rewritten to use replace_scores +
ScoresFile throughout (no more GridScore / upsert_scores paths).
2026-04-14 15:13:01 -05:00
7fb340bc35 Fix all remaining credo --strict issues (0 issues)
Aliases: add module aliases for 9 nested module references
Apply: replace apply/3 with direct module attribute calls
Line length: break 1 long spec line
Refactoring: extract helpers to reduce complexity and nesting
in show.ex, radio.ex, weather workers, terrain, duct detection,
backfill dashboard, contact map, and mix tasks
2026-04-12 10:26:53 -05:00
e7a7ae073d Phase 9.3, 9.4, and Phase 3 NEXRAD pipeline
Task 9.3 - Weight recalibration via gradient descent:
- Recalibrator module fits logistic regression weights using Nx
- Trains on QSO positives vs random baseline negatives
- Cross-validates by month, normalizes weights to sum to 1.0
- Mix task: mix recalibrate_scorer --sample 5000 --epochs 2000

Task 9.4 - Side-by-side scorer comparison:
- ScorerDiff.compare/3 re-scores grid with old vs new weights
- Reports mean diff, regressions, improvements, per-band breakdown
- Mix task: mix scorer_diff --new-weights '{...}'

Phase 3 - NEXRAD ingestion pipeline:
- NexradClient fetches IEM n0q composite PNGs, extracts per-point
  box statistics (mean/max dBZ, texture variance)
- NexradObservation schema with unique (lat, lon, observed_at)
- NexradWorker on :nexrad queue for background processing
- nexrad_texture backtest feature in Features module
- mix nexrad_backfill --limit 200

All tasks added to AdminTaskWorker and Release for production use.
1116 tests, 0 failures.
2026-04-10 12:48:36 -05:00
01909dbe66 Run admin tasks as Oban jobs instead of blocking eval
Release.backtest_all, climatology, native_derive now enqueue an
AdminTaskWorker job on the new :admin queue and return immediately.
Progress visible in Oban Web at /admin/oban.
2026-04-10 12:26:01 -05:00
d5842cb45c Add release tasks for backtest, climatology, and native backfill
All mix tasks now have equivalents in Microwaveprop.Release for
production use via `bin/microwaveprop eval`.
2026-04-10 12:14:41 -05:00
e0ab6560d0
docker 2026-03-28 11:41:31 -05:00