Commit graph

1929 commits

Author SHA1 Message Date
Graham McInitre
3297147c31 feat: add total spot count and per-band breakdown to /pskreporter header 2026-07-15 14:32:55 -05:00
Graham McInitre
51dda456aa feat: track sender/receiver callsigns in pskr spots + full-width /pskreporter page 2026-07-15 14:29:23 -05:00
Graham McInitre
a2e2776e7c fix: batch pskr calibration upsert into 1,000-row chunks to avoid Postgres 65,535 param limit 2026-07-15 14:18:43 -05:00
Graham McInitre
38ee285833 fix: add limit assign so @limit works in pskr_spots template 2026-07-15 14:15:51 -05:00
Graham McInitre
5698d32865 fix: revert Docker builder to Elixir 1.20.1 / OTP 29.0.2 2026-07-15 13:58:06 -05:00
Graham McInitre
986efb2345 Revert "fix: bump Docker debian date suffix from 20260518 to 20260713"
This reverts commit ee2d94ce57.
2026-07-15 13:57:04 -05:00
Graham McInitre
ee2d94ce57 fix: bump Docker debian date suffix from 20260518 to 20260713 2026-07-15 13:36:34 -05:00
Graham McInitre
d76c36d25f feat: add /pskreporter page showing last 100 PSK Reporter spots 2026-07-15 13:30:34 -05:00
Graham McInitre
a8ecee32b0 feat: add Nix flake dev environment with Erlang 29.0.3 + Elixir 1.20.2
- Add flake.nix + flake.lock pinning nixpkgs-unstable
- Add nix/shell.nix with PostgreSQL, Rust, Elixir LSP, pre-commit hooks
- Add shell.nix flake-compat shim for non-flake nix-shell users
- Bump Dockerfile to Elixir 1.20.2 / OTP 29.0.3
- Add nix.md with setup and troubleshooting docs
- Gitignore Nix runtime state and generated pre-commit config

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-15 12:57:10 -05:00
aa6c885683
fix: repair 18 failing tests (atom naming + stale assertions)
- Replace Module.safe_concat/1 with PID-based names in IemRateLimiterTest
  and AggregatorTest (8+6 tests) to avoid binary_to_existing_atom errors
  after Elixir 1.20 upgrade
- Fix ContactLive ShowCoverageTest assertions (3 tests) that referenced
  removed template text ('in queue') and incompatible weather_status
- Use conn.remote_ip instead of init_test_session for internal_network?
  tests since the store_remote_ip plug uses atom keys
2026-07-08 11:10:29 -05:00
ecb425780e
chore: bump deps (phoenix 1.8.9, live_view 1.2.6, swoosh 1.26.3, redix 1.6.0, websock_adapter 0.6.0) 2026-07-08 10:51:45 -05:00
5f154a20f3
refactor: apply functional programming patterns to accounts + map_live
- accounts.ex: replace if with with-chain in get_user_by_email_and_password
  for explicit pattern matching on User struct + valid_password? result.
  Collapse nested case into with in create_api_token.

- map_live.ex: extract connected? PubSub subscription block from mount into
  named subscribe_if_connected/1 function (cleaner than inline if with _ =).
  Deduplicate identical select_time + set_selected_time handlers into shared
  handle_set_time/2 helper. Extract toggle_grid + toggle_radar into shared
  handle_toggle/3 helper (functions-as-values pattern for assign key).
2026-07-01 17:56:06 -05:00
c2efead65c
chore: bump deps, fix doctests, fix credo issues 2026-07-01 17:46:03 -05:00
00f6d8248b
update elixir 2026-07-01 15:46:51 -05:00
0d00d1777c
fix: 9 bugs across tests and production code
- Fix Module.safe_concat -> Module.concat in tests with dynamic process names
  (safe_concat calls binary_to_existing_atom, but test names are newly generated)
- Fix PSKR AggregatorTest sandbox ownership by switching to async: false
- Fix MapLiveTest assert_patch: regex unsupported in LiveView 1.2, use string match
- Fix WeatherMapLiveTest toggle_grid: assert on checked attribute, not data-grid
- Fix BeaconLive.Index leaking unapproved beacons: wire data_provider to approved_beacons_query
- Fix ContactLive.Index leaking private contacts: wire data_provider with visibility filter
- Fix RecalibratorTest: train() expects factor vectors, not {vector, datetime, band} tuples
- Fix toggle_sort: compare current_field to new_field, not current_order
- Fix internal_network?: handle both atom and string session keys from Plug sessions

Test results: 3979/4001 -> 3997/4001 (18 previously-failing tests now pass)
2026-06-21 12:13:58 -05:00
6bd4361ed1
updates 2026-06-16 12:38:08 -05:00
49108423d6
cleanup 2026-06-13 14:14:18 -05:00
acfdca35df
update credo 2026-06-12 16:14:14 -05:00
cb8445f329
fix: resolve 158/183 credo --strict warnings
- Add 17 missing @spec annotations (layouts, error_json, error_html, skewt_svg)
- Move 12+ nested alias/import/require to module top level
- Add phx-change/id attributes to 11 raw HTML <form> tags
- Remove 4 unused LiveView assigns (:bounds, :data_provider)
- Add 3 missing doctest references (HrrrNativeClient, BulkFetch, Accounts)
- Break 2 long lines (path_compute.ex:382)
- Strengthen weak test assertions (is_binary→byte_size, is_list→!=[])
- Replace Module.concat with Module.safe_concat (2 occurrences)
- Replace length/1 > 0 with list != [] (9 occurrences)
- Remove no-op assert true, fix no-assertion tests

Remaining: 24 socket.assigns introspection warnings (deliberate test
pattern for observable behavior testing), 1 formatter-resistant long
line, 3 app-code usage warnings.
2026-06-12 15:47:15 -05:00
4a2f259f49
fix: additional @spec and test assertion fixes from agents 2026-06-12 13:53:25 -05:00
fd976b0cd5
fix: resolve 391 Credo issues across codebase
- Add jump_credo_checks ~> 0.4 with all 20 checks enabled
- Fix all standard Credo issues: 139 @spec (113 done, 26 remain),
  4 refactoring, 3 alias usage, 9 System.cmd env, 5 unsafe_to_atom,
  2 max line length, 9 assert_receive timeout
- Fix 170+ jump_credo_checks warnings:
  - 117 TopLevelAliasImportRequire: move nested alias/import to module top
  - 32 UseObanProWorker: switch to Oban.Pro.Worker
  - 4 DoctestIExExamples: add doctests / create test file
  - ~20 WeakAssertion: strengthen type-check assertions
  - Various ConditionalAssertion, AssertReceiveTimeout fixes
- Exclude vendor/ from Credo analysis
- Remaining: 175 warnings (mostly opinionated WeakAssertion,
  AvoidSocketAssignsInTest), 26 @spec annotations
2026-06-12 13:51:32 -05:00
e879f291f7
chore: update phoenix_live_view to 1.2.0 and fix test failures
Dependency updates:
- phoenix_live_view: ~> 1.2.0 (was ~> 1.1.0)
- Transitive bumps: bandit 1.11.1→1.12.0, credo 1.7.18→1.7.19,
  phoenix 1.8.7→1.8.8, req 0.5.18→0.6.1, swoosh 1.26.0→1.26.1

Fixes:
- core_components.ex: add 'type' to button :global allowlist
  (Phoenix 1.8+ attribute validation)
- status_live_test.exs: invalidate {:all_stats} cache key to
  prevent stale ETS cache from leaking between tests
- path_live_test.exs: accept all valid async states in assertion
  (race between progress frame and compute completion)
- skewt_live_test.exs: accept loading or completed state in assertion

HEEx auto-formatting from mix format (self-closing tag normalization)
2026-06-12 10:42:41 -05:00
5c62243719
chore: bump builder image to Elixir 1.20.1 / Erlang 29.0.2 2026-06-10 17:36:06 -05:00
9db2cd20f5
fix: resolve 7 pre-existing test failures and ScoreCache DateTime bug
- CacheTest: fix sweep timing race by using negative TTL (-1)
  instead of positive TTL (1) for already-expired entries
- ScoreCache: replace ETS match-spec DateTime comparisons with
  :ets.foldl + DateTime.compare — DateTime structs are maps in
  Elixir >= 1.15 and ETS can't compare maps with :< / :> guards
- Accounts: drop unsupported returning: true on delete_all,
  return [] for expired tokens list
- Backtest: catch ArgumentError from String.to_existing_atom
  for unknown feature names, preserving the helpful Mix.Error
- ContactLive IndexTest: invalidate monthly_bars cache before
  assertion so test data is visible
- RoverLocationsLive MapTest: invalidate cached points before
  assertion
- StatusLiveTest: add DB cleanup in setup to reduce test
  interference; parameterize NARR candidate coordinates
2026-06-10 13:50:24 -05:00
aab3c3736c
feat: extend HRRR forecast horizon from 18h to 48h
HRRR publishes f21-f48 at 3-hourly intervals in addition to the
hourly f01-f18 we already fetch. This extends the grid_tasks seed
from 19 rows (1 analysis + 18 forecast) to 29 rows (1 analysis +
18 hourly + 10 3-hourly), and bumps the UI forecast window
constant from 18h to 48h so the map timeline and path calculator
forecast chart automatically show the extended horizon.

No Rust logic changes needed — the pipeline is data-driven and
u8 forecast_hour already handles f48.
2026-06-10 11:36:58 -05:00
d67186176f
fix: resolve all dialyzer type errors across project (46→0 project errors)
Type spec fixes:
- duct_usable_* return boolean→float (delegate to duct_usable_for_band)
- sanitize/1 spec includes :unicode error tuples
- match_delete/1 broadened from :ets.match_spec()
- telemetry_event local type replaces :telemetry.event/0
- wgrib2 parse_lon_val_segment corrected to tuple spec
- preloaded Ecto assoc types in get_mission/get_contact! specs

Unmatched returns:
- _ = prefix on Task.start, Oban.insert, Repo.query!, PubSub.subscribe,
  :ets.new, and if-expression returns across 18 files

Pattern match fixes:
- markdown: restructure acc!=[] guard as direct pattern match
- path_compute/pskr/skewt_location_resolver: remove dead clauses
- calibrate.aprs_144: remove unreachable format_float catch-all
- unused.ex: suppress MapSet.union no_opaque

Also: remove unused unicode_util_compat from mix.lock
2026-06-08 17:51:13 -05:00
aa5c2b4808
chore: update elixir 1.20.0 / erlang 29.0.1 / debian trixie, add libsctp1 2026-06-04 15:41:20 -05:00
ea3033da03
chore: update to elixir 1.20 / otp 29, fix compile warnings
- Update .tool-versions to elixir 1.20.0-otp-29 / erlang 29.0.1
- Update all hex dependencies
- Remove unused aliases in path_compute.ex and rover_planning_live/show.ex
- Fix Oban unique states to include :suspended (use :incomplete group)
2026-06-03 14:34:39 -05:00
ec1529fa6b
fix: propagation overlay broken by iolist→binary crash in encode_binary
Two regressions from the recent performance commits:

1. ScoresController.encode_binary/1 (9cce257d): Enum.reduce produces
   iolists, but <<lats::binary>> requires actual binaries. Caused HTTP
   500 on every /scores/cells request when score files exist on disk,
   making the entire propagation heatmap overlay invisible.

2. ScoreCache.valid_times/1 (15f4175c): Pipe syntax feeds @table as
   the first argument to :ets.foldl/3, creating a 4-argument call
   that crashes at runtime.
2026-06-02 10:44:09 -05:00
e8b143813e
fix: add error handling to unsupervised Task.start calls in application.ex
- Grid cache warm-up and ML model load now wrapped in try/rescue
  so silent failures are logged instead of lost
2026-06-01 15:40:38 -05:00
bca2174f71
chore: ignore .omo/ directory 2026-06-01 15:39:40 -05:00
15f4175cce
perf: fix remaining medium/low severity inefficiencies
- Radio.group_reciprocals: replace O(n²) nested Enum.filter with O(n)
  Enum.group_by using a hash key (sorted station pair + band + hour)
- contact_live/index.ex: cache monthly bars Repo.all(Contact) query
  with 60s TTL
- ScoreCache.valid_times: replace :ets.select with :ets.foldl to
  avoid intermediate list allocation
- Accounts.list_users: add limit(100) to unbounded query
- Accounts.backfill_missing_home_qth: batch individual Oban.insert
  calls into single Oban.insert_all
2026-06-01 15:39:25 -05:00
9cce257db7
perf: fix medium-severity N+1 and inefficiency patterns
- pending_edit_for_user: add preload for [:user, :contact] to avoid
  lazy-loaded N+1 when templates access those associations
- ScoresController.encode_binary: combine 3 Enum.map passes into a
  single Enum.reduce, halving list traversals for score binary encoding
- find_duplicate_contact: push station-pair/grid matching into SQL
  WHERE clause so dedup uses the DB index instead of loading all
  matching rows + Enum.find in Elixir
2026-06-01 15:32:36 -05:00
473c2ab0ec
perf: fix high-severity LiveView blocking queries
- rover_planning_live/show.ex: use pre-loaded @rover_sites assign
  instead of Repo.all(Location) on every keystroke
- rover_locations_live/map.ex: cache locations query with 30s TTL
  (was loading + JSON-encoding all good locations in every mount)
- status_live.ex: consolidate 5 separate stat fetches into single
  cached blob, so PubSub-triggered refreshes hit cache instead of
  running 10+ DB queries each time
2026-06-01 15:31:28 -05:00
26be066f77
perf: fix critical performance issues
- NexradCache: replace :ets.tab2list() with foldl to avoid ~1.3 GB heap
  allocation on full cache (OOM risk)
- GridCache: replace :ets.select full-table copies with foldl in
  ets_latest_valid_time and ets_prune_keep_latest
- Weather.backfill_hrrr_batch: batch N individual UPDATEs into single
  VALUES-based SQL statement eliminating ~500 round-trips per batch
- Add 9 missing database indexes across 6 tables for hot-path queries
  (contacts, contact_edits, hrrr_profiles, hrrr_native_profiles,
  grid_tasks, rover_mission_paths, fixed_stations, beacons)
2026-06-01 15:29:29 -05:00
1899b3fb5a
fix web stuff 2026-05-31 16:47:11 -05:00
c722a77dcd
Add test coverage for Mailer and AboutLive
- Mailer: apply_defaults/1 sets From and Reply-To headers
- AboutLive: content rendering, empty stats, and contact count display
- Fix has_many :beacons association (wrong module path)
- Fix router pipeline order: serve_markdown before accepts,
  after secure headers so markdown responses get security headers
- Fix notify_listener_test Process.sleep regression
- Update findings.md test coverage gap status
2026-05-29 17:56:03 -05:00
316fb2fbc7
Fix low-severity bugs and re-enable Credo checks
- Bug #12: Lower rate limit to 15/min
- Bug #13: Wrap model loading in Task.start
- Bug #14: Fix classify_time_period guard gap at -3.0
- Bug #15: Not applicable (Elixir has no ?? operator)
- Bug #16: Remove fallback Repo.get for station preload
- Bug #17: Extract cache_key() in ContactMapController
- Bug #18: Add has_many :contacts and :beacons to User schema
- A&D #4: Move serve_markdown_if_requested after secure headers
- Config #1: Move signing_salt to runtime.exs env var
- Config #2: Use --check-unused instead of --unused
- Config #3: Re-enable UnsafeToAtom Credo check
- Config #5: Re-enable LeakyEnvironment Credo check
- Add @spec annotations to fix re-enabled Specs violations
- Replace String.to_atom with to_existing_atom where guarded
2026-05-29 17:29:22 -05:00
cc3dc41a6b
Fix all medium findings and split contact_live_test.exs
- Fix N+1 queries in radio.ex (preload :user)
- Add encryption_salt to session options
- Consolidate token deletion to single query
- Wrap gunzip in try/rescue for corrupt files
- Add Cache.match_delete/1 to encapsulate ETS access
- Precompute sec_i_factor_ref as module attribute
- Guard EXLA.Backend config to dev/test only
- Split 3505-line contact_live_test.exs into 4 files
- Replace Process.sleep with :sys.get_state synchronization
- Replace Process.alive? with DOM output assertions
- Clarify router.ex comment for non-live_session routes
- Update findings.md to reflect fixes
2026-05-29 15:53:04 -05:00
151baf8496
Fix all 4 performance hotspots
- grid.ex: memoize conus_points/0 with module attribute (was ~94k recompute per call)
- path_compute.ex: reduce 6 list traversals to single Enum.reduce pass
- radio.ex + application.ex: fix unused variable e and redundant case warnings
- priv: add 5 partial indexes on (qso_timestamp) for enrichment queries (weather, hrrr,
  terrain, iemre, radar) matching the WHERE status IN ('pending','failed') AND pos1 IS NOT NULL
  pattern used by contacts_needing_enrichment
- findings.md: mark all performance hotspots and corresponding items as fixed
2026-05-29 15:38:39 -05:00
5ca5edd554
Fix 7 high-severity bugs
- mechanism_classifier: add nil guards on get_lat/get_lon (no crash on bad radar lookup)
- path_compute: use Map.get with fallback for abs_humidity (no nil * dist_km crash)
- beacon_live/index: fix path_with_prefix double-slash with trim_leading
- map_live: inline flash rendering instead of calling Layouts.flash_group
- runtime.exs: raise on missing EMAIL_SERVER (prevents silent TLS failure)
- notify_listener: wrap Task.start in try/rescue (prevents linked crash on GenServer)
- notify_listener: add Process.monitor on PG notifications (detect connection loss)
- findings.md: remove fixed high-severity items
2026-05-29 15:31:47 -05:00
3bdc4b6a11
Fix 4 critical bugs and document remaining findings
- TOCTOU race: add partial unique index + rescue constraint violation on
  concurrent contact insert (radio.ex:737-768)
- Mass assignment: remove :user_id from @optional_fields (contact.ex:85)
- Stale path_live result: clear result when URL params change (path_live.ex:105)
- Blocking migrations: wrap Release.migrate() in Task.start (application.ex:79)
- Rate limiter: fix guard for monitor tokens (not is_nil vs is_binary)
- findings.md: document remaining non-critical bugs and improvements
2026-05-29 15:27:33 -05:00
828814e767
fix: resolve all compiler warnings except framework-generated phoenix_component_verify
- Replace deprecated xref:exclude with elixirc_options in vendor/oban_pro
- Remove unused require Logger from 8 files
- Pin bitstring size variables with ^ in simple_packing, wgrib2,
  complex_packing, section, nexrad_client, mqtt, and radar worker
- Remove dead defp clauses (format/1 nil, depression/2 nil)
- Rename Buildings.Parser type record/0 -> building_record/0 to avoid
  overriding built-in type
- Remove redundant catch-all in candidate_detail.ex
- Simplify contact_live/show.ex conditional based on type narrowing
- Fix DateTime.from_iso8601 return pattern in vendor/oban_web
- Upgrade phoenix_live_view to 1.1.31
- Drop --warnings-as-errors from precommit alias; known false positives
  from @after_verify-generated code in Elixir 1.20.0-rc.6 + PLV 1.1.x
- Add credo --strict to precommit as replacement static-analysis gate
2026-05-29 10:18:52 -05:00
daafa5a02a
perf(algo): unify recalibration into single Python pipeline with JSON output
Replace the two-script Python pipeline (analysis report + Elixir-source
emitter) with a single `scripts/recalibrate.py` that fits per-band
weights from PSKR spot density (VHF/UHF) and contacts↔HRRR correlations
(microwave), writing `priv/algo/band_weights.json` as a machine-readable
artifact. A new Elixir BandWeights module loads this JSON once via
`:persistent_term` cache; BandConfig.weights/1 consults it before falling
back to in-source overrides or global defaults. The script never touches
Elixir source — recalibration is now `python3 scripts/recalibrate.py`
followed by an app restart.
2026-05-25 14:45:55 -05:00
580b9e781b
chore(deps): update hex + cargo deps, fix unused variable warning 2026-05-24 12:05:43 -05:00
4d3b61c740
fix remaining bugs and quality improvements
Bug fixes:
- path_compute: use midpoint longitude (src+dest)/2 for time-of-day score
- rate_limiter: guard nil token id from burning auth rate limit bucket

Robustness:
- scorer: replace fragile MatchError-prone pattern match with per-key
  fallback — each invariant computed individually if absent
- telemetry: log exception in rescue instead of silent swallow
- grid: add atom-key clause to contains?() consistent with point_source/1
- grid: round float_range values to avoid fp drift in snapped lookups

Cleanup:
- path_compute: remove unused alias Geo (import-only module)
2026-05-24 11:56:17 -05:00
e3d430f8c4
feat(weather): add duct cutoff band map layer
Show on /weather where the detected duct geometry traps each microwave
band. Overview mode bins cells by their lowest trapped frequency
(Bean & Dutton cutoff); band-picker mode masks cells whose duct doesn't
reach the selected band. Cutoff is pre-computed per cell in both
writers (Elixir derive + Rust derive_row reads best_duct_freq_ghz from
CellValues), persisted to ScalarFile, and shipped to the JS hook via
the existing binary cell pack. Also cleans up six pre-existing
length/1 credo warnings in unrelated test files.
2026-05-15 19:51:24 -05:00
43239dbb15
fix(path): show non-zero gas/O₂/H₂O rates on loss budget detail
format_number rounds to 1 decimal, so the tiny per-km coefficients
(o2_db_km: 0.007 at 10 GHz, gas rate ~0.007 dB/km) all displayed as
'0.0'. Added format_small with 4-decimal precision for those fields.
2026-05-15 09:06:23 -05:00
fc82d3f5de
feat(nav): add Rover dropdown linking rover planner, locations, planning
Adds a Rover dropdown to the top nav bar and lists the three rover
tools in the homepage markdown Primary Resources block.
2026-05-14 17:28:00 -05:00
bd731685de
feat(api): ingest endpoint for propmonitor beacon measurements
POST /api/v1/beacon-monitor/measurements accepts one measurement per
integration window from a propmonitor client, authenticated by the
BeaconMonitor token. Records noise floor, signal peak/avg dBFS, SNR,
signal-active-fraction, gain, and frequency for later correlation
against weather and propagation scores.

Beacon UUID must resolve to an approved + on-the-air beacon (404
otherwise — the client drops 404s without retry). Retries with the
same (monitor, beacon, measured_at) are idempotent: a unique index
makes the second insert a 409. Every accepted upload stamps the
monitor's last_seen_at — measurements *are* the heartbeat.

MonitorAuth is a separate plug from the user API-token Auth plug;
monitors are not users. The rate limiter buckets each monitor by id so
retry storms after a 5xx don't burn the shared anon-IP bucket.
2026-05-13 16:07:04 -05:00