Commit graph

1937 commits

Author SHA1 Message Date
0c3be97abb
fix: resolve 27 security, architecture, test, and performance audit findings
Some checks failed
Build base image / Build and push base image (push) Successful in 3m10s
Build and Push / Build and Push Docker Image (push) Failing after 14s
Build prop-grid-rs / Test, build, push (push) Successful in 12m52s
P0 (security-critical):
- Gate CSV/ADIF upload tabs behind authentication, add 30s cooldown to all upload handlers
- Cap CSV/ADIF imports at 2,000 rows server-side in both parsers
- Add submitter_verified boolean to contacts (client-cannot-set, anonymous=false)
- Create k8s/secret.example.yaml with placeholders, add LIVE_VIEW_SIGNING_SALT

P1 (high-priority):
- Add Mox.verify_on_exit!() to valkey_test.exs
- Replace DateTime.utc_now() truncation with static ~U literals in map_live_test.exs
- Replace Process.sleep with render_async in pskr_spots_live_test.exs (6 occurrences)
- Add MonitorLive.Show test coverage (4 tests: owner view, non-owner redirect, config success/error)
- Extract duct-detection and mechanism-classification logic from ContactLive.Show into Propagation.PathAnalysis
- Split ContactLive.Show render into 12 function components
- Update CLAUDE.md: remove stale ML model, mark HRDPS active, add backtest/pskr dirs
- Batch CSV import enrichment jobs via new enqueue_for_contacts/1

P2 (medium-priority):
- Set secure:true on session and remember-me cookies in production
- Change SMTP TLS from verify_none to verify_peer with public_key cacerts
- Make /metrics fail-closed in production when PROMETHEUS_AUTH_TOKEN unset
- Add RateLimiter (anon_limit:10, auth_limit:60) to /api/contacts/map
- Add content-security-policy-report-only header
- Add comment noting String.to_atom is compile-time safe in hrdps_client.ex
- Delegate duplicated haversine_km to canonical Microwaveprop.Geo.haversine_km/4
- Consolidate score-tier/color/verdict formatting into Microwaveprop.Format
- Update CLAUDE.md testing section to match actual raw-string-matching practice
- Batch HrrrPointEnqueuer Repo.insert_all calls to single round-trip
- Split weather.ex (1696→216 lines) and radio.ex (1285→54 lines) into purpose-based sub-facades

P3 (low-priority):
- Add LIVE_VIEW_SIGNING_SALT warning comment, extend filter_parameters
- Add host/community validation to snmp_client.ex
- Add raw/1 safety comment in algo_live.ex
- Add hex-audit and cargo-audit Makefile targets
- Add privacy_live smoke test
- Replace notify_listener busy-poll loop with Process.monitor/1 + assert_receive
- Add ContactCommonVolumeRadar changeset validation tests (5 tests)
2026-07-27 18:19:37 -05:00
7bda38260d
fix: use full forgejo URL for kaniko-build action instead of github resolution
Some checks failed
Build base image / Build and push base image (push) Failing after 6s
Build and Push / Build and Push Docker Image (push) Failing after 7s
Build prop-grid-rs / Test, build, push (push) Failing after 6m38s
2026-07-27 14:22:14 -05:00
f753de6ff8
fix: use Forgejo checkout mirror instead of bare actions/checkout ref
Some checks failed
Build base image / Build and push base image (push) Failing after 2s
Build prop-grid-rs / Test, build, push (push) Failing after 2s
Build and Push / Build and Push Docker Image (push) Failing after 2s
2026-07-27 12:12:06 -05:00
83d3d68f12 chore: update all dependencies, fix Bandit CVE-2026-65623 HIGH
Some checks failed
Build prop-grid-rs / Test, build, push (push) Failing after 2s
Build and Push / Build and Push Docker Image (push) Failing after 2s
Elixir:
- bandit 1.12.0 → 1.12.4 (fixes quadratic CPU blow-up in WebSocket reassembly)
- phoenix_live_view 1.2.7 → 1.2.8
- plug_crypto 2.1.1 → 2.2.0
- lazy_html, elixir_make, glob_ex, tidewave minor bumps

Rust (83 crates):
- tokio 1.52.3 → 1.53.1
- sqlx → 0.8.6
- aws-lc-rs → 1.17.3
- uuid → 1.24.0, serde → 1.0.229
- hyper → 1.11.0, rustls → 0.23.42

mix hex.audit: clean
cargo clippy --release + cargo test --release: clean, 169 tests pass

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-27 11:35:06 -05:00
5c2ba2cd9f
chore: retrigger CI
Some checks failed
Build and Push / Build and Push Docker Image (push) Failing after 1s
2026-07-27 10:35:53 -05:00
31b2516d0e
fix: use bare action refs
Some checks failed
Build base image / Build and push base image (push) Failing after 1s
Build prop-grid-rs / Test, build, push (push) Failing after 2s
Build and Push / Build and Push Docker Image (push) Failing after 2s
2026-07-27 10:19:36 -05:00
39f899ce2e
fix: use forgejo action mirrors, add pre-build compile and clippy checks
Some checks failed
Build base image / Build and push base image (push) Failing after 3s
Build prop-grid-rs / Test, build, push (push) Failing after 2s
Build and Push / Build and Push Docker Image (push) Failing after 2s
2026-07-27 08:55:38 -05:00
2af49f2ae4
Replace codeberg.org references with git.mcintire.me in Dockerfile
Some checks failed
Build and Push / Build and Push Docker Image (push) Failing after 1s
2026-07-25 09:55:45 -05:00
ab23031761 Migrate images from codeberg.org to git.mcintire.me
Some checks failed
Build and Push / Build and Push Docker Image (push) Failing after 2s
2026-07-24 14:38:54 -05:00
9369257ed3 Migrate images from codeberg.org to git.mcintire.me
Some checks are pending
Build and Push / Build and Push Docker Image (push) Waiting to run
2026-07-24 14:38:53 -05:00
5bb79746a4 Migrate images from codeberg.org to git.mcintire.me
Some checks are pending
Build and Push / Build and Push Docker Image (push) Waiting to run
2026-07-24 14:38:53 -05:00
0b1bc6ae24 Migrate images from codeberg.org to git.mcintire.me
Some checks are pending
Build and Push / Build and Push Docker Image (push) Waiting to run
2026-07-24 14:38:52 -05:00
0643ff76c8 Use Kaniko for Docker builds
Some checks failed
Build prop-grid-rs / Test, build, push (push) Failing after 2s
Build and Push / Build and Push Docker Image (push) Failing after 1s
2026-07-24 14:36:27 -05:00
8597b721c5 Use Kaniko for Docker builds
Some checks are pending
Build base image / Build and push base image (push) Waiting to run
Build and Push / Build and Push Docker Image (push) Waiting to run
2026-07-24 14:36:27 -05:00
6197c649b3 Use Kaniko for Docker builds
Some checks failed
Build prop-grid-rs / Test, build, push (push) Waiting to run
Build and Push / Build and Push Docker Image (push) Waiting to run
Build base image / Build and push base image (push) Failing after 2s
2026-07-24 14:36:26 -05:00
2a177676e5 Migrate from codeberg.org to git.mcintire.me
All checks were successful
Build and Push / Build and Push Docker Image (push) Successful in 3m21s
2026-07-24 14:23:22 -05:00
56b4116f6f Migrate from codeberg.org to git.mcintire.me
Some checks are pending
Build and Push / Build and Push Docker Image (push) Waiting to run
Build prop-grid-rs / Test, build, push (push) Successful in 6m44s
2026-07-24 14:23:21 -05:00
26e1163107 Migrate from codeberg.org to git.mcintire.me
Some checks are pending
Build prop-grid-rs / Test, build, push (push) Waiting to run
Build and Push / Build and Push Docker Image (push) Waiting to run
Build base image / Build and push base image (push) Successful in 4m35s
2026-07-24 14:23:20 -05:00
51678e0fb9
fix: two test issues causing 37 failures
Some checks failed
Build and Push / Build and Push Docker Image (push) Failing after 39s
1. RoverPathProfileWorker sandbox ownership: fallback_hits/2 used
   Task.async_stream spawning separate DB-querying processes that
   lacked Ecto sandbox ownership in test mode. Replaced with
   sequential Enum.map since miss list is ≤9 points — no meaningful
   perf impact and eliminates the sandbox race entirely.

2. PSKR client test: asserted '6m' band in defaults, but the actual
   microwave band name is '6cm'. Fixed assertion.
2026-07-22 16:37:17 -05:00
ca842e3add
feat(admin): beacon monitor reassign, token regen, assigned_by display, user monitor management
Uncommitted work from previous session:
- Add list_users_select/0 helper for admin select dropdowns
- Add regenerate_token/1 context function
- Preload assigned_by association on monitor queries
- Add admin nav link to beacon monitors
- Fix JS event targets (remove redundant target: @myself)
- Add reassign user form to admin monitor show page
- Add regenerate token button to admin monitor show page
- Show assigned_by admin on monitor detail page
- Add beacon monitors section to user management edit page
- Add unassign monitor capability from user management
- Add full test coverage for all new functionality
- Fix pre-existing /account route warning (redirect → /users/settings)
2026-07-22 16:27:21 -05:00
f35da3a935
beacon monitor work 2026-07-22 10:30:21 -05:00
Graham McInitre
49ade78766 fix: wire pending_edits_query as data_provider for contact edit review table
The LiveTable on /admin/contact-edits used the bare ContactEdit schema
as its data source, which caused three symptoms:
- '0 pending' counter but stale approved/rejected edits still visible
- Blank contact/submitted-by cells (select_columns stripped preloaded
  associations, cell renderers received flat maps with no :contact/:user)
- Approve/reject didn't remove the row from the table

Fix: assign {Radio, :pending_edits_query, []} as the data_provider in
mount so handle_params threads it to stream_resources. The query variant
of list_resources preserves preloaded associations and includes the
WHERE status = :pending filter.

Added two tests that verify the table rendering and edit removal.
2026-07-22 08:54:46 -05:00
fb49eb016d
feat(monitors): schema migration + remove user self-service creation
- Add hardware/config fields migration to beacon_monitors table
- Update BeaconMonitor schema with provision/config changesets
- Add context functions: create_hardware, update_config, list_all_monitors
- Remove user-facing monitor creation (browser POST + API POST)
- Update settings page: show assigned monitors table with hardware info
- Update profile page: show assigned monitors, remove register links
- Fix all tests to match new API
2026-07-21 18:28:51 -05:00
739984d3bc
feat(profile): show beacon monitors section on own user profile page
Add a Beacon monitors card to /u/:callsign that appears only when the
viewer is the profile owner. Shows registered monitors with truncated
token and last-seen timestamp, with a link to manage monitors in
settings. Includes tests for visibility rules.
2026-07-21 18:08:05 -05:00
Graham McInitre
c193f35a0c k8s/argo optimizations 2026-07-21 12:14:01 -05:00
Graham McInitre
5c6cef2227 refactor: DRY token verification — extract shared verify_hashed_token_query helper
Consolidate verify_magic_link/confirm/password_reset_token_query into
one-line delegations to a private helper, removing 29 lines of
duplicated decode/hash/query logic.
2026-07-21 11:18:15 -05:00
Graham McInitre
aa6f2ea647 fix: run all precommit targets via Makefile, exit with error at end if any fail 2026-07-21 11:15:30 -05:00
Graham McInitre
343c8ea339 fix: use Makefile for precommit — each step runs in isolated shell with MIX_ENV=test 2026-07-21 11:08:45 -05:00
Graham McInitre
e186c4c4ef fix: use sh -c wrapper for cmd in precommit to set MIX_ENV=test 2026-07-21 11:03:33 -05:00
Graham McInitre
b1b9ff63e8 fix: isolate precommit steps with cmd, split test alias from DB setup
- Move ecto.create/ecto.migrate from test alias to test.setup
- Use cmd MIX_ENV=test in precommit to isolate each step in a fresh process
- Move Mox.defmock from test_helper.exs to valkey_test.exs
2026-07-21 11:01:06 -05:00
Graham McInitre
255c99cb36 fix: unwrap Repo.transaction return values for ecto_sql 3.14, fix sandbox deadlock in rover_planning tests
- accounts.ex: Repo.transaction in ecto_sql 3.14 wraps all return values in {:ok, ...},
  causing callers to receive {:ok, {:error, ...}} and {:ok, {:ok, ...}} instead of
  direct tuples. Unwrap in update_user_email and update_user_and_delete_all_tokens.
- rover_planning_test.exs: switch to async: false + Oban.Testing manual mode so
  RoverPathProfileWorker jobs don't deadlock on sandbox connections inside
  Repo.transaction. Introduced create_and_complete_mission!/2 and run_backfill!/1
  helpers that separate transaction lifecycle from worker execution.
2026-07-21 10:24:06 -05:00
Graham McInitre
d30b6d3022 refactor: DRY extract_bearer, reuse validation helpers, add query limits
- Extract shared extract_bearer/1 from api/monitor_auth.ex → api/auth.ex
- Refactor admin_changeset to pipeline through validate_callsign/validate_name/validate_email
- Add limit: 500 to list_beacons/0 and candidate_rover_locations/1
- Combine get_mission_with_paths preloads into single round-trip
2026-07-21 10:06:12 -05:00
Graham McInitre
d3a7cfe733 fix: replace stale Flux reference with ArgoCD in kustomization.yaml 2026-07-21 08:44:19 -05:00
Graham McInitre
ae7969521a update ferqs 2026-07-20 19:55:52 -05:00
Graham McInitre
3079e8aee6 fix: point cargo pre-commit hooks at rust/prop_grid_rs/Cargo.toml
The cargo clippy and cargo fmt hooks ran from the repo root but the
Rust project lives in rust/prop_grid_rs/. Add --manifest-path so they
find the Cargo.toml and actually run.
2026-07-20 19:52:18 -05:00
Graham McInitre
b3ad3bdbe6 fix: sweep bugs across Python MQTT listener, Elixir PSKR/propagation, and Rust
Python pskr_mqtt_listen.py:
- Guard against malformed CONNACK/SUBACK/PUBLISH packets
- Fix _s() truthiness: is not None instead of if v (zero SNR was lost)
- EINTR-safe select loop, OOB data handling, VBInt overflow check
- Proper socket cleanup with try/finally + DISCONNECT on shutdown

Elixir:
- Log dropped spot errors in aggregator (was silently discarded)
- Wire retain_scores_window into NotifyListener chain completion
- Recurse sweep_tmp_dir into band/weather_scalars subdirectories
- Rescue recalibrator.run/0 to write failed status row on crash
- Handle nil in fmt_snr/fmt_callsigns (no more nil dB crashes)
- Replace Stream.run with Enum.reduce logging exits in poll_worker
- Handle File.stat race in ms_footprints prune, grid_center nil log
- Fix unused variables, stale comments, missing get_path!/1

Rust:
- Graceful JoinError handling in fetcher/hrdps_fetcher (no more panics)
- round_to_5min returns Option (leap-second safe)
- Acquire/Release ordering on shutdown flags (was Relaxed)
- Parameterized NOTIFY in db.rs, defensive scalar sweep, NaN guard
- OsString::push for tmp naming, clippy fixes
2026-07-20 19:48:45 -05:00
Graham McInitre
18e6103bed Track .envrc with use flake, move secrets to .envrc.local
Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-20 18:35:13 -05:00
Graham McInitre
ed741a8c0f Add PSK Reporter MQTT listener script
Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-20 18:32:33 -05:00
Graham McInitre
3f5b4cd60b fix: accept 4-char Maidenhead grids in PSK Reporter spots
Previously required ≥6-char locators; 4-char fields (~70×100 km)
were dropped. They're less precise for HRRR calibration but still
useful for spot display and coarse path analysis. Lowers the
minimum from 6 to 4 characters.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-20 13:54:32 -05:00
Graham McInitre
ba17a41683 fix: extract callsigns from MQTT topic when JSON payload omits sc/rc
Some FT8/Q65 multi-decoder spots omit sender/receiver callsigns
from the JSON payload, but the MQTT topic routing path always
carries them in positions 5 (sc) and 6 (rc):

  pskr/filter/v2/<band>/<mode>/<sc>/<rc>/<sl>/<rl>/<sa>/<ra>

Client now extracts callsigns from the topic and threads them
through Aggregator.ingest → Pskr.parse_spot, where they act as
fallback when the JSON sc/rc keys are nil or empty.

JSON payload values still take precedence when present — the topic
is only used as a safety net.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-20 13:37:37 -05:00
Graham McInitre
af50631a2a fix: make RepoListener resilient, bypass pgbouncer for NOTIFY/LISTEN
RepoListener now connects directly to Postgres on port 5432 for
NOTIFY/LISTEN, bypassing pgbouncer (port 6432) which does not
support session-level LISTEN in transaction-pooling mode.

Additionally, connection and LISTEN are moved from init/1 to
handle_continue/2 so the GenServer starts immediately and the
pod boots healthy even when the DB is temporarily unreachable.
Connection failures are retried with exponential backoff
(200ms → 30s max). The notifications process is monitored and
reconnected if it dies.

Falls back to the configured port if port 5432 is unreachable
(e.g. dev/test environments without pgbouncer).

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-20 13:15:45 -05:00
Graham McInitre
b65d3227cd feat: clickable band filters on PSK Reporter tab, fix all test & credo issues
PSK Reporter tab:
- Fix handle_info(:refresh_spots) to use start_async instead of blocking
- Add require Logger to fix compiler warning
- Make band counts clickable: clicking a band filters table to that band's
  last 100 spots; clicking again clears the filter
- Highlight active filter badge, show Clear filter link when filtered
- Update subtitle and empty state dynamically based on filter
- Filter persists across auto-refresh (60s)
- Fix empty state condition to render immediately without async guard

Credo fixes (mix credo --strict now passes):
- weather_map_component.ex: replace @doc false with @impl true (missing spec)
- contact_weather_enqueue_worker.ex: extract hrrr_placeholder_for_contact/3
  to reduce nesting depth
- profile_lookup.ex: replace MapSet+then+reject pattern with Enum.filter
  to fix both nesting depth and cyclomatic complexity

Test infrastructure fixes:
- Fix Release.migrate/0 to handle repos without migration directories
  (AprsRepo in test has no migrations)
- Fix insert_spot helper: add missing 19th param for inserted_at/updated_at
- Fix String.index/2 -> :binary.match for Elixir 1.20 compat
- Fix DateTime.add!/3 -> DateTime.add/3 for Elixir 1.20 compat
- Fix substring matching in limit test (GRID1 matched GRID10)
- Add Process.sleep+render calls for async band_counts in new tests
- Replace impossible 'empty state for filtered band' test with
  'switching band filter updates table' test
- All 19 tests pass, mix credo --strict clean
2026-07-20 13:01:00 -05:00
Graham McInitre
cdf4c75dd5 feat: clickable band filters on PSK Reporter tab, fix sync blocking in refresh
- Fix handle_info(:refresh_spots) to use start_async instead of blocking
  LiveView process on DB fetches
- Add require Logger to fix pre-existing compiler warning
- Make band counts clickable: clicking a band filters table to that band's
  last 100 spots; clicking again clears the filter
- Highlight active filter badge, show Clear filter link when filtered
- Update subtitle and empty state dynamically based on filter
- Filter persists across auto-refresh (60s)
- Add 5 test cases for band filtering behavior
2026-07-20 12:41:18 -05:00
Graham McInitre
b91ad05c78 deps: update all dependencies to latest versions
- Relax constraints: live_stash ~> 0.2 → ~> 1.0, nx ~> 0.9 → ~> 0.13,
  sutra_ui ~> 0.3.0 → ~> 0.4.0
- Within-constraint bumps: exla 0.12→0.13, live_table 0.4.1→0.4.2,
  phoenix_live_view 1.2.6→1.2.7, postgrex 0.22.2→0.22.3, req 0.6.2→0.6.3,
  stream_data 1.3→1.4, plus transitive (castore, circular_buffer, ecto,
  mint, plug)
- Breaking: live_table 0.4.2 removed get_merged_table_options/0 — replaced
  with LiveTable.TableConfig.get_table_options(table_options()) in
  RoverLocationsLive and RoverPlanningLive
- igniter stays at ~> 0.7 (pinned by live_table ~> 0.7.0)
2026-07-20 09:59:13 -05:00
Graham McInitre
4ac606e682 fix: ensure all unused historical NFS data gets cleaned up
Close 6 cleanup gaps on the shared /data NFS mount:

- HRDPS .hrdps.prop score files: parse_valid_time regex now matches
  the .hrdps.prop extension, so these files are found and pruned
- HRDPS scalar dirs (weather_scalars/{iso}.hrdps/): strip .hrdps suffix
  before DateTime.from_iso8601 in list_valid_time_dirs
- ScalarFile: prune_older_than was defined but never called from
  Propagation.prune_old_scores - now called alongside Scores/Profiles
- Orphan .tmp.* files: sweep all three base dirs (scores/profiles/scalars)
  for .tmp.* files left by crashed atomic-write processes
- Building cache (/data/buildings/): add MsFootprints.prune_older_than
  with 30-day mtime cutoff
- Canopy cache (/data/canopy/ + staging/): add Canopy.prune_older_than
  with 30-day cutoff, cleanup empty staging dir

All cleanup is cutoff-based (older than X time) so if the prune worker
is missed for any period it catches up fully on the next run.

PropagationPruneWorker updated to call all new cleanup functions.
2026-07-20 09:27:36 -05:00
Graham McInitre
b9aac1b210 update for ip changes 2026-07-20 09:01:36 -05:00
Graham McInitre
2e720875c4 chore: bump nix postgres from 16 to 17 2026-07-16 09:23:39 -05:00
Graham McInitre
57bd89166f chore: drop 6m from PSK Reporter subscription — only 2m and up 2026-07-16 08:40:02 -05:00
Graham McInitre
e6018c6fd9 fix: pskreporter — eliminate @spots crash, duplicate stream IDs, and slow aggregate blocking with start_async 2026-07-16 08:36:00 -05:00
Graham McInitre
5eeac23e83 refactor: extract ProfileLookup module from weather.ex (2072→1696 lines, -376) 2026-07-16 07:47:47 -05:00