Commit graph

1944 commits

Author SHA1 Message Date
c4e71e41b3 feat: remove skippy HRRR proxy + disk cache, fetch directly from NOAA S3
Some checks failed
Build and Push / Build and Push Docker Image (push) Failing after 4m45s
- Drop skippy.w5isp.com:8080 as HRRR_BASE_URL in dev/prod config and
  all K8s manifests. Default to direct NOAA S3 reads.
- Strip hrrr_cache_dir disk-caching infrastructure from HrrrClient
  (read_cache, write_cache, download_and_cache_grib_ranges). Files are
  now fetched fresh each time — no local filesystem cache.
- Update stale skippy references in network-policy, test comments, docs.
2026-07-28 14:36:54 -05:00
db6d231887 ci: add postgres health-check options, remove fragile /dev/tcp wait
Some checks failed
Build and Push / Build and Push Docker Image (push) Failing after 18m2s
2026-07-28 08:31:28 -05:00
1ceb932321 fix: use apply/3 to avoid undefined-module warnings with Conditional ML module
Some checks failed
Build and Push / Build and Push Docker Image (push) Failing after 3m24s
Microwaveprop.Propagation.Model only exists in lib_ml/ which is excluded
from prod elixirc_paths. Using apply/3 instead of direct @ml_module
calls avoids compile-time warnings when the module is absent. Added
credo:disable-for-next-line inline annotations since Credo flags
apply/3 with known arity but the runtime dispatch is intentional.

Also adds --warnings-as-errors to Dockerfile RUN mix compile so the
build fails on any project-level warning.
2026-07-28 08:19:23 -05:00
9242c58a5a feat(ci): add mix test step with postgres before build-and-push
Some checks failed
Build and Push / Build and Push Docker Image (push) Failing after 2m36s
Adds a postgres:17-alpine service, a wait-for-postgres step, and a
Run tests step that runs mix ecto.create/migrate/test in the same
docker-run pattern. If tests fail, the Build and push step is
automatically skipped.
2026-07-28 08:05:32 -05:00
486c1e5c66 fix(ci): install build-essential in verify step for bcrypt_elixir NIF compilation
Some checks failed
Build and Push / Build and Push Docker Image (push) Has been cancelled
2026-07-28 08:03:41 -05:00
cfa2b06617 fix(ci): install git+curl in verify-compilation container
Some checks failed
Build and Push / Build and Push Docker Image (push) Failing after 30s
The hexpm/elixir slim image lacks git (needed for heroicons git dep)
and curl (needed for esbuild/tailwind binary downloads).
2026-07-28 08:02:26 -05:00
31f7db8cb1 fix(ci): replace broken volume mount with tar pipe in verify-compilation step
Some checks failed
Build and Push / Build and Push Docker Image (push) Failing after 10s
The runner is containerised so the host Docker daemon cannot see its
filesystem. Replace `-v ${{ github.workspace }}:/app` with a
`tar | docker run -i ... tar xf -` pipeline to stream the workspace
into the verification container.
2026-07-28 07:35:54 -05:00
0c3be97abb
fix: resolve 27 security, architecture, test, and performance audit findings
Some checks failed
Build base image / Build and push base image (push) Successful in 3m10s
Build and Push / Build and Push Docker Image (push) Failing after 14s
Build prop-grid-rs / Test, build, push (push) Successful in 12m52s
P0 (security-critical):
- Gate CSV/ADIF upload tabs behind authentication, add 30s cooldown to all upload handlers
- Cap CSV/ADIF imports at 2,000 rows server-side in both parsers
- Add submitter_verified boolean to contacts (client-cannot-set, anonymous=false)
- Create k8s/secret.example.yaml with placeholders, add LIVE_VIEW_SIGNING_SALT

P1 (high-priority):
- Add Mox.verify_on_exit!() to valkey_test.exs
- Replace DateTime.utc_now() truncation with static ~U literals in map_live_test.exs
- Replace Process.sleep with render_async in pskr_spots_live_test.exs (6 occurrences)
- Add MonitorLive.Show test coverage (4 tests: owner view, non-owner redirect, config success/error)
- Extract duct-detection and mechanism-classification logic from ContactLive.Show into Propagation.PathAnalysis
- Split ContactLive.Show render into 12 function components
- Update CLAUDE.md: remove stale ML model, mark HRDPS active, add backtest/pskr dirs
- Batch CSV import enrichment jobs via new enqueue_for_contacts/1

P2 (medium-priority):
- Set secure:true on session and remember-me cookies in production
- Change SMTP TLS from verify_none to verify_peer with public_key cacerts
- Make /metrics fail-closed in production when PROMETHEUS_AUTH_TOKEN unset
- Add RateLimiter (anon_limit:10, auth_limit:60) to /api/contacts/map
- Add content-security-policy-report-only header
- Add comment noting String.to_atom is compile-time safe in hrdps_client.ex
- Delegate duplicated haversine_km to canonical Microwaveprop.Geo.haversine_km/4
- Consolidate score-tier/color/verdict formatting into Microwaveprop.Format
- Update CLAUDE.md testing section to match actual raw-string-matching practice
- Batch HrrrPointEnqueuer Repo.insert_all calls to single round-trip
- Split weather.ex (1696→216 lines) and radio.ex (1285→54 lines) into purpose-based sub-facades

P3 (low-priority):
- Add LIVE_VIEW_SIGNING_SALT warning comment, extend filter_parameters
- Add host/community validation to snmp_client.ex
- Add raw/1 safety comment in algo_live.ex
- Add hex-audit and cargo-audit Makefile targets
- Add privacy_live smoke test
- Replace notify_listener busy-poll loop with Process.monitor/1 + assert_receive
- Add ContactCommonVolumeRadar changeset validation tests (5 tests)
2026-07-27 18:19:37 -05:00
7bda38260d
fix: use full forgejo URL for kaniko-build action instead of github resolution
Some checks failed
Build base image / Build and push base image (push) Failing after 6s
Build and Push / Build and Push Docker Image (push) Failing after 7s
Build prop-grid-rs / Test, build, push (push) Failing after 6m38s
2026-07-27 14:22:14 -05:00
f753de6ff8
fix: use Forgejo checkout mirror instead of bare actions/checkout ref
Some checks failed
Build base image / Build and push base image (push) Failing after 2s
Build prop-grid-rs / Test, build, push (push) Failing after 2s
Build and Push / Build and Push Docker Image (push) Failing after 2s
2026-07-27 12:12:06 -05:00
83d3d68f12 chore: update all dependencies, fix Bandit CVE-2026-65623 HIGH
Some checks failed
Build prop-grid-rs / Test, build, push (push) Failing after 2s
Build and Push / Build and Push Docker Image (push) Failing after 2s
Elixir:
- bandit 1.12.0 → 1.12.4 (fixes quadratic CPU blow-up in WebSocket reassembly)
- phoenix_live_view 1.2.7 → 1.2.8
- plug_crypto 2.1.1 → 2.2.0
- lazy_html, elixir_make, glob_ex, tidewave minor bumps

Rust (83 crates):
- tokio 1.52.3 → 1.53.1
- sqlx → 0.8.6
- aws-lc-rs → 1.17.3
- uuid → 1.24.0, serde → 1.0.229
- hyper → 1.11.0, rustls → 0.23.42

mix hex.audit: clean
cargo clippy --release + cargo test --release: clean, 169 tests pass

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-27 11:35:06 -05:00
5c2ba2cd9f
chore: retrigger CI
Some checks failed
Build and Push / Build and Push Docker Image (push) Failing after 1s
2026-07-27 10:35:53 -05:00
31b2516d0e
fix: use bare action refs
Some checks failed
Build base image / Build and push base image (push) Failing after 1s
Build prop-grid-rs / Test, build, push (push) Failing after 2s
Build and Push / Build and Push Docker Image (push) Failing after 2s
2026-07-27 10:19:36 -05:00
39f899ce2e
fix: use forgejo action mirrors, add pre-build compile and clippy checks
Some checks failed
Build base image / Build and push base image (push) Failing after 3s
Build prop-grid-rs / Test, build, push (push) Failing after 2s
Build and Push / Build and Push Docker Image (push) Failing after 2s
2026-07-27 08:55:38 -05:00
2af49f2ae4
Replace codeberg.org references with git.mcintire.me in Dockerfile
Some checks failed
Build and Push / Build and Push Docker Image (push) Failing after 1s
2026-07-25 09:55:45 -05:00
ab23031761 Migrate images from codeberg.org to git.mcintire.me
Some checks failed
Build and Push / Build and Push Docker Image (push) Failing after 2s
2026-07-24 14:38:54 -05:00
9369257ed3 Migrate images from codeberg.org to git.mcintire.me
Some checks are pending
Build and Push / Build and Push Docker Image (push) Waiting to run
2026-07-24 14:38:53 -05:00
5bb79746a4 Migrate images from codeberg.org to git.mcintire.me
Some checks are pending
Build and Push / Build and Push Docker Image (push) Waiting to run
2026-07-24 14:38:53 -05:00
0b1bc6ae24 Migrate images from codeberg.org to git.mcintire.me
Some checks are pending
Build and Push / Build and Push Docker Image (push) Waiting to run
2026-07-24 14:38:52 -05:00
0643ff76c8 Use Kaniko for Docker builds
Some checks failed
Build prop-grid-rs / Test, build, push (push) Failing after 2s
Build and Push / Build and Push Docker Image (push) Failing after 1s
2026-07-24 14:36:27 -05:00
8597b721c5 Use Kaniko for Docker builds
Some checks are pending
Build base image / Build and push base image (push) Waiting to run
Build and Push / Build and Push Docker Image (push) Waiting to run
2026-07-24 14:36:27 -05:00
6197c649b3 Use Kaniko for Docker builds
Some checks failed
Build prop-grid-rs / Test, build, push (push) Waiting to run
Build and Push / Build and Push Docker Image (push) Waiting to run
Build base image / Build and push base image (push) Failing after 2s
2026-07-24 14:36:26 -05:00
2a177676e5 Migrate from codeberg.org to git.mcintire.me
All checks were successful
Build and Push / Build and Push Docker Image (push) Successful in 3m21s
2026-07-24 14:23:22 -05:00
56b4116f6f Migrate from codeberg.org to git.mcintire.me
Some checks are pending
Build and Push / Build and Push Docker Image (push) Waiting to run
Build prop-grid-rs / Test, build, push (push) Successful in 6m44s
2026-07-24 14:23:21 -05:00
26e1163107 Migrate from codeberg.org to git.mcintire.me
Some checks are pending
Build prop-grid-rs / Test, build, push (push) Waiting to run
Build and Push / Build and Push Docker Image (push) Waiting to run
Build base image / Build and push base image (push) Successful in 4m35s
2026-07-24 14:23:20 -05:00
51678e0fb9
fix: two test issues causing 37 failures
Some checks failed
Build and Push / Build and Push Docker Image (push) Failing after 39s
1. RoverPathProfileWorker sandbox ownership: fallback_hits/2 used
   Task.async_stream spawning separate DB-querying processes that
   lacked Ecto sandbox ownership in test mode. Replaced with
   sequential Enum.map since miss list is ≤9 points — no meaningful
   perf impact and eliminates the sandbox race entirely.

2. PSKR client test: asserted '6m' band in defaults, but the actual
   microwave band name is '6cm'. Fixed assertion.
2026-07-22 16:37:17 -05:00
ca842e3add
feat(admin): beacon monitor reassign, token regen, assigned_by display, user monitor management
Uncommitted work from previous session:
- Add list_users_select/0 helper for admin select dropdowns
- Add regenerate_token/1 context function
- Preload assigned_by association on monitor queries
- Add admin nav link to beacon monitors
- Fix JS event targets (remove redundant target: @myself)
- Add reassign user form to admin monitor show page
- Add regenerate token button to admin monitor show page
- Show assigned_by admin on monitor detail page
- Add beacon monitors section to user management edit page
- Add unassign monitor capability from user management
- Add full test coverage for all new functionality
- Fix pre-existing /account route warning (redirect → /users/settings)
2026-07-22 16:27:21 -05:00
f35da3a935
beacon monitor work 2026-07-22 10:30:21 -05:00
Graham McInitre
49ade78766 fix: wire pending_edits_query as data_provider for contact edit review table
The LiveTable on /admin/contact-edits used the bare ContactEdit schema
as its data source, which caused three symptoms:
- '0 pending' counter but stale approved/rejected edits still visible
- Blank contact/submitted-by cells (select_columns stripped preloaded
  associations, cell renderers received flat maps with no :contact/:user)
- Approve/reject didn't remove the row from the table

Fix: assign {Radio, :pending_edits_query, []} as the data_provider in
mount so handle_params threads it to stream_resources. The query variant
of list_resources preserves preloaded associations and includes the
WHERE status = :pending filter.

Added two tests that verify the table rendering and edit removal.
2026-07-22 08:54:46 -05:00
fb49eb016d
feat(monitors): schema migration + remove user self-service creation
- Add hardware/config fields migration to beacon_monitors table
- Update BeaconMonitor schema with provision/config changesets
- Add context functions: create_hardware, update_config, list_all_monitors
- Remove user-facing monitor creation (browser POST + API POST)
- Update settings page: show assigned monitors table with hardware info
- Update profile page: show assigned monitors, remove register links
- Fix all tests to match new API
2026-07-21 18:28:51 -05:00
739984d3bc
feat(profile): show beacon monitors section on own user profile page
Add a Beacon monitors card to /u/:callsign that appears only when the
viewer is the profile owner. Shows registered monitors with truncated
token and last-seen timestamp, with a link to manage monitors in
settings. Includes tests for visibility rules.
2026-07-21 18:08:05 -05:00
Graham McInitre
c193f35a0c k8s/argo optimizations 2026-07-21 12:14:01 -05:00
Graham McInitre
5c6cef2227 refactor: DRY token verification — extract shared verify_hashed_token_query helper
Consolidate verify_magic_link/confirm/password_reset_token_query into
one-line delegations to a private helper, removing 29 lines of
duplicated decode/hash/query logic.
2026-07-21 11:18:15 -05:00
Graham McInitre
aa6f2ea647 fix: run all precommit targets via Makefile, exit with error at end if any fail 2026-07-21 11:15:30 -05:00
Graham McInitre
343c8ea339 fix: use Makefile for precommit — each step runs in isolated shell with MIX_ENV=test 2026-07-21 11:08:45 -05:00
Graham McInitre
e186c4c4ef fix: use sh -c wrapper for cmd in precommit to set MIX_ENV=test 2026-07-21 11:03:33 -05:00
Graham McInitre
b1b9ff63e8 fix: isolate precommit steps with cmd, split test alias from DB setup
- Move ecto.create/ecto.migrate from test alias to test.setup
- Use cmd MIX_ENV=test in precommit to isolate each step in a fresh process
- Move Mox.defmock from test_helper.exs to valkey_test.exs
2026-07-21 11:01:06 -05:00
Graham McInitre
255c99cb36 fix: unwrap Repo.transaction return values for ecto_sql 3.14, fix sandbox deadlock in rover_planning tests
- accounts.ex: Repo.transaction in ecto_sql 3.14 wraps all return values in {:ok, ...},
  causing callers to receive {:ok, {:error, ...}} and {:ok, {:ok, ...}} instead of
  direct tuples. Unwrap in update_user_email and update_user_and_delete_all_tokens.
- rover_planning_test.exs: switch to async: false + Oban.Testing manual mode so
  RoverPathProfileWorker jobs don't deadlock on sandbox connections inside
  Repo.transaction. Introduced create_and_complete_mission!/2 and run_backfill!/1
  helpers that separate transaction lifecycle from worker execution.
2026-07-21 10:24:06 -05:00
Graham McInitre
d30b6d3022 refactor: DRY extract_bearer, reuse validation helpers, add query limits
- Extract shared extract_bearer/1 from api/monitor_auth.ex → api/auth.ex
- Refactor admin_changeset to pipeline through validate_callsign/validate_name/validate_email
- Add limit: 500 to list_beacons/0 and candidate_rover_locations/1
- Combine get_mission_with_paths preloads into single round-trip
2026-07-21 10:06:12 -05:00
Graham McInitre
d3a7cfe733 fix: replace stale Flux reference with ArgoCD in kustomization.yaml 2026-07-21 08:44:19 -05:00
Graham McInitre
ae7969521a update ferqs 2026-07-20 19:55:52 -05:00
Graham McInitre
3079e8aee6 fix: point cargo pre-commit hooks at rust/prop_grid_rs/Cargo.toml
The cargo clippy and cargo fmt hooks ran from the repo root but the
Rust project lives in rust/prop_grid_rs/. Add --manifest-path so they
find the Cargo.toml and actually run.
2026-07-20 19:52:18 -05:00
Graham McInitre
b3ad3bdbe6 fix: sweep bugs across Python MQTT listener, Elixir PSKR/propagation, and Rust
Python pskr_mqtt_listen.py:
- Guard against malformed CONNACK/SUBACK/PUBLISH packets
- Fix _s() truthiness: is not None instead of if v (zero SNR was lost)
- EINTR-safe select loop, OOB data handling, VBInt overflow check
- Proper socket cleanup with try/finally + DISCONNECT on shutdown

Elixir:
- Log dropped spot errors in aggregator (was silently discarded)
- Wire retain_scores_window into NotifyListener chain completion
- Recurse sweep_tmp_dir into band/weather_scalars subdirectories
- Rescue recalibrator.run/0 to write failed status row on crash
- Handle nil in fmt_snr/fmt_callsigns (no more nil dB crashes)
- Replace Stream.run with Enum.reduce logging exits in poll_worker
- Handle File.stat race in ms_footprints prune, grid_center nil log
- Fix unused variables, stale comments, missing get_path!/1

Rust:
- Graceful JoinError handling in fetcher/hrdps_fetcher (no more panics)
- round_to_5min returns Option (leap-second safe)
- Acquire/Release ordering on shutdown flags (was Relaxed)
- Parameterized NOTIFY in db.rs, defensive scalar sweep, NaN guard
- OsString::push for tmp naming, clippy fixes
2026-07-20 19:48:45 -05:00
Graham McInitre
18e6103bed Track .envrc with use flake, move secrets to .envrc.local
Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-20 18:35:13 -05:00
Graham McInitre
ed741a8c0f Add PSK Reporter MQTT listener script
Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-20 18:32:33 -05:00
Graham McInitre
3f5b4cd60b fix: accept 4-char Maidenhead grids in PSK Reporter spots
Previously required ≥6-char locators; 4-char fields (~70×100 km)
were dropped. They're less precise for HRRR calibration but still
useful for spot display and coarse path analysis. Lowers the
minimum from 6 to 4 characters.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-20 13:54:32 -05:00
Graham McInitre
ba17a41683 fix: extract callsigns from MQTT topic when JSON payload omits sc/rc
Some FT8/Q65 multi-decoder spots omit sender/receiver callsigns
from the JSON payload, but the MQTT topic routing path always
carries them in positions 5 (sc) and 6 (rc):

  pskr/filter/v2/<band>/<mode>/<sc>/<rc>/<sl>/<rl>/<sa>/<ra>

Client now extracts callsigns from the topic and threads them
through Aggregator.ingest → Pskr.parse_spot, where they act as
fallback when the JSON sc/rc keys are nil or empty.

JSON payload values still take precedence when present — the topic
is only used as a safety net.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-20 13:37:37 -05:00
Graham McInitre
af50631a2a fix: make RepoListener resilient, bypass pgbouncer for NOTIFY/LISTEN
RepoListener now connects directly to Postgres on port 5432 for
NOTIFY/LISTEN, bypassing pgbouncer (port 6432) which does not
support session-level LISTEN in transaction-pooling mode.

Additionally, connection and LISTEN are moved from init/1 to
handle_continue/2 so the GenServer starts immediately and the
pod boots healthy even when the DB is temporarily unreachable.
Connection failures are retried with exponential backoff
(200ms → 30s max). The notifications process is monitored and
reconnected if it dies.

Falls back to the configured port if port 5432 is unreachable
(e.g. dev/test environments without pgbouncer).

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-20 13:15:45 -05:00
Graham McInitre
b65d3227cd feat: clickable band filters on PSK Reporter tab, fix all test & credo issues
PSK Reporter tab:
- Fix handle_info(:refresh_spots) to use start_async instead of blocking
- Add require Logger to fix compiler warning
- Make band counts clickable: clicking a band filters table to that band's
  last 100 spots; clicking again clears the filter
- Highlight active filter badge, show Clear filter link when filtered
- Update subtitle and empty state dynamically based on filter
- Filter persists across auto-refresh (60s)
- Fix empty state condition to render immediately without async guard

Credo fixes (mix credo --strict now passes):
- weather_map_component.ex: replace @doc false with @impl true (missing spec)
- contact_weather_enqueue_worker.ex: extract hrrr_placeholder_for_contact/3
  to reduce nesting depth
- profile_lookup.ex: replace MapSet+then+reject pattern with Enum.filter
  to fix both nesting depth and cyclomatic complexity

Test infrastructure fixes:
- Fix Release.migrate/0 to handle repos without migration directories
  (AprsRepo in test has no migrations)
- Fix insert_spot helper: add missing 19th param for inserted_at/updated_at
- Fix String.index/2 -> :binary.match for Elixir 1.20 compat
- Fix DateTime.add!/3 -> DateTime.add/3 for Elixir 1.20 compat
- Fix substring matching in limit test (GRID1 matched GRID10)
- Add Process.sleep+render calls for async band_counts in new tests
- Replace impossible 'empty state for filtered band' test with
  'switching band filter updates table' test
- All 19 tests pass, mix credo --strict clean
2026-07-20 13:01:00 -05:00
Graham McInitre
cdf4c75dd5 feat: clickable band filters on PSK Reporter tab, fix sync blocking in refresh
- Fix handle_info(:refresh_spots) to use start_async instead of blocking
  LiveView process on DB fetches
- Add require Logger to fix pre-existing compiler warning
- Make band counts clickable: clicking a band filters table to that band's
  last 100 spots; clicking again clears the filter
- Highlight active filter badge, show Clear filter link when filtered
- Update subtitle and empty state dynamically based on filter
- Filter persists across auto-refresh (60s)
- Add 5 test cases for band filtering behavior
2026-07-20 12:41:18 -05:00