Commit graph

1929 commits

Author SHA1 Message Date
ab23031761 Migrate images from codeberg.org to git.mcintire.me
Some checks failed
Build and Push / Build and Push Docker Image (push) Failing after 2s
2026-07-24 14:38:54 -05:00
9369257ed3 Migrate images from codeberg.org to git.mcintire.me
Some checks are pending
Build and Push / Build and Push Docker Image (push) Waiting to run
2026-07-24 14:38:53 -05:00
5bb79746a4 Migrate images from codeberg.org to git.mcintire.me
Some checks are pending
Build and Push / Build and Push Docker Image (push) Waiting to run
2026-07-24 14:38:53 -05:00
0b1bc6ae24 Migrate images from codeberg.org to git.mcintire.me
Some checks are pending
Build and Push / Build and Push Docker Image (push) Waiting to run
2026-07-24 14:38:52 -05:00
0643ff76c8 Use Kaniko for Docker builds
Some checks failed
Build prop-grid-rs / Test, build, push (push) Failing after 2s
Build and Push / Build and Push Docker Image (push) Failing after 1s
2026-07-24 14:36:27 -05:00
8597b721c5 Use Kaniko for Docker builds
Some checks are pending
Build base image / Build and push base image (push) Waiting to run
Build and Push / Build and Push Docker Image (push) Waiting to run
2026-07-24 14:36:27 -05:00
6197c649b3 Use Kaniko for Docker builds
Some checks failed
Build prop-grid-rs / Test, build, push (push) Waiting to run
Build and Push / Build and Push Docker Image (push) Waiting to run
Build base image / Build and push base image (push) Failing after 2s
2026-07-24 14:36:26 -05:00
2a177676e5 Migrate from codeberg.org to git.mcintire.me
All checks were successful
Build and Push / Build and Push Docker Image (push) Successful in 3m21s
2026-07-24 14:23:22 -05:00
56b4116f6f Migrate from codeberg.org to git.mcintire.me
Some checks are pending
Build and Push / Build and Push Docker Image (push) Waiting to run
Build prop-grid-rs / Test, build, push (push) Successful in 6m44s
2026-07-24 14:23:21 -05:00
26e1163107 Migrate from codeberg.org to git.mcintire.me
Some checks are pending
Build prop-grid-rs / Test, build, push (push) Waiting to run
Build and Push / Build and Push Docker Image (push) Waiting to run
Build base image / Build and push base image (push) Successful in 4m35s
2026-07-24 14:23:20 -05:00
51678e0fb9
fix: two test issues causing 37 failures
Some checks failed
Build and Push / Build and Push Docker Image (push) Failing after 39s
1. RoverPathProfileWorker sandbox ownership: fallback_hits/2 used
   Task.async_stream spawning separate DB-querying processes that
   lacked Ecto sandbox ownership in test mode. Replaced with
   sequential Enum.map since miss list is ≤9 points — no meaningful
   perf impact and eliminates the sandbox race entirely.

2. PSKR client test: asserted '6m' band in defaults, but the actual
   microwave band name is '6cm'. Fixed assertion.
2026-07-22 16:37:17 -05:00
ca842e3add
feat(admin): beacon monitor reassign, token regen, assigned_by display, user monitor management
Uncommitted work from previous session:
- Add list_users_select/0 helper for admin select dropdowns
- Add regenerate_token/1 context function
- Preload assigned_by association on monitor queries
- Add admin nav link to beacon monitors
- Fix JS event targets (remove redundant target: @myself)
- Add reassign user form to admin monitor show page
- Add regenerate token button to admin monitor show page
- Show assigned_by admin on monitor detail page
- Add beacon monitors section to user management edit page
- Add unassign monitor capability from user management
- Add full test coverage for all new functionality
- Fix pre-existing /account route warning (redirect → /users/settings)
2026-07-22 16:27:21 -05:00
f35da3a935
beacon monitor work 2026-07-22 10:30:21 -05:00
Graham McInitre
49ade78766 fix: wire pending_edits_query as data_provider for contact edit review table
The LiveTable on /admin/contact-edits used the bare ContactEdit schema
as its data source, which caused three symptoms:
- '0 pending' counter but stale approved/rejected edits still visible
- Blank contact/submitted-by cells (select_columns stripped preloaded
  associations, cell renderers received flat maps with no :contact/:user)
- Approve/reject didn't remove the row from the table

Fix: assign {Radio, :pending_edits_query, []} as the data_provider in
mount so handle_params threads it to stream_resources. The query variant
of list_resources preserves preloaded associations and includes the
WHERE status = :pending filter.

Added two tests that verify the table rendering and edit removal.
2026-07-22 08:54:46 -05:00
fb49eb016d
feat(monitors): schema migration + remove user self-service creation
- Add hardware/config fields migration to beacon_monitors table
- Update BeaconMonitor schema with provision/config changesets
- Add context functions: create_hardware, update_config, list_all_monitors
- Remove user-facing monitor creation (browser POST + API POST)
- Update settings page: show assigned monitors table with hardware info
- Update profile page: show assigned monitors, remove register links
- Fix all tests to match new API
2026-07-21 18:28:51 -05:00
739984d3bc
feat(profile): show beacon monitors section on own user profile page
Add a Beacon monitors card to /u/:callsign that appears only when the
viewer is the profile owner. Shows registered monitors with truncated
token and last-seen timestamp, with a link to manage monitors in
settings. Includes tests for visibility rules.
2026-07-21 18:08:05 -05:00
Graham McInitre
c193f35a0c k8s/argo optimizations 2026-07-21 12:14:01 -05:00
Graham McInitre
5c6cef2227 refactor: DRY token verification — extract shared verify_hashed_token_query helper
Consolidate verify_magic_link/confirm/password_reset_token_query into
one-line delegations to a private helper, removing 29 lines of
duplicated decode/hash/query logic.
2026-07-21 11:18:15 -05:00
Graham McInitre
aa6f2ea647 fix: run all precommit targets via Makefile, exit with error at end if any fail 2026-07-21 11:15:30 -05:00
Graham McInitre
343c8ea339 fix: use Makefile for precommit — each step runs in isolated shell with MIX_ENV=test 2026-07-21 11:08:45 -05:00
Graham McInitre
e186c4c4ef fix: use sh -c wrapper for cmd in precommit to set MIX_ENV=test 2026-07-21 11:03:33 -05:00
Graham McInitre
b1b9ff63e8 fix: isolate precommit steps with cmd, split test alias from DB setup
- Move ecto.create/ecto.migrate from test alias to test.setup
- Use cmd MIX_ENV=test in precommit to isolate each step in a fresh process
- Move Mox.defmock from test_helper.exs to valkey_test.exs
2026-07-21 11:01:06 -05:00
Graham McInitre
255c99cb36 fix: unwrap Repo.transaction return values for ecto_sql 3.14, fix sandbox deadlock in rover_planning tests
- accounts.ex: Repo.transaction in ecto_sql 3.14 wraps all return values in {:ok, ...},
  causing callers to receive {:ok, {:error, ...}} and {:ok, {:ok, ...}} instead of
  direct tuples. Unwrap in update_user_email and update_user_and_delete_all_tokens.
- rover_planning_test.exs: switch to async: false + Oban.Testing manual mode so
  RoverPathProfileWorker jobs don't deadlock on sandbox connections inside
  Repo.transaction. Introduced create_and_complete_mission!/2 and run_backfill!/1
  helpers that separate transaction lifecycle from worker execution.
2026-07-21 10:24:06 -05:00
Graham McInitre
d30b6d3022 refactor: DRY extract_bearer, reuse validation helpers, add query limits
- Extract shared extract_bearer/1 from api/monitor_auth.ex → api/auth.ex
- Refactor admin_changeset to pipeline through validate_callsign/validate_name/validate_email
- Add limit: 500 to list_beacons/0 and candidate_rover_locations/1
- Combine get_mission_with_paths preloads into single round-trip
2026-07-21 10:06:12 -05:00
Graham McInitre
d3a7cfe733 fix: replace stale Flux reference with ArgoCD in kustomization.yaml 2026-07-21 08:44:19 -05:00
Graham McInitre
ae7969521a update ferqs 2026-07-20 19:55:52 -05:00
Graham McInitre
3079e8aee6 fix: point cargo pre-commit hooks at rust/prop_grid_rs/Cargo.toml
The cargo clippy and cargo fmt hooks ran from the repo root but the
Rust project lives in rust/prop_grid_rs/. Add --manifest-path so they
find the Cargo.toml and actually run.
2026-07-20 19:52:18 -05:00
Graham McInitre
b3ad3bdbe6 fix: sweep bugs across Python MQTT listener, Elixir PSKR/propagation, and Rust
Python pskr_mqtt_listen.py:
- Guard against malformed CONNACK/SUBACK/PUBLISH packets
- Fix _s() truthiness: is not None instead of if v (zero SNR was lost)
- EINTR-safe select loop, OOB data handling, VBInt overflow check
- Proper socket cleanup with try/finally + DISCONNECT on shutdown

Elixir:
- Log dropped spot errors in aggregator (was silently discarded)
- Wire retain_scores_window into NotifyListener chain completion
- Recurse sweep_tmp_dir into band/weather_scalars subdirectories
- Rescue recalibrator.run/0 to write failed status row on crash
- Handle nil in fmt_snr/fmt_callsigns (no more nil dB crashes)
- Replace Stream.run with Enum.reduce logging exits in poll_worker
- Handle File.stat race in ms_footprints prune, grid_center nil log
- Fix unused variables, stale comments, missing get_path!/1

Rust:
- Graceful JoinError handling in fetcher/hrdps_fetcher (no more panics)
- round_to_5min returns Option (leap-second safe)
- Acquire/Release ordering on shutdown flags (was Relaxed)
- Parameterized NOTIFY in db.rs, defensive scalar sweep, NaN guard
- OsString::push for tmp naming, clippy fixes
2026-07-20 19:48:45 -05:00
Graham McInitre
18e6103bed Track .envrc with use flake, move secrets to .envrc.local
Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-20 18:35:13 -05:00
Graham McInitre
ed741a8c0f Add PSK Reporter MQTT listener script
Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-20 18:32:33 -05:00
Graham McInitre
3f5b4cd60b fix: accept 4-char Maidenhead grids in PSK Reporter spots
Previously required ≥6-char locators; 4-char fields (~70×100 km)
were dropped. They're less precise for HRRR calibration but still
useful for spot display and coarse path analysis. Lowers the
minimum from 6 to 4 characters.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-20 13:54:32 -05:00
Graham McInitre
ba17a41683 fix: extract callsigns from MQTT topic when JSON payload omits sc/rc
Some FT8/Q65 multi-decoder spots omit sender/receiver callsigns
from the JSON payload, but the MQTT topic routing path always
carries them in positions 5 (sc) and 6 (rc):

  pskr/filter/v2/<band>/<mode>/<sc>/<rc>/<sl>/<rl>/<sa>/<ra>

Client now extracts callsigns from the topic and threads them
through Aggregator.ingest → Pskr.parse_spot, where they act as
fallback when the JSON sc/rc keys are nil or empty.

JSON payload values still take precedence when present — the topic
is only used as a safety net.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-20 13:37:37 -05:00
Graham McInitre
af50631a2a fix: make RepoListener resilient, bypass pgbouncer for NOTIFY/LISTEN
RepoListener now connects directly to Postgres on port 5432 for
NOTIFY/LISTEN, bypassing pgbouncer (port 6432) which does not
support session-level LISTEN in transaction-pooling mode.

Additionally, connection and LISTEN are moved from init/1 to
handle_continue/2 so the GenServer starts immediately and the
pod boots healthy even when the DB is temporarily unreachable.
Connection failures are retried with exponential backoff
(200ms → 30s max). The notifications process is monitored and
reconnected if it dies.

Falls back to the configured port if port 5432 is unreachable
(e.g. dev/test environments without pgbouncer).

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-20 13:15:45 -05:00
Graham McInitre
b65d3227cd feat: clickable band filters on PSK Reporter tab, fix all test & credo issues
PSK Reporter tab:
- Fix handle_info(:refresh_spots) to use start_async instead of blocking
- Add require Logger to fix compiler warning
- Make band counts clickable: clicking a band filters table to that band's
  last 100 spots; clicking again clears the filter
- Highlight active filter badge, show Clear filter link when filtered
- Update subtitle and empty state dynamically based on filter
- Filter persists across auto-refresh (60s)
- Fix empty state condition to render immediately without async guard

Credo fixes (mix credo --strict now passes):
- weather_map_component.ex: replace @doc false with @impl true (missing spec)
- contact_weather_enqueue_worker.ex: extract hrrr_placeholder_for_contact/3
  to reduce nesting depth
- profile_lookup.ex: replace MapSet+then+reject pattern with Enum.filter
  to fix both nesting depth and cyclomatic complexity

Test infrastructure fixes:
- Fix Release.migrate/0 to handle repos without migration directories
  (AprsRepo in test has no migrations)
- Fix insert_spot helper: add missing 19th param for inserted_at/updated_at
- Fix String.index/2 -> :binary.match for Elixir 1.20 compat
- Fix DateTime.add!/3 -> DateTime.add/3 for Elixir 1.20 compat
- Fix substring matching in limit test (GRID1 matched GRID10)
- Add Process.sleep+render calls for async band_counts in new tests
- Replace impossible 'empty state for filtered band' test with
  'switching band filter updates table' test
- All 19 tests pass, mix credo --strict clean
2026-07-20 13:01:00 -05:00
Graham McInitre
cdf4c75dd5 feat: clickable band filters on PSK Reporter tab, fix sync blocking in refresh
- Fix handle_info(:refresh_spots) to use start_async instead of blocking
  LiveView process on DB fetches
- Add require Logger to fix pre-existing compiler warning
- Make band counts clickable: clicking a band filters table to that band's
  last 100 spots; clicking again clears the filter
- Highlight active filter badge, show Clear filter link when filtered
- Update subtitle and empty state dynamically based on filter
- Filter persists across auto-refresh (60s)
- Add 5 test cases for band filtering behavior
2026-07-20 12:41:18 -05:00
Graham McInitre
b91ad05c78 deps: update all dependencies to latest versions
- Relax constraints: live_stash ~> 0.2 → ~> 1.0, nx ~> 0.9 → ~> 0.13,
  sutra_ui ~> 0.3.0 → ~> 0.4.0
- Within-constraint bumps: exla 0.12→0.13, live_table 0.4.1→0.4.2,
  phoenix_live_view 1.2.6→1.2.7, postgrex 0.22.2→0.22.3, req 0.6.2→0.6.3,
  stream_data 1.3→1.4, plus transitive (castore, circular_buffer, ecto,
  mint, plug)
- Breaking: live_table 0.4.2 removed get_merged_table_options/0 — replaced
  with LiveTable.TableConfig.get_table_options(table_options()) in
  RoverLocationsLive and RoverPlanningLive
- igniter stays at ~> 0.7 (pinned by live_table ~> 0.7.0)
2026-07-20 09:59:13 -05:00
Graham McInitre
4ac606e682 fix: ensure all unused historical NFS data gets cleaned up
Close 6 cleanup gaps on the shared /data NFS mount:

- HRDPS .hrdps.prop score files: parse_valid_time regex now matches
  the .hrdps.prop extension, so these files are found and pruned
- HRDPS scalar dirs (weather_scalars/{iso}.hrdps/): strip .hrdps suffix
  before DateTime.from_iso8601 in list_valid_time_dirs
- ScalarFile: prune_older_than was defined but never called from
  Propagation.prune_old_scores - now called alongside Scores/Profiles
- Orphan .tmp.* files: sweep all three base dirs (scores/profiles/scalars)
  for .tmp.* files left by crashed atomic-write processes
- Building cache (/data/buildings/): add MsFootprints.prune_older_than
  with 30-day mtime cutoff
- Canopy cache (/data/canopy/ + staging/): add Canopy.prune_older_than
  with 30-day cutoff, cleanup empty staging dir

All cleanup is cutoff-based (older than X time) so if the prune worker
is missed for any period it catches up fully on the next run.

PropagationPruneWorker updated to call all new cleanup functions.
2026-07-20 09:27:36 -05:00
Graham McInitre
b9aac1b210 update for ip changes 2026-07-20 09:01:36 -05:00
Graham McInitre
2e720875c4 chore: bump nix postgres from 16 to 17 2026-07-16 09:23:39 -05:00
Graham McInitre
57bd89166f chore: drop 6m from PSK Reporter subscription — only 2m and up 2026-07-16 08:40:02 -05:00
Graham McInitre
e6018c6fd9 fix: pskreporter — eliminate @spots crash, duplicate stream IDs, and slow aggregate blocking with start_async 2026-07-16 08:36:00 -05:00
Graham McInitre
5eeac23e83 refactor: extract ProfileLookup module from weather.ex (2072→1696 lines, -376) 2026-07-16 07:47:47 -05:00
Graham McInitre
695bbf5cee perf: add batch hrrr_profiles_for_contacts/1 to collapse N+1 per-contact queries into one DB round-trip 2026-07-16 07:37:34 -05:00
Graham McInitre
82e02ae0b8 perf: reuse cached hrrr_path in terrain_ready handler instead of re-fetching 2026-07-16 07:34:15 -05:00
Graham McInitre
fc51f8afe8 perf: concurrent ASOS fetches + bulk upsert in commercial poll worker
- Replace sequential Enum.each with Task.async_stream for station I/O
- Replace per-row upsert_surface_observation with batch upsert_surface_observations
2026-07-16 07:33:03 -05:00
Graham McInitre
93b8f881e2 perf: batch fixes for site-wide audit — indexes, N+1, streams, atomicity
- Add index on weather_stations(station_type, lat, lon) for nearby_stations
- Add UPPER(station1/2) expression indexes for callsign search
- Batch sync_network ~3K individual Repo.insert calls into one insert_all
- Batch Oban.insert calls in insert_unique into Oban.insert_all
- Log warnings on station elevation Repo.update errors instead of discarding
- Wrap reconcile_mission_paths delete+insert in Repo.transaction
- BeaconLive.Index: targeted stream_insert/delete instead of full re-query+push_patch
- RoverPlanningLive.Show: re-fetch single path instead of re-querying all
- PskrSpotsLive: convert to LiveView streams, add 60s auto-refresh
- ImportConfetti: add missing phx-update=ignore
2026-07-16 07:31:19 -05:00
Graham McInitre
15c4ba11a1 fix: Repo.transact -> Repo.transaction (crash on email change + token revocation) 2026-07-16 07:26:49 -05:00
Graham McInitre
c4cd659321 perf: add index on pskr_spots_hourly.last_spot_at for /pskreporter ORDER BY 2026-07-16 07:20:03 -05:00
Graham McInitre
beed193605 fix: handle GIRO API 404 cleanly — don't dump raw HTML into logs 2026-07-16 07:17:52 -05:00
Graham McInitre
65df3f9242 fix: override Repo.all/2 to accept :limit/:offset opts for Oban Pro compat 2026-07-15 14:41:02 -05:00