Commit graph

71 commits

Author SHA1 Message Date
c020d9c788
live_table polish + status page cleanup
- Compat CSS layer (assets/css/live_table_compat.css) maps the
  shadcn tokens live_table/sutra_ui use (bg-muted, text-foreground,
  border-border, bg-background, select-trigger, input-group, etc.)
  onto daisyUI base colors, and provides hand-written component CSS
  for select/dropdown/input-group/empty so the table has a proper
  surface + a select popover that actually hides when closed.
- Default-sort the Contacts table by most recent inserted_at.
- Beacon detail "Plot path" now encodes lat/lon to an 8-char
  Maidenhead grid when coordinates are known, so the destination
  handed to the path calculator is more precise than the beacon's
  stored 4-char grid.
- Rename BackfillLive -> StatusLive, move route from /admin/backfill
  to /status, retitle "Backfill Dashboard" -> "Status", drop the
  enqueue form (LiveStash + BackfillEnqueueWorker no longer needed
  on this page). Contact edit admin back-link now points at /status.
2026-04-12 18:19:41 -05:00
f733ddb6ac
Convert /contacts list to live_table
Replaces the hand-rolled sort/search/pagination on /contacts with
LiveTable.LiveResource. Sortable columns for station1/2, grid1/2,
band, mode, distance_km, qso_timestamp, and inserted_at; searchable
across both stations, both grids, and mode. Custom renderers preserve
the enrichment summary badge (with per-status tooltip), the invalid
flag column, and the qso/inserted_at formatters.

The reciprocal-grouping block has been dropped per the brainstorming
decision — every QSO now shows as its own row, which avoids fighting
live_table's one-row-per-record model. A "View" action links to the
detail page instead of the previous row-click handler (live_table's
stream dom_ids are random UUIDs so row-click by record id is no
longer practical without forking the library).

Test updates:
- sort test switched to ?sort_params[station1]=asc (live_table URL
  format; the old ?sort_by/?sort_order params are gone)
- pair-search test removed since live_table does a single ILIKE
  across searchable fields and no longer supports the two-callsign
  intersection that Radio.list_contacts/1 used to do; single-term
  search is exercised instead
- pagination test asserts the presence of "Page" (live_table's
  pagination text is "Page N" without the total count)

Also converts the stray cond-with-single-clause in
UserManagementLive.Index's delete handler to if/else (credo fix).
2026-04-12 17:22:15 -05:00
67d64109e6
Contact detail link to open in Path Calculator
Adds an "Open in Path Calculator" action to the contact detail
header. The link navigates to /path with source/destination prefilled
(preferring Maidenhead grids over station callsigns, since grids
are exact and callsigns go through a QRZ + geocoder round-trip)
and the contact's band selected. PathLive's handle_params already
auto-runs the calculation when source + destination are present,
so the user lands straight on the computed path ready to tweak
TX power, antenna heights, and gains.
2026-04-12 17:13:05 -05:00
bcfda54897
Assume 10 ft AGL antennas on contact detail path analysis
TerrainAnalysis.analyse was being called with default 0 m antenna
heights on both ends of a QSO's path, which makes the knife-edge
diffraction math pretend the radios are sitting directly on the
terrain surface. Pass 3.048 m (10 ft) for both ant_ht_a and ant_ht_b,
and shift the forward/reverse elevation angles to match. This is
only a display assumption on /contacts/:id — terrain worker paths
computed elsewhere are unaffected.
2026-04-12 17:11:49 -05:00
fc245367e3
User profiles at /u/:callsign, flexible band input, assorted UX cleanup
Profile page:
  * New MicrowavepropWeb.UserProfileLive at /u/:callsign is a public
    page showing a user's contacts and beacons. Resolves case-
    insensitively so /u/w5isp and /u/W5ISP are the same thing; unknown
    callsigns redirect to /. Uses daisyUI card / stats / table
    components with an avatar-placeholder initial and hero icons.
  * Accounts.get_user_by_callsign/1 (case-insensitive) plus
    Radio.list_contacts_for_user/1 and Beacons.list_beacons_for_user/1
    back the page. Beacons list includes both approved and pending so
    owners see their drafts.
  * The top nav bar and the three LiveView sidebars (MapLive,
    WeatherMapLive, ContactMapLive) now render the logged-in callsign
    as a navigate link to /u/:callsign instead of a static label.
  * Nine new tests cover the lookup, the LiveView render, and the
    ownership-scoped queries.

Flexible band input:
  * New Microwaveprop.Radio.BandResolver module converts any of:
    ADIF wavelength labels ("33cm", "1.25cm", "6mm", case/whitespace
    insensitive), numeric frequency strings ("903.100", "10368.000"),
    and canonical MHz integers into the one of the site's known bands.
    Returns the nearest allowed band for numeric inputs >= 900 MHz,
    nil otherwise.
  * 902 MHz is added to Contact.@allowed_bands, ContactEdit.@allowed_bands,
    AdifImport.@allowed_bands, and the BandResolver list so "33cm"
    round-trips end-to-end.
  * AdifImport and CsvImport now delegate band resolution to
    BandResolver, and Radio.create_contact/2 normalizes the :band attr
    on the way in so the manual form and any API callers benefit too.
    CsvImport's "invalid band" tests previously used 99999 MHz which
    the new resolver snaps to the nearest allowed band; swapped to
    "notaband" which is truly unresolvable.

Contacts and beacons list UX:
  * Remove the "Submitted" column from /contacts — it duplicated info
    already visible on the detail page and was pushing the real
    columns off narrow viewports. submitted_cell/1 and its three
    column-specific tests go with it.
  * Hide the Lat / Lon columns from /beacons — six decimal places of
    coordinates weren't useful next to the grid square and took a
    disproportionate amount of row width.
2026-04-12 16:13:08 -05:00
b49c08914e
Force 24h timestamps and let logged-in owners edit their contacts directly
Submit and contact-detail forms now use a plain text input for the QSO
timestamp with a "YYYY-MM-DD HH:MM" placeholder and regex pattern. The
native datetime-local input ignores lang="en-GB" on macOS/iOS when the
system clock is 12-hour, so it was rendering AM/PM against the user's
wishes. Ecto's :utc_datetime cast already accepts this format (verified
space+seconds, T+seconds, and both with and without seconds/Z).

Contact edit workflow grows a third branch: admins still apply directly,
non-owners still go through the admin review queue, and now logged-in
owners of a contact (user_id == current_user.id) can apply edits
directly too. New helpers Radio.owner?/2 and Radio.apply_owner_edit/3
reuse the existing normalize + diff + apply_edit_to_contact path so the
grid-change → enrichment re-enqueue pipeline kicks in automatically.
Anonymous contacts (user_id nil) and other users' contacts both return
{:error, :not_owner}.

Nine new tests cover owner?/2 and apply_owner_edit/3 including the
no-changes, wrong-user, and anonymous-contact paths.
2026-04-12 15:18:47 -05:00
c318c3a932
Nudge HRRR scores with live ASOS obs between runs
Adds Microwaveprop.Propagation.AsosNudge: a pure IDW bias-field module
that takes ASOS observations + HRRR profiles and returns re-scored grid
rows for every cell within 250km of a reporting station. Upper-air
fields (min_refractivity_gradient, pwat_mm, hpbl_m, profile, duct
metadata) pass through unchanged so HRRR's signal isn't clobbered.

The old AsosAdjustmentWorker was unwired and buggy — nil'd out ~22% of
the scoring weight and wrote orphan timestamps. Replaced with a slim
worker that queries the latest HRRR valid_time, fetches live ASOS
currents, calls AsosNudge.compute/3, and upserts onto
(lat, lon, valid_time, band_mhz) so nudged values overwrite the HRRR
hour cleanly instead of polluting available_valid_times. After each
upsert it warms ScoreCache and broadcasts propagation:updated so live
/map clients refresh.

Cron hooked up every 10 minutes in config.exs and dev.exs. Also cleaned
up the stale "dev has propagation disabled" note in CLAUDE.md.

13 new AsosNudge unit tests cover: residual computation (co-located,
out-of-grid, nil fields), IDW weighting (single station, far station,
two equidistant stations, nil component handling), upper-air
preservation, and the compute/3 entry point's shape and radius filter.

Drive-by Styler formatting touched a handful of unrelated files from
`mix format`.
2026-04-12 14:27:27 -05:00
7ed429364a Stream contact detail sections in as each enrichment loads
The contact detail page used to gather every enrichment source in one
parallel_hydrate/2 bundle and await all five tasks before updating any
assigns, so users watched the whole right-hand column flash in at the
speed of the slowest query (usually hrrr_profiles_for_path against a
42M-row table).

Each source now runs in its own start_async task: :weather, :solar,
:hrrr_path, :terrain, :iemre. Matching handle_async/3 clauses assign
the slot and call refresh_computed/1, which rebuilds elevation_profile
+ propagation_analysis + data_sources from whatever's currently in
assigns. Sections appear independently as their data arrives, and the
existing PubSub handlers (:hrrr_ready, :weather_ready, :terrain_ready,
:solar_ready) still pick up later backfill completions the same way
they did before.
2026-04-12 14:09:45 -05:00
578b4ede86 Serve /contacts/map payload via HTTP endpoint, parallel hydrate
- Add ContactMapController serving /api/contacts/map with pre-gzipped
  JSON from Microwaveprop.Cache. Browsers handle Content-Encoding: gzip
  natively; payload drops from 5.7 MB raw to 1.5 MB on the wire (73%).
  Cache TTL 10 min, invalidated on contact insert alongside the other
  contact-map cache keys.
- Move contact map payload out of the LiveView's initial HTML entirely.
  Shell renders with the filter chrome (count + band list); the hook
  fetches contacts from the HTTP endpoint in parallel with map init.
  This also avoids Phoenix.Socket's 64 KB frame limit (push_event would
  have required chunked transfer or a global frame size bump).
- Parallelize ContactLive.Show hydrate: five independent DB loads
  (weather, solar, hrrr_path, terrain, iemre) now run concurrently via
  Task.async + Task.await_many. Total latency drops from sum(each) to
  max(each), cutting contact show hydrate time roughly 5x.
2026-04-12 13:10:51 -05:00
253adaf89b Cache /weather grid, defer contact show mount, cache stats
- Add Weather.GridCache: ETS cache of derived HRRR grid rows keyed by
  valid_time, cluster-synced via PubSub. Eagerly warmed from
  PropagationGridWorker after each upsert so /weather map pan/zoom and
  weather_point_detail hit zero DB on warm cache.
- Replace latest_weather_grid DB query path with cache-first lookup +
  DB fallback. hrrr_profiles is 42M rows partitioned; pulling 3-10k
  rows per viewport on every pan was the main cost.
- ContactLive.Show: defer the heavy enrichment loads (weather, solar,
  HRRR, terrain, IEMRE, elevation profile, ITU-R propagation analysis,
  data_sources) into a handle_info(:hydrate) that runs after the shell
  renders. Initial mount now returns nil placeholders; template already
  had :if guards for all of them. Shell-to-first-paint goes from
  ~500ms-2s down to ~20ms.
- Cache fetch_queue_counts for 5s in ContactLive.Show — oban_jobs group
  by query was running on every contact page view.
- Backfill stats: wrap count_unprocessed, fetch_stats, fetch_db_stats
  in Microwaveprop.Cache with 2-5s TTLs; bump refresh debounce from 1s
  to 2s so bulk enrichment events don't thrash the DB.
2026-04-12 12:55:50 -05:00
dd13a5f2a5 Fix three security issues
1. Prevent user_id impersonation in contact submissions
   - Remove user_id from submission_changeset cast fields
   - Pass user_id as a separate server-side argument to create_contact/2
   - Client form params can no longer forge account-linked contacts

2. Restrict flag toggle to admins only
   - Add admin? guard to toggle_flag event handler
   - Hide flag button from non-admin users in template

3. Reset enrichment on timestamp edits
   - Add qso_timestamp to the fields that trigger enrichment reset
   - Prevents stale weather/HRRR/IEMRE data after time changes
2026-04-12 10:31:23 -05:00
7fb340bc35 Fix all remaining credo --strict issues (0 issues)
Aliases: add module aliases for 9 nested module references
Apply: replace apply/3 with direct module attribute calls
Line length: break 1 long spec line
Refactoring: extract helpers to reduce complexity and nesting
in show.ex, radio.ex, weather workers, terrain, duct detection,
backfill dashboard, contact map, and mix tasks
2026-04-12 10:26:53 -05:00
9abbb83469 Fix credo warnings: struct specs, length/1, and test patterns
- Replace %Struct{} with Struct.t() in all @spec annotations
- Replace length(x) > 0 with x != [] in test assertions
- Fix multi-line spec struct references in weather.ex
2026-04-12 10:26:53 -05:00
d5d0c32745 Use daisyUI components for stats, alerts, cards, and tabs
- about: stat cards → daisyUI stat, warning/donate boxes → alert
- submit: tab buttons → daisyUI tabs-box, info/instruction boxes → alert
- backfill: enrichment/DB stat boxes → daisyUI stat
- beacon show: stats container → daisyUI card
- contact show: info/status boxes → alert, data source cards → card
- ADIF: strip APP_ prefix from app-defined fields, normalize grid case
2026-04-12 09:20:20 -05:00
1174ecd9e5 Fix all 12 dialyzer warnings
- Replace MapSet with plain list + `in` (features.ex, scorer_diff.ex)
- Remove undefined Beacon.t() type reference (range_estimate.ex)
- Remove dead else branch in find_region (inversion.ex)
- Handle Nx special values in to_float catch-all (recalibrator.ex)
- Remove unreachable catch-all clauses (hrrr_native_client.ex, ncei_metar_client.ex)
- Remove unnecessary nil guards on always-typed values (show.ex)
- Remove dead sky_note/wind_note non-nil clauses (show.ex)
- Remove dead if-guard on always-truthy derive result (hrrr_native_derive.ex)
- Add @spec to path_integrated_conditions (scorer.ex)
2026-04-11 18:08:18 -05:00
d990884cea Force 24-hour time on UTC timestamp inputs 2026-04-11 17:44:39 -05:00
4ca8c5d3bc Add Q65 contact mode 2026-04-11 17:21:45 -05:00
08ffc7e028 Admin direct edit on contact show page, suggest edit for non-admins 2026-04-11 16:46:46 -05:00
a66d3094ca Add contact edit approval system with admin review queue
Registered users can suggest edits to any contact's core fields
(callsigns, grids, band, mode, timestamp). Edits enter an admin
approval queue with field-by-field diff view. On approve, changes
are applied and enrichment re-enqueued if grids/band changed.
Users receive email notification on approve or reject.

Also updates dependabot.yml for mix ecosystem.
2026-04-11 16:15:49 -05:00
9fea93f59c Show earth curvature in contact elevation profile
Terrain elevations now include the earth bulge correction, so the
profile visually humps up in the middle relative to the straight
LOS beam — showing how the curved earth rises into the signal path.
Uses actual k-factor from HRRR refractivity when available, otherwise
standard 4/3. For a 300 km path at k=4/3, the midpoint bulge is
~2,650 m (8,700 ft).
2026-04-11 14:55:21 -05:00
a40ea0d62d Show submitter callsign on the contacts index
When a contact is linked to a logged-in user account, display that
user's callsign in the Submitted column instead of a generic "Yes".
Anonymous submissions still show "Yes"; scraped contacts show an em
dash.
2026-04-09 14:02:30 -05:00
27170b5139 Merge auth links into main nav and lower password min to 8
- Register/Log in (or callsign/Settings/Log out) now live next to
  the Map/Path/Rover links in the main header instead of a separate
  top menu bar
- Add on_mount in UserAuth to assign current_scope on LiveView mount,
  and pass current_scope through to Layouts.app from every LiveView
- Drop the old top <ul> from root.html.heex
- Password minimum lowered from 12 to 8 characters
2026-04-08 10:39:51 -05:00
7652e41971 Path-integrated HRRR scoring across pos1/mid/pos2
Contact scoring now uses all HRRR profiles along the path instead of
just pos1. Aggregation strategy:
- Best along path for beneficial factors (refractivity, pressure)
- Worst along path for harmful factors (rain, wind)
- Average for neutral factors (temp, dewpoint, PWAT, BL depth)

Scorer.path_integrated_conditions/2 merges multiple profiles into a
single conditions map. Falls back gracefully to single-profile scoring
when only one profile is available.
2026-04-07 12:04:16 -05:00
d1b1b4cc7e
Show flag emoji on contacts list for flagged_invalid contacts 2026-04-05 09:15:02 -05:00
a209fc65ae
Restrict enrichment enqueue to T-Mobile CGNAT subnet (172.56.0.0/13)
Store remote_ip in session, check subnet in LiveView mount before
enqueuing weather/HRRR/terrain/solar enrichment jobs.
2026-04-04 18:55:42 -05:00
383c18e50a
Show IEMRE gridded reanalysis data on contact detail page 2026-04-04 11:35:30 -05:00
da490f7c59
Add tooltip to enrichment badge showing per-type status 2026-04-04 11:26:35 -05:00
6d2f1fb4c3
Fix swapped Submitted/Enriched column headers 2026-04-04 11:22:06 -05:00
72b272cf5c
Add enrichment status badge column to contacts table 2026-04-04 11:15:18 -05:00
915e8bdccf
Add UTC labels to Timestamp and Added column headers 2026-04-04 11:11:01 -05:00
95a2afaa01
Add inserted_at column to contacts table 2026-04-04 10:51:39 -05:00
e107706915
Show dead tuple counts on backfill dashboard, fix table visibility during vacuum
Removed n_live_tup > 0 filter that hid tables when pg_stats reports 0 rows
during VACUUM. Added dead tuple column with warning highlight to monitor
vacuum progress. Also includes Styler reformatting and unused dep cleanup.
2026-04-04 10:45:06 -05:00
4759c65809
Support two-callsign search on contacts page to find contacts between a pair 2026-04-04 09:29:32 -05:00
78bc577824
Show 'processing' instead of '1 job in queue' 2026-04-02 16:46:32 -05:00
f9c6e3730a
Allow retry of unavailable enrichment on page load 2026-04-02 16:46:12 -05:00
12d7dbf7ea
Fix phantom queued status when Oban unique constraint rejects job 2026-04-02 16:22:06 -05:00
d36256e730
Show queue position on all loading spinners, fix solar backfill
- Unified queue_info helper shows "(N jobs in queue)" on all spinners
- Solar index auto-enqueues when missing on contact page
- Freshness monitor disabled in dev
- Fix test stubs for solar client
2026-04-02 16:17:31 -05:00
0e089e8493
Auto-fetch solar index data on contact page when missing
- SolarIndexWorker accepts date arg for specific date fetch
- Contact show page enqueues solar fetch when data missing
- PubSub broadcast on solar data arrival, live-updates page
- Spinner shown while fetching
2026-04-02 16:07:38 -05:00
9fe53b4e7b
Fix terrain enqueue: check actual data not just status flag 2026-04-02 16:04:38 -05:00
da75977a55
Rename qsos table to contacts, qso_id to contact_id
- Migration renames table, column, all indexes, and FK constraints
- Contact schema now references "contacts" table
- TerrainProfile foreign key renamed from qso_id to contact_id
- All code and tests updated to use contact_id
2026-04-02 15:52:19 -05:00
d275e9e7c4
fix enrichment 2026-04-02 15:30:41 -05:00
15a09f0d7b
Fix enrichment: check actual data presence, not just status flag 2026-04-02 15:27:18 -05:00
f897a9a321
Re-enqueue enrichment jobs when status is queued but data missing
Previously, if a job was cancelled or lost while status was :queued,
viewing the contact detail page would not re-enqueue. Now HRRR,
weather/soundings, and terrain all re-enqueue when :queued with no
data, relying on Oban uniqueness for deduplication.
2026-04-02 15:07:49 -05:00
66de041886
Show HRRR queue position on contact detail page
When HRRR data is loading, display how many jobs are ahead
in the queue so users can estimate wait time.
2026-04-02 14:42:45 -05:00
faa23a9693
Add Data Sources section to contact detail page
Shows provenance of all data used in the analysis:
- HRRR: path profile count, pressure levels, valid time, grid point
- Elevation: sample count, path distance, source (SRTM), duct layers
- Terrain: sample count, verdict, max elevation, diffraction loss
- Surface observations: station count and time range
- Soundings: profile count, stations, levels, ducting detection
- Solar indices: SFI, SSN, Ap
2026-04-02 13:28:58 -05:00
1da1cbed43
Fix nil access bugs found in security/bug audit
- TerrainProfileWorker: guard pos1/pos2 nil before accessing lat/lon,
  mark unavailable if missing instead of crashing
- Propagation analysis: guard pos1 nil in compute_factors
- contact_path_points: return [] instead of [{nil, nil}] on missing coords
- JS elevation hook: null check canvas before getContext
2026-04-02 12:38:49 -05:00
2b14e836f1
Fix CSV parsing and page number input validation
- RFC 4180 CSV parser: handles quoted fields with commas and escaped
  quotes ("Smith, John" no longer breaks parsing)
- Safe page number parsing: non-numeric input defaults to page 1
  instead of raising ArgumentError
2026-04-02 12:32:59 -05:00
79af4e4959
Ensure positions computed from grids before any enrichment
- Radio.ensure_positions!/1 computes pos1/pos2/distance from grids
- Called in contact show page mount and enqueue_for_contact
- Migration backfills positions for existing contacts missing them
- Migration corrects enrichment statuses based on actual data presence
- Backfill dashboard includes contacts with grids but no positions
2026-04-02 12:28:48 -05:00
55a51f69ab
Replace boolean enrichment flags with enum status fields
- New fields: hrrr_status, weather_status, terrain_status, iemre_status
  with values: pending, queued, processing, complete, failed, unavailable
- EnrichmentStatus module defines valid state transitions
- Migration backfills: queued=true → complete, false → pending
- Workers set status on transitions (queued on enqueue, complete on finish)
- Show page reads status from contact struct, not computed dynamically
- Backfill dashboard queries status fields for accurate progress counts
- Remove cron-scheduled ContactWeatherEnqueueWorker (enrichment only on
  submission, page view, or manual backfill)
- Partial indexes on status != 'complete' for fast unprocessed lookups
2026-04-02 12:07:51 -05:00
e13519bc3e
Add timing note to HRRR loading message 2026-04-02 11:07:23 -05:00