Commit graph

277 commits

Author SHA1 Message Date
bbeb95ff5d
CSV/ADIF import: upsert existing contacts on grid/mode refinement
Widen dedup match to 4-char grid prefix so an upload with a longer grid
(e.g. FN42aa25) finds the existing FN42 contact. Within a match, classify
as a refinement when the upload strictly extends the existing grid or
fills a missing mode; as a contradiction (still skipped) when grids or
modes genuinely disagree. Refinements flow through a new preview bucket,
update the existing row in place, and re-enqueue enrichment when the
position changed.
2026-04-16 10:57:37 -05:00
55d4f289c2
NarrClient.in_coverage?/1 + callsite guards (1979 → 2014-10-02)
Post-2014 contacts with hrrr_status=:unavailable were being dispatched to
NARR, which 404s because NCEI's archive ends 2014-10-02. 14K of the 14.3K
candidates in prod are post-2014 — these are HRRR's responsibility.

- NarrClient.in_coverage?/1 returns true only inside 1979-01-01 → 2014-10-02
- narr_jobs_for_contact and maybe_enqueue_narr (contact_live) bail out of
  coverage returning []
- BackfillEnqueueWorker.type_filter scopes :narr to qso_timestamp <
  coverage_end so the cron doesn't keep picking post-2014 candidates
2026-04-16 09:31:19 -05:00
3604896726
Rename ERA5 → NARR across the codebase
- Schema: Era5Profile → NarrProfile; table renamed via migration
  rename_era5_profiles_to_narr_profiles (ALTER TABLE + rename indexes,
  atomic and instant, no row rewrite)
- Weather helpers: find_nearest_era5/era5_for_contact/era5_profiles_for_path
  → find_nearest_narr/narr_for_contact/narr_profiles_for_path
- BackfillEnqueueWorker: :era5 → :narr type (virtual, no narr_status column);
  reconcile_stale_queued_to_unavailable and status_priority_order now skip
  virtual types via @virtual_types. Fixes the prod crash where the cron tried
  to update a non-existent era5_status column.
- ContactWeatherEnqueueWorker: build_era5_jobs → build_narr_jobs,
  era5_jobs_for_contact → narr_jobs_for_contact
- ContactLive: @era5 → @narr, maybe_enqueue_era5 → maybe_enqueue_narr;
  UI label "ERA5 (0.25°)" → "NARR (32 km)"
- Cron (runtime.exs): args types list now "narr" instead of "era5"
- /status: progress row, status-by-type card, totals.narr_profiles, and
  table-count lookup all target narr_profiles
- Drop obsolete Era5CdsJob schema + era5_cds_jobs table (inspection
  artifact from the retired CDS pipeline; 34 orphan rows in prod)
- Misc docstring/comment cleanups (skew_t, about, wgrib2, propagation_train)

Includes a regression test for the virtual-type crash.
2026-04-16 09:22:23 -05:00
7857d3bc5a
Status page: relabel ERA5 → NARR, include NARR in backfill cron
- /status progress bar, status-by-type card, and internal map keys now
  read "NARR" (data-progress-key="narr", narr_candidates/narr_done).
  era5_profiles table stays as the DB landing spot (historical artifact).
- BackfillEnqueueWorker cron now passes "era5" in types so pre-2014
  contacts whose hrrr_status is :unavailable actually get NarrFetchWorker
  jobs enqueued every 30 min. The :era5 type key still maps internally
  to NarrFetchWorker via era5_jobs_for_contact/1.
2026-04-16 08:57:01 -05:00
1f2a729d40
Drop dead Era5*/CDS code paths
After the NARR backfill landed, nothing reachable from the app calls
any of these modules. Net removal: ~2700 lines.

Deleted:
- Microwaveprop.Workers.{Era5Fetch,Era5Submit,Era5Poll,Era5MonthBatch}Worker
- Microwaveprop.Weather.{Era5BatchClient,Era5Client}
- Mix.Tasks.Era5Backfill (superseded by BackfillEnqueueWorker cron)
- Matching test files (~1100 LOC of test)
- era5/era5_submit/era5_poll/era5_batch queue blocks from runtime.exs
- era5_req_options Req.Test stub config from test.exs

Kept (actively used or intentionally preserved):
- Era5Profile schema (the era5_profiles table is the NARR target)
- Era5CdsJob schema + its test (inspection handle on the era5_cds_jobs
  table, which retains rows from the failed CDS runs)
- :era5 type key in ContactWeatherEnqueueWorker / BackfillEnqueueWorker
  (the symbol still means "historical backfill", just routed to NARR now)

Swapped the one remaining live Era5FetchWorker call site in
contact_live/show.ex's maybe_enqueue_era5 path to NarrFetchWorker
(with NarrClient.snap_to_analysis_hour for the 3-hourly slot).

Test suite: 1513 tests, 0 failures (-50 from the deleted era5 test
files). No compile warnings. Credo clean.
2026-04-16 08:26:29 -05:00
83ea123e22
PathLive: ionosphere readout panel for VHF paths
Closes the GIRO → SporadicE → display loop. When a 144 or 440 MHz path
is calculated, PathLive looks up the nearest polled ionosonde's latest
foEs via Ionosphere.nearest_foes/3, runs SporadicE.es_score for the
actual (band, distance) pair, and renders a panel showing:

* Live foEs / foF2 / station-code / timestamp
* Computed single-hop Es MUF for this exact path length
* 0-100 Es score for the selected band
* Human-readable interpretation ("strong opening", "tropo only",
  "out of single-hop window", etc.)

Panel only appears for 144/440 (Es isn't relevant on microwave) and
only when the nearest station has data within the last 2 hours — if
ionosonde data is stale or missing, the panel is omitted entirely
rather than shown with misleading values.

Does not touch the grid scorer on /map — a grid cell has no intrinsic
path length, so Es scoring there is conceptually muddy. PathLive is
the right home for path-specific predictions.
2026-04-15 14:48:47 -05:00
33347f7fc2
WeatherMap: legend in top-right, description back in sidebar, play/stop, fix reload loop
Corrections to the previous weather-map pass:

* The "helper" that was supposed to live in the top-right was the
  color legend, not the layer description. Move the Leaflet legend
  control to `topright` (was `bottomright` on desktop).

* Put the layer description back in the desktop sidebar where it
  used to live, now with a "Group · Label" header above the body
  text so it matches the new style in the mockup.

* Add play/stop controls to the weather forecast timeline, ported
  from the propagation map: start plays "now" → end at 1s/step and
  loops; stop returns to the time closest to wall clock. A manual
  click interrupts playback the same way it does on /map.

* Fix the reconnect/reload loop I introduced by making mount do a
  synchronous ProfilesFile read + derive for the full 92k-point
  grid. That was pushing mount past the LV socket join timeout on
  a cold pod, which makes the LV client fall back to a full HTTP
  reload every ~20 seconds (no server-side error). Mount now uses
  the original `latest_weather_grid/1` cache-only hot path and only
  `weather_grid_at/2` (sync disk read) on user timeline scrubs.

* Clean up the playback timer in the hook's `destroyed` callback.
2026-04-15 14:11:53 -05:00
725a7e6bda
WeatherMap: forecast-hour timeline + top-right layer description
Two related UI changes driven by the same data:

* Move the selected-layer description out of the sidebar and into a
  dedicated top-right overlay on the map. The sidebar kept the same
  layer buttons but dropped the descriptive text; the new overlay
  shows group + layer name + description in one card. Mobile still
  shows the description in the collapsible panel since it has no
  top-right free real estate.

* Add a forecast-hour timeline bar at the bottom of the map, matching
  the propagation map. WeatherMapLive enumerates ProfilesFile on mount
  (the grid worker has been persisting f00..f18 on disk since
  commit 07ffcf5), pushes data-valid-times for the JS hook to render
  as buttons, and handles a new select_time event by reading the
  per-hour ProfilesFile on demand via Weather.weather_grid_at/2.

weather_grid_at/2 deliberately skips the GridCache write path for
non-latest hours — caching 18 × 92k rows would add ~300 MB per pod.
A ~2 MB ETF decode per scrub is fast enough for a click.

Subscribes to propagation:pipeline so the timeline picks up new
forecast hours live as they land, without waiting for the full chain
to finish.
2026-04-15 13:57:23 -05:00
790eef1726
PathLive: live-refresh the forecast when new grid scores land
Subscribe to "propagation:updated" on connected mount and, when the
grid worker finishes a forecast-hour step, re-read point_forecast for
the path midpoint using the stashed band_mhz and source/destination.
Terrain, HRRR profiles, and the loss/power budget are independent of
grid scores and stay as-is. Before this, a /path page rendered once at
mount and only refreshed on manual reload.
2026-04-15 13:42:44 -05:00
c3616b31c7
Surface HRRR weather details on /path
Adds an Atmospheric Profile section to the link calculator that shows
the HRRR data actually feeding the prediction: valid/run time, a
per-point table (src/mid/dst) with surface temp, dewpoint, pressure,
HPBL, PWAT, surface refractivity and dN/dh, plus a skew-T log-P chart
for the midpoint vertical profile.

Pulls the skew-T chart function out of contact_live into a shared
component so both LiveViews can render it without duplication.
2026-04-15 11:57:51 -05:00
7fcb2715a0
Enlarge /path forecast chart so detail is legible
Viewbox goes from 300x96 to 1200x240 with aspect-[5/1] so the SVG
fills the card instead of shrinking to a ~300 px strip. Fonts,
gridlines (0/25/50/75/100), and point markers scale up to match,
and the x-axis now shows up to 7 evenly spaced hour labels instead
of just start/middle/end.
2026-04-15 11:09:05 -05:00
222d0fdca3
Fix mojibake in /path loss budget gaseous label
HEEx attribute value syntax (label="...") treats the content as a
raw attribute string, not as an Elixir string literal — so escape
sequences like \u2082 are NOT interpreted and show up verbatim in
the rendered page. Replace with the literal subscript-2 characters
so the label reads 'Gaseous (O₂+H₂O)'.
2026-04-15 10:37:58 -05:00
2d9e5a2d1f
Make the map timeline read .ntms files authoritatively
Two related fixes so the main map reliably picks up new binary
propagation score files as soon as PropagationGridWorker writes them.

1. Propagation.available_valid_times/1 previously preferred ScoreCache
   over ScoresFile, using the cache as an index of what was available.
   The cache is a lazy ETS of whatever hours have been fetched or
   broadcast, which is a strict subset of what's on disk. A new
   forecast hour landing on disk while the cache was warm with older
   entries was invisible to the timeline until the cache happened to
   catch up. Read directly from ScoresFile so the disk store is the
   source of truth.

2. Add Propagation.scores_at_fresh/3 that always reads the .ntms file
   and overwrites the cache entry, and use it from MapLive's
   propagation_updated handler. PropagationGridWorker publishes the
   cache_refresh on `propagation:cache` and the timeline ping on
   `propagation:updated` as separate PubSub broadcasts, so by the time
   MapLive runs through scores_at the ScoreCache GenServer may not
   have processed the refresh yet — fetch_bounds then returns the
   previous chain's bytes. scores_at_fresh takes disk as the source
   of truth for the refresh path and warms the cache as a side effect
   so subsequent readers see the new data.
2026-04-14 17:42:30 -05:00
30c1018400
Extend skew-T plot to cover the full troposphere
Historical contacts showed a skew-T log-P diagram that stopped at 700 mb
because HrrrClient and Era5Client only fetched pressure-level data down
to the top of the boundary layer. The chart canvas already ran up to
100 mb, so the trace clipped mid-atmosphere.

Two complementary fixes:

1. Extend @pressure_levels in HrrrClient and Era5Client with
   650/600/550/500/450/400/350/300/250/200/150/100 mb so new fetches
   cover the full troposphere + lower stratosphere.

2. Prefer the native hybrid-sigma profile for the contact-detail
   skew-T when one has been backfilled for the contact's hour. The
   native profile already stores all 50 hybrid levels up to ~19 km,
   so historical contacts covered by the native backfill get a full
   trace without re-hitting S3. A new HrrrNativeProfile.to_skew_t_profile/1
   converts the parallel arrays into the %{"pres","tmpc","dwpc","hght"}
   list shape the renderer expects, deriving dewpoint from SPFH via the
   Magnus inverse. Weather.find_nearest_native_profile/3 mirrors
   find_nearest_hrrr/3 for the lookup.
2026-04-14 17:25:58 -05:00
92ef6c67fc
Defer beacon coverage estimate until the toggle is flipped on
Previously RangeEstimate.estimate/1 ran on every beacon page mount,
even though the cells data was only rendered after the user flipped
the coverage toggle — a wasted bbox grid pass per page load. Start
with estimate: nil, compute lazily in handle_event("toggle_coverage")
when flipping on, and push the cells payload to the Leaflet hook via
a new load_coverage event. The JS hook no longer reads data-cells on
mount and only builds the cellLayer when the server sends it. Cached
in the socket so re-toggling is instant.
2026-04-14 17:13:05 -05:00
d9ec8a91c3
Stop HRRR badge from wrapping mid-phrase on the contact page
The atmospheric profile header was a single-line flex row with no
wrap, so on narrower viewports the HRRR (3 km) badge and the unit
suffixes were breaking mid-phrase — the badge showed "HRRR (3 /
km)" with borders crossing through the text. Add flex-wrap plus
whitespace-nowrap on each cell so the whole row wraps cleanly
onto a second line instead of shrinking each cell vertically.
2026-04-14 16:59:32 -05:00
7f1a7fb369
Add skew-T log-P diagram to the contact detail page
Render the HRRR pressure-level profile as a full skew-T log-P
diagram on the contact detail page: skewed isotherms, isobars,
dry adiabats, saturation mixing ratio lines, plus the T and Td
traces. Math lives in MicrowavepropWeb.SkewT (Magnus formula,
dry adiabat potential temperature, log-P projection) and is
exercised by a dedicated test module. The atmospheric profile
section now expands by default so the chart is visible without
an extra click.
2026-04-14 16:52:23 -05:00
e3ddf107d9
Gate beacon coverage toggle to 5.76 GHz and above
The estimated current coverage map is only meaningful on bands
where atmospheric attenuation shapes the reach — below 5.76 GHz
it's dominated by free-space loss and misleads. Disable the
toggle, show a helper note explaining the limit, and guard the
handle_event against clients that try to force it.
2026-04-14 16:43:07 -05:00
c7a1e352a7
Show all executing pipeline workers in the status chip
The :propagation Oban queue runs with concurrency 2, so the grid
worker chain step and the 10-minute ASOS nudge can (and do) execute
in parallel. PipelineStatus.current/0 used to return only the
most-recently-started worker, which made the chip label flip to
"ASOS nudge" at every :00 / :10 tick even though HRRR was still
chugging through forecast hours behind it.

Replace the single :detail field with a list of running_detail
maps, each tagged with :grid or :asos so the chip can:

  - Render one sub-line per currently-executing worker
  - Sort them deterministically (grid before asos) so the eye
    doesn't see rows swap
  - Still override the grid entry with the forecast-hour progress
    label when a propagation_pipeline_progress broadcast arrives
2026-04-14 14:04:44 -05:00
0ed47db8b6
Process one forecast hour per PropagationGridWorker perform
A full f00-f18 sweep takes ~170 min of wall time, longer than the
~90 min gap between deploys on this cluster. Over the last 7 days
every run was killed mid-sweep and zero runs completed. The
propagation_scores table only ever held the scraps from partial
runs that landed before the pod died, which is why the map looked
like it was showing "the current hour" (or nothing).

The worker now processes exactly one forecast hour per perform/1
(~8-10 min) and enqueues the next hour as a fresh Oban job. A cron
fire with empty args seeds the chain at f00; subsequent runs carry
run_time + forecast_hour args. At f18 the chain stops and the
pruner runs. Each step broadcasts propagation:updated immediately
so new hours appear on the map as they land.

Wall time per perform drops from ~170 min to ~10 min, so Lifeline's
2h rescue window is no longer a factor and a pod restart loses at
most one forecast hour instead of the whole sweep. Retries and
max_attempts now describe a single fh, not the whole chain.

Also: bump the :propagation queue from 1 to 2 slots so
PropagationPruneWorker can run alongside the chain job, drop the
worker timeout/1 from 90 min to 20 min to match single-fh runs,
and pull Lifeline rescue_after from 120 min to 45 min to keep the
safety net above the step timeout.

Adds a "Data from HH:MM UTC · Nh ago/now/+Nh" indicator above the
pipeline chip in both the mobile and desktop sidebars so the user
can tell which valid_time the map is showing when the bottom
timeline isn't visible.
2026-04-14 13:24:59 -05:00
fac604b289
Fix ERA5 cap math + stuck threshold, split chip label onto two lines
Era5SubmitWorker was counting era5_cds_jobs rows 1:1 against the CDS
150-job ceiling, but each row holds TWO CDS job IDs (single-level +
pressure-level). Prod hit 74 rows = 148 in-flight jobs and got stuck
in a reject→resubmit spiral. The cap guard now multiplies row count
by 2 to match what CDS sees.

Era5PollWorker @stuck_after_seconds goes from 4h to 18h. CDS
single-level routinely sits at 'accepted' for 12+h under load while
pressure finishes in ~90 min; the 4h threshold was firing on normal
slow runs and feeding the same spiral.

PipelineStatus now returns `label` + `detail` separately so the map
chip can render "Updating propagation" on one line and the
sub-detail ("HRRR run" / "ASOS nudge" / "+3h" / "now") on a second
line below it. running_label/1 is renamed to running_detail/1 and
returns just the sub-part.
2026-04-14 12:06:39 -05:00
434fca2ad1
Show forecast-hour progress on the map pipeline chip
PropagationGridWorker now broadcasts a {:propagation_pipeline_progress,
%{forecast_hour:, valid_time:}} message on the propagation:pipeline
PubSub topic at the start of each process_forecast_hour/4 call.
MapLive subscribes on mount, stashes the last message in the new
:pipeline_progress assign, and clears it on the refresh tick whenever
PipelineStatus flips out of :running.

The chip component consults a new PipelineStatus.running_label/1
helper that formats the payload as "Updating propagation · now" for
f00 and "Updating propagation · +Nh" for f01-f18, falling back to
the base running label until the first progress message arrives.
Covered by unit tests on the formatter and an integration test in
MapLiveTest that seeds an executing grid job, broadcasts progress,
and asserts the rendered chip.
2026-04-14 11:12:51 -05:00
8672ddbafc
Relocate pipeline status chip to the sidebar footer
The running label "Updating propagation (HRRR run)" didn't fit in
the 224 px desktop sidebar alongside the NTMS title block, so it
was being truncated. Move the chip to the bottom of the sidebar
(below the auth menu) where it can use the full sidebar width,
let the text wrap naturally instead of truncating, and drop the
trailing ellipsis. Mobile chip stays at the top since the mobile
layout collapses menu + auth into a dropdown.
2026-04-14 10:41:11 -05:00
554bc9f339
Show aggregated propagation pipeline status on the map
Adds a small chip below the NTMS title in both the mobile and
desktop sidebars of /map that collapses PropagationGridWorker +
AsosAdjustmentWorker Oban state into one of four states —
running / idle / stale / unknown — with a plain-language label
("Updating propagation (HRRR run)…", "Up to date · 12m ago").
The chip refreshes on a 15s timer and whenever the pipeline
broadcasts propagation:updated, so a long HRRR sweep is visible
from the moment a visitor opens the page.
2026-04-14 10:01:34 -05:00
2b323189d0
Add grid squares and weather radar toggles to the weather map
Ports the same two overlay toggles from /map to /weather:
- Maidenhead grid squares via the shared updateGridOverlay helper,
  redrawn on moveend when visible.
- ECCC GeoMet WMS radar (CONUS + Canada dBZ), lazy-created on first
  toggle with a 6-minute tile refresh timer.

Both toggles live in the desktop sidebar below the layer description
and in the mobile pulldown panel. Hook cleanup clears the radar
refresh timer on destroyed().
2026-04-13 16:31:54 -05:00
6c99798ddc
Shrink status page DB stat values so 8-digit counts fit
HRRR Profiles (76M+ rows) was overflowing the stat box at text-2xl.
Drop all four Database stats to text-xl for uniform sizing.
2026-04-13 15:52:37 -05:00
d4b849f976
Show ERA5 fallback on contact detail, fix status page ERA5 progress
Contact detail page now renders an atmospheric profile section backed by
HRRR when available and ERA5 (reanalysis) otherwise, labeled with the
data source. When both sources are missing and HRRR is known-unavailable
(pre-2014 contacts), the Era5FetchWorker is enqueued so the month-tile
backfill gets triggered from a user visit.

Status page ERA5 progress previously computed "complete" as
(total_contacts - hrrr_unavailable_contacts), which treated every contact
with any HRRR status as ERA5-complete — inflating the bar to 76% while
era5_profiles had 0 rows. Replaced with a per-candidate EXISTS query
against era5_profiles using the same 0.15°/30-min tolerance
Weather.find_nearest_era5/3 uses at lookup time.
2026-04-13 13:44:31 -05:00
e46da74472
Show windowed page list in live_table footer
Previously rendered only the current page button. Now renders 1…(current-2)..(current+1)
using just has_next_page, collapsing with an ellipsis past page 4.
2026-04-13 10:02:19 -05:00
a8d4d70abe
live_table polish: border color, control layout, row clicks, pagination
- Tailwind source(none) wasn't scanning deps, so shadcn tokens
  (border-border, bg-muted, text-foreground, ...) used by live_table
  and sutra_ui weren't generated, leaving `border` to fall back to
  currentcolor — a heavy dark border around every table in light mode.
  Add @source directives for deps/live_table/lib and deps/sutra_ui/lib.
- Drop the hardcoded width:100% on the outer .select wrapper so the
  per-page selector's Tailwind w-24 wins instead of stretching across
  the header and overlapping the search box.
- Make whole rows clickable: global click delegator in app.ts forwards
  tr clicks to the first <a href> in the row (the Actions "View" link),
  skipping inner interactive elements. cursor + hover highlight added.
- Replace the default Prev/Next footer with a daisyUI .join/.btn-based
  pager (MicrowavepropWeb.LiveTableFooter), wired up globally via
  :live_table defaults so it applies to every live_table at once.
2026-04-13 08:38:24 -05:00
c020d9c788
live_table polish + status page cleanup
- Compat CSS layer (assets/css/live_table_compat.css) maps the
  shadcn tokens live_table/sutra_ui use (bg-muted, text-foreground,
  border-border, bg-background, select-trigger, input-group, etc.)
  onto daisyUI base colors, and provides hand-written component CSS
  for select/dropdown/input-group/empty so the table has a proper
  surface + a select popover that actually hides when closed.
- Default-sort the Contacts table by most recent inserted_at.
- Beacon detail "Plot path" now encodes lat/lon to an 8-char
  Maidenhead grid when coordinates are known, so the destination
  handed to the path calculator is more precise than the beacon's
  stored 4-char grid.
- Rename BackfillLive -> StatusLive, move route from /admin/backfill
  to /status, retitle "Backfill Dashboard" -> "Status", drop the
  enqueue form (LiveStash + BackfillEnqueueWorker no longer needed
  on this page). Contact edit admin back-link now points at /status.
2026-04-12 18:19:41 -05:00
263c1bbdf5
Fix /path crash on clear-terrain diffraction loss
TerrainAnalysis.deygout_diffraction/2 short-circuits to integer 0
when the path has no interior points or the principal edge nu is
below the shadow boundary, which then made Float.round/2 blow up
inside compute_loss_budget when the rest of the losses were all
floats. Multiply the diffraction value by 1.0 before it feeds into
the total so clear paths coerce cleanly without touching
TerrainAnalysis's (number()) contract.
2026-04-12 17:50:04 -05:00
8121257c59
Line-chart forecast + unit fix on /path
Replace the stacked-bar 18-hour forecast on the path calculator with
an SVG line chart that mirrors the look of the point-detail sparkline
on /map:

- polyline through every hour with per-tier stroke color
- score gridlines at 0 / 50 / 100
- white "now" marker outlined in the tier color
- highlighted best-hour dot with score label
- first / middle / last time labels along the x-axis
- Improving / Steady / Declining trend chip in the header

Rendered server-side in HEEx as a viewBox="0 0 300 96" SVG so it
scales responsively and inherits the daisyUI text color for axes.
The Best / Worst summary line under the chart is preserved.

Also fixes the Path Weather Avg card: the "Refrac Gradient" cell was
the only stat that split its units out onto a second line in a
smaller muted row. Move "N/km" up beside the value so the cell reads
the same as Pressure (mb), PWAT (mm), BL Depth (m), etc.
2026-04-12 17:49:16 -05:00
547b8451c0
Render literal × in "Feed Loss (×2)" on /path
HEEx attribute values aren't run through Elixir's string escape
machinery, so "Feed Loss (\u00d72)" was showing up verbatim in the
Power Budget card instead of collapsing to the multiplication sign.
Swap the escape for the literal U+00D7 character.
2026-04-12 17:46:59 -05:00
99e7560601
Drop gridmap.org dependency, resolve callsigns locally
Instead of shelling out to https://gridmap.org/locate/:callsign for
callsign → lat/lon lookups, ports the resolver pipeline from gridmap-web
into this project so the whole flow runs in-process.

New modules:

- Microwaveprop.Qrz — cache facade over the QRZ.com XML callsign API.
  Looks up from qrz_callsigns first, falls back to a live fetch, and
  upserts the result with a configurable cache_ttl_hours (default
  168h / 7 days).
- Microwaveprop.Qrz.Client — HTTP/XML client against
  https://xmldata.qrz.com/xml/current/. Holds the session key in an
  Agent, transparently re-logs-in on :session_expired, and parses
  responses via xmerl.
- Microwaveprop.Qrz.Callsign — Ecto schema for the qrz_callsigns
  cache table, binary_id primary key per project convention.
- Microwaveprop.Qrz.Record — slim struct with only the 11 fields
  we actually consume (identity, name, grid, address, lat/lon).
  The full XML payload stays in the raw :data jsonb column for
  anyone who wants the other ~40 QRZ fields.
- Microwaveprop.Geocoder — Req-based client against the Google Maps
  Geocoding API. Only called as a fallback when QRZ has no explicit
  <lat>/<lon> for the callsign.
- Microwaveprop.CallsignLocation — orchestrator. Reads the
  callsign_locations cache, on miss calls Qrz then either uses QRZ's
  coords directly or geocodes the formatted address, snaps to an
  8-char Maidenhead grid via Microwaveprop.Radio.Maidenhead, and
  upserts the result.

Microwaveprop.Radio.CallsignClient.locate/1 is rewritten to delegate
to CallsignLocation.lookup/1 and shape the response back to the
existing {:ok, %{callsign, gridsquare, lat, lon}} contract so the
callers in PathLive and RoverLive don't change.

Wiring:

- priv/repo/migrations/20260413000000_create_qrz_callsigns_and_callsign_locations.exs
  creates qrz_callsigns and callsign_locations with unique indexes
  on :callsign.
- Microwaveprop.Qrz.Client added to the application supervision tree
  so the session Agent is started.
- :xmerl added to extra_applications so the release bundles it.
- config/test.exs wires Req.Test plugs for Microwaveprop.Qrz.Client
  and Microwaveprop.Geocoder, forces cache_ttl_hours: 0 so the cache
  never short-circuits test-level stubs, and supplies dummy QRZ
  credentials.
- config/runtime.exs pulls QRZ_USERNAME / QRZ_PASSWORD / QRZ_AGENT
  and GOOGLE_API_KEY from the environment so prod and dev can
  configure both upstream keys out of band.

Tests (ported verbatim from gridmap-web):

- test/microwaveprop/qrz/callsign_test.exs — schema/changeset
- test/microwaveprop/qrz_test.exs — cache hit, cache miss, TTL
  behavior, upsert on stale, error passthrough, case-insensitive
  input
- test/microwaveprop/geocoder_test.exs — success, zero results,
  request denied, transport error
- test/microwaveprop/callsign_location_test.exs — end-to-end flow
  including the QRZ-lat/lon shortcut and the missing-address error
  path

All 1294 tests still pass. Credo strict clean.
2026-04-12 17:43:29 -05:00
f733ddb6ac
Convert /contacts list to live_table
Replaces the hand-rolled sort/search/pagination on /contacts with
LiveTable.LiveResource. Sortable columns for station1/2, grid1/2,
band, mode, distance_km, qso_timestamp, and inserted_at; searchable
across both stations, both grids, and mode. Custom renderers preserve
the enrichment summary badge (with per-status tooltip), the invalid
flag column, and the qso/inserted_at formatters.

The reciprocal-grouping block has been dropped per the brainstorming
decision — every QSO now shows as its own row, which avoids fighting
live_table's one-row-per-record model. A "View" action links to the
detail page instead of the previous row-click handler (live_table's
stream dom_ids are random UUIDs so row-click by record id is no
longer practical without forking the library).

Test updates:
- sort test switched to ?sort_params[station1]=asc (live_table URL
  format; the old ?sort_by/?sort_order params are gone)
- pair-search test removed since live_table does a single ILIKE
  across searchable fields and no longer supports the two-callsign
  intersection that Radio.list_contacts/1 used to do; single-term
  search is exercised instead
- pagination test asserts the presence of "Page" (live_table's
  pagination text is "Page N" without the total count)

Also converts the stray cond-with-single-clause in
UserManagementLive.Index's delete handler to if/else (credo fix).
2026-04-12 17:22:15 -05:00
606340e1c7
Convert /admin/contact-edits list to live_table
The pending-edits list on the admin review page now sorts through
live_table. Columns for contact, submitted-by user, changed-fields
summary, and submitted timestamp are rendered via 2-arity custom
renderers so the Ecto-preloaded :contact and :user associations stay
accessible — live_table's data_provider path skips select_columns so
the full struct (with preloads) reaches the renderer.

The review drawer, diff table, approve/reject flow, and note textarea
all stay as-is; after approve/reject the page push_patches back to
its current path so live_table re-runs handle_params and the list
refreshes.

Pending count in the header now reads Radio.pending_edit_count/0
instead of length(@edits) since we no longer hold the full list in
socket state.
2026-04-12 17:18:47 -05:00
f9264908db
Convert /beacons approved list to live_table
Replaces the stream-backed static table for approved beacons with
LiveTable.LiveResource, giving users column sorting and full-text
search across callsign and grid. Custom renderers preserve the
frequency/EIRP formatting, keying label, and on-air badge so the
visual output matches the previous daisyUI cells.

A new Beacons.approved_beacons_query/0 returns the base Ecto query
(`where: approved == true`), assigned onto socket.assigns.data_provider
in mount so live_table's handle_params uses it as the root query and
applies sort/search/paginate on top. The leaflet map above the table
keeps its existing phx-hook+phx-update=ignore setup, and the pending-
approval admin section stays as a stream-backed `<.table>` — live_table
is single-resource-per-liveview and the pending list is a distinct
admin concern anyway.

PubSub updates (beacon created/updated/deleted) now push_patch to the
current path so live_table re-runs handle_params and the table
refreshes in place.
2026-04-12 17:15:05 -05:00
67d64109e6
Contact detail link to open in Path Calculator
Adds an "Open in Path Calculator" action to the contact detail
header. The link navigates to /path with source/destination prefilled
(preferring Maidenhead grids over station callsigns, since grids
are exact and callsigns go through a QRZ + geocoder round-trip)
and the contact's band selected. PathLive's handle_params already
auto-runs the calculation when source + destination are present,
so the user lands straight on the computed path ready to tweak
TX power, antenna heights, and gains.
2026-04-12 17:13:05 -05:00
bcfda54897
Assume 10 ft AGL antennas on contact detail path analysis
TerrainAnalysis.analyse was being called with default 0 m antenna
heights on both ends of a QSO's path, which makes the knife-edge
diffraction math pretend the radios are sitting directly on the
terrain surface. Pass 3.048 m (10 ft) for both ant_ht_a and ant_ht_b,
and shift the forward/reverse elevation angles to match. This is
only a display assumption on /contacts/:id — terrain worker paths
computed elsewhere are unaffected.
2026-04-12 17:11:49 -05:00
be9c215515
Tooltip explaining callsign-based destination lookup
Hovering the dotted-underlined "callsign" in the Destination label
on /path now surfaces a tooltip explaining that callsign lookup
queries QRZ for the licensee's mailing address and geocodes it, so
the resulting position may differ from where the station actually
operates. Gives users a reason to enter a grid directly when they
know it's more accurate than a QRZ address.
2026-04-12 17:07:56 -05:00
da6b312213
Add live_table dep and convert /users to use it
Wire up gurujada/live_table 0.4.1 as a dependency (alongside its
transitive sutra_ui and optional igniter requirement) with the
:live_table config pointing at our Repo and PubSub. Forced oban_web
override so the path-pinned vendored copy still wins over live_table's
hex constraint.

Convert the admin /users page to use LiveTable.LiveResource with
sortable, searchable columns for callsign, name, and email. Custom
renderers preserve the daisyUI admin badge and the pending/confirmed
date cell. Hidden id field stays in the query so the edit and delete
actions can target a record by id.

Updated the delete test to push the delete event directly with an id
payload; the stream dom_ids under live_table are random UUIDs so
tr#users-<id> selectors no longer work.
2026-04-12 17:07:07 -05:00
f3d75373dd
Admin-only nav dropdown
Collapse the three admin-only links (Users, Contact edits, Oban) into
a single "Admin" dropdown in the horizontal nav so the top bar isn't
cluttered with admin controls for the one admin account. Non-admins
see nothing extra.
2026-04-12 17:07:07 -05:00
7463cdcc7b
Cap "Contacts I'm in" at 100 most recent
The profile page's involving-contacts query now limits to the 100
most recent rows (ordered by qso_timestamp desc). A small note is
shown under the heading when the cap is hit so users know there's
more data they're not seeing.
2026-04-12 17:04:11 -05:00
47b104e3cf
Full-row click navigation in "Contacts I'm in"
Clicking anywhere in a row on the /u/:callsign involving-contacts
table now navigates to that contact's detail page, matching the
click affordance on the main /contacts list.
2026-04-12 16:58:15 -05:00
1013df6858
"Contacts callsign is in" section on /u/:callsign
Add a new card on the profile page listing every contact where the
callsign appears as either station1 or station2, case-insensitive,
newest first. Backed by a new `Radio.list_contacts_involving_callsign/1`
query with unit coverage for station1/station2 matching, case handling,
ordering, and blank input.
2026-04-12 16:41:28 -05:00
ad041ccf4a
Nav reorder + path calc link from beacon detail
Move Scoring Algorithm directly above About in map and weather-map
sidebars; reorder the horizontal nav so Submit sits before Contacts
to match.

Add a "Plot path to beacon" button on the beacon detail page that
navigates to /path with destination, band (snapped via BandResolver),
and dst_height_ft prefilled from the beacon.
2026-04-12 16:33:09 -05:00
810f5a22ce
Beacon list map, profile page polish, nav ordering fix
/beacons gains a Leaflet map above the table showing every approved
beacon as a circle marker with a popup that links to the detail page.
New BeaconsListMap JS hook (assets/js/beacons_list_map_hook.ts) walks a
data-beacons JSON payload, fitBounds over the marker set, and mirrors
the format_freq/1 integer-comma formatting used on the detail view so
the popups match the rest of the UI. Approved/on-air beacons render
green, off-air beacons render gray. BeaconLive.Index.mount/3 now
materializes the list into an assign so it can encode it into the data
attribute; the live PubSub path keeps the map in sync on
create/update/delete.

Profile (/u/:callsign) polish: drop the hero-radio, hero-signal, and
hero-information-circle icons per request, and replace the daisyUI
`avatar placeholder` wrapper with a plain flex-centered circle so the
initial letter actually lives inside the circle instead of anchored to
the top-left corner (daisyUI 5 removed the old placeholder-centering
behavior).

Navigation ordering: in the top navbar (layouts.ex) Contacts now sits
immediately before Contact Map instead of after it. All three vertical
sidebars already had them adjacent in the correct order, so only the
horizontal nav changed.
2026-04-12 16:23:09 -05:00
fc245367e3
User profiles at /u/:callsign, flexible band input, assorted UX cleanup
Profile page:
  * New MicrowavepropWeb.UserProfileLive at /u/:callsign is a public
    page showing a user's contacts and beacons. Resolves case-
    insensitively so /u/w5isp and /u/W5ISP are the same thing; unknown
    callsigns redirect to /. Uses daisyUI card / stats / table
    components with an avatar-placeholder initial and hero icons.
  * Accounts.get_user_by_callsign/1 (case-insensitive) plus
    Radio.list_contacts_for_user/1 and Beacons.list_beacons_for_user/1
    back the page. Beacons list includes both approved and pending so
    owners see their drafts.
  * The top nav bar and the three LiveView sidebars (MapLive,
    WeatherMapLive, ContactMapLive) now render the logged-in callsign
    as a navigate link to /u/:callsign instead of a static label.
  * Nine new tests cover the lookup, the LiveView render, and the
    ownership-scoped queries.

Flexible band input:
  * New Microwaveprop.Radio.BandResolver module converts any of:
    ADIF wavelength labels ("33cm", "1.25cm", "6mm", case/whitespace
    insensitive), numeric frequency strings ("903.100", "10368.000"),
    and canonical MHz integers into the one of the site's known bands.
    Returns the nearest allowed band for numeric inputs >= 900 MHz,
    nil otherwise.
  * 902 MHz is added to Contact.@allowed_bands, ContactEdit.@allowed_bands,
    AdifImport.@allowed_bands, and the BandResolver list so "33cm"
    round-trips end-to-end.
  * AdifImport and CsvImport now delegate band resolution to
    BandResolver, and Radio.create_contact/2 normalizes the :band attr
    on the way in so the manual form and any API callers benefit too.
    CsvImport's "invalid band" tests previously used 99999 MHz which
    the new resolver snaps to the nearest allowed band; swapped to
    "notaband" which is truly unresolvable.

Contacts and beacons list UX:
  * Remove the "Submitted" column from /contacts — it duplicated info
    already visible on the detail page and was pushing the real
    columns off narrow viewports. submitted_cell/1 and its three
    column-specific tests go with it.
  * Hide the Lat / Lon columns from /beacons — six decimal places of
    coordinates weren't useful next to the grid square and took a
    disproportionate amount of row width.
2026-04-12 16:13:08 -05:00
eedd91c6ee
Fix /map prod crash on reconnect and redraw reach polygon on band change
Two bugs on the propagation map:

1. LiveStash reconnect path crashed the LiveView with KeyError on
   :initial_scores_json. stash_assigns/2 only persists selected_band and
   selected_time, but the recovered-socket branch returned early without
   rebuilding any of the other mount-time assigns (bands, bounds, the
   initial score JSON payload, grid/radar toggles, antenna height).
   Users on the Phoenix longpoll fallback — or anyone whose websocket
   reconnected after a brief network blip — saw a blank page and a
   server-side 500. Refactor mount/3 to always compute ephemeral assigns,
   using recovered selected_band / selected_time when present and
   falling back to defaults otherwise.

2. The propagation reach polygon shown after clicking a point never
   redrew when the user changed bands or scrubbed the forecast timeline.
   The redraw logic only ran inside the point_detail handler and was
   gated on `hull.length >= 3`, so the stale polygon from the old band
   stuck around whenever the new band had no coverage at the clicked
   point. Extract a redrawReachPolygon/0 helper on the Leaflet hook that
   always clears the previous polygon first and pulls the tier color
   from the current score at the clicked location, then call it from
   update_scores (band change + server time scrub) and the cached
   timeline-scrub path. The point_detail handler delegates to the same
   helper so the polygon stays in sync with both the score grid and the
   detail panel.
2026-04-12 15:42:45 -05:00
85036eff21
Add toggleable weather radar overlay, disable rain-scatter display
New "Weather radar" toggle in /map's control panel (mobile + desktop)
layers an ECCC GeoMet WMS composite dBZ mosaic over the propagation
heatmap. The Radar_1km_dBZ-Extrapolation layer covers CONUS + Canada at
1 km resolution, updated every ~6 minutes. Tiles are lazy-created on
first toggle so visitors who never open the panel never hit the WMS
service. Refresh is driven by a setInterval that calls setParams with a
cache-busting timestamp, matching ECCC's publish cadence.

Rain-scatter display (NEXRAD cell markers + "Rain Scatter" section in
the point-detail panel) is disabled for now. The live radar overlay
answers "is there rain near this point?" more directly, and the scatter
propagation feature needs more work before it's useful. The Elixir
start_async, handle_async, and fetch_rain_scatter helpers are removed.
The JS drawScatterMarkers / buildScatterBlock paths remain as harmless
dead code since they're guarded by a `rain_scatter` field the server no
longer sends — easy to re-enable later by restoring the payload key.
2026-04-12 15:36:29 -05:00