Commit graph

305 commits

Author SHA1 Message Date
44b169b234 CSV upload preview with duplicate detection before commit
Uploads now show a review page with three summary cards (valid,
duplicates, invalid), tables for invalid and duplicate rows, and a
sample of the first 20 valid rows. Nothing is written to the database
until the user clicks "Looks good — submit N contacts".

Duplicate detection treats two rows as the same contact if they share
the same pair of (callsign, grid) tuples (direction-agnostic), the
same band, and timestamps within 60 minutes. Dedup runs against both
existing DB contacts (single band/time-scoped query) and earlier rows
in the same upload, and the preview labels which side matched.

CsvImport grows preview/2 + commit/1; the old import/2 is kept as a
thin backward-compat wrapper that skips dedup. Added 11 new CsvImport
tests and reworked the SubmitLive CSV tests for the two-step flow.
2026-04-09 12:30:10 -05:00
664f1353db Decouple propagation_scores pruning from the compute worker
Pruning used to only run at the end of a successful PropagationGridWorker
pass, so a stretch of failed compute jobs (k8s OOM kills, SIGTERM)
stopped prune from running and let the table accumulate ~5h of stale
rows. A dedicated PropagationPruneWorker now runs every 15 minutes on
its own Oban cron, and PropagationGridWorker also calls prune_old_scores
at the start of each run as a second safety net. Bumped the delete
timeout from 2m to 5m so the first catch-up pass has enough headroom.
2026-04-09 12:30:10 -05:00
442d1d1bdc Add /privacy page with shared footer across non-map pages
New PrivacyLive page explains that collected/submitted data is used
only for propagation research and will never be sold, traded, or
disclosed. Layouts.app now renders a small footer with the NTMS
attribution and a Privacy link on every page that uses it, so the
full-screen map pages are untouched. The duplicate NTMS credit on
the About page is removed since it now lives in the shared footer.
2026-04-09 09:54:54 -05:00
abe0aa9ec6 Pin referrer policy so OSM tile requests always send a Referer
OSM's tile usage policy requires a Referer header identifying the site.
strict-origin-when-cross-origin is the current browser default and sends
the origin on HTTPS→HTTPS cross-origin requests; pinning it explicitly
prevents silently losing the header if a browser default ever changes.
2026-04-09 09:30:23 -05:00
a95d2ada52 Compact beacon detail page, toggleable coverage, About page formatting
- Beacon map defaults to just the marker at zoom 11. A daisyUI toggle
  switch shows/hides the estimated-coverage cell layer; the coverage
  info line and tier legend are hidden unless the toggle is on.
- Beacon detail list replaced with a compact dl grid (2-4 columns).
  Lat/lon rounded to 5 decimals, power/height/beamwidth formatted via
  Beacon.format_mw/1 so no more "2.5e3" scientific notation or "280.0".
  format_mw/1 hoisted onto the schema so index and show share it.
- About page no longer uses prose classes (this project doesn't load
  @tailwindcss/typography). Headings, lists, and paragraphs now use
  explicit utility classes so they render as intended.
2026-04-09 09:12:32 -05:00
fbae931d7e Add beacon bearing, beamwidth, and weak-signal keyings
Beacons can now record a directional bearing ("omni" or 0-360 degrees)
and a beamwidth in degrees. The keying list grows from {on_off, fsk}
to include FM Voice, WSPR, and the full Q65A/B/C/D/E × 15/30/60/120
family, grouped via optgroups in the submission form.
2026-04-09 08:59:16 -05:00
e327ff584e Map UX polish and populated About page
- /map now centers on the visitor's Cloudflare geolocation when the
  cf-iplatitude/cf-iplongitude headers are present, falling back to DFW.
  Initial HRRR score bounds track the chosen center.
- Added /about link to the /map side navigation.
- Renamed "Propagation Map" label to "Propagation Prediction Map" in
  the /map page title, sidebars, and the /weather back-link.
- Populated /about with mission, approach, stack, roadmap, and a live
  stat grid fed from DB counts (estimates via pg_class.reltuples for
  the multi-million-row tables).
2026-04-08 17:06:53 -05:00
d1ec45b6d3 Move live_dashboard to /admin/dashboard, admin-only 2026-04-08 16:25:19 -05:00
ba0f1161a7 Beacon improvements: anon submit, notes, /map nav, admin backfill
- Anonymous users can submit beacons (held pending admin approval)
- Added notes field to beacons (textarea on form, shown on detail page)
- Added Beacons link to /map sidebar nav (mobile + desktop),
  removed stale Rover Planner link
- Moved /backfill to /admin/backfill under the admin live_session
- Beacon detail map zooms in two extra levels after fitBounds
2026-04-08 16:23:59 -05:00
f5b5af171e Add beacon keying field (on/off or FSK) 2026-04-08 16:13:35 -05:00
6476bc8045 Dedupe ERA5 fetch jobs by (lat, lon, valid_time)
The Era5FetchWorker now declares an Oban unique constraint on
{lat, lon, valid_time} for any job in available/scheduled/executing/
retryable, so two backfill runs targeting the same contact grid point
can no longer spawn parallel CDS requests for the same hour.

Because Oban OSS insert_all doesn't honor unique, ERA5 jobs are now
routed through Oban.insert/1 from ContactWeatherEnqueueWorker and the
era5_backfill mix task. Other worker types still use insert_all.
2026-04-08 15:52:53 -05:00
80f2725cd5 Beacon submission approval workflow
Non-admin users can now submit beacons but they are held in an
unapproved state until an admin approves them. Only approved beacons
appear in the public list; pending submissions are shown in a separate
admin-only section on /beacons with Approve and Delete actions. The
show page surfaces a pending badge and an admin-only Approve button
when viewing an unapproved beacon.

Also formats EIRP (mW) on the index page without scientific notation.
2026-04-08 15:42:32 -05:00
f8763feb38 Add About page, hide Rover from menu 2026-04-08 15:42:32 -05:00
cda54e3514 Per-HRRR-cell beacon reception plot
Replace the idealised concentric-circle range rings with a realistic
per-HRRR-grid-cell reception map. For every 0.125° grid point within
the band's exceptional range, the estimator computes great-circle
distance, FSPL + atmospheric loss, and applies a cell-specific score
adjustment of (50 - score) * 0.3 dB — score 100 gives a 15 dB ducting
boost, score 0 a 15 dB penalty. Cells below the -145 dBm detection
floor are dropped.

The beacon map hook now renders each surviving cell as a 0.125°
filled rectangle with a tooltip showing tier, Rx dBm, distance, and
HRRR score, so the coverage footprint bulges where ducting conditions
are good and cuts off where they aren't, instead of being a perfect
circle. Falls back to score 50 for cells that don't have HRRR data
yet, so the plot is always useful.

Also fixes the prior all-grey-tiles regression by restoring an
explicit setView() at map creation and adding a post-mount
invalidateSize() for when the map is placed inside a flex container.
2026-04-08 13:43:01 -05:00
67e095bd47 Label beacon TX power as EIRP
The power_mw field represents the beacon's effective radiated power,
not just transmitter output. Relabel it as "TX power (EIRP)" on the
form and detail list, "EIRP (mW)" in the index, and drop the now-
meaningless tx_gain_dbi constant in RangeEstimate since the stored
value already includes antenna gain.
2026-04-08 13:32:32 -05:00
a5b3f1f3da Beacon detail map with range estimate and on_the_air flag
- Add on_the_air boolean to beacons (default true); surfaced as a
  checkbox on the form, a badge on the index, and in the detail list.
- Render a Leaflet map on the beacon show page with a marker at the
  beacon's lat/lon tooltipped with callsign + frequency. Marker is
  green when on air, gray when off.
- Compute and draw a reception-range estimate as concentric signal-
  strength rings. New Microwaveprop.Beacons.RangeEstimate solves a
  link budget (FSPL + O2/H2O absorption from BandConfig) at five RX
  thresholds (-100 to -145 dBm), then scales by 0.5 + score/100 from
  the latest Propagation.point_detail at the beacon's grid square, so
  current HRRR conditions shift the rings in or out.
- Re-enable the hourly PropagationGridWorker cron and freshness
  monitor in dev.exs so dev actually has HRRR-backed scores to feed
  the new estimator.
2026-04-08 13:29:58 -05:00
374f9b106a Beacon height in feet, auto-fill lat/lon from grid
- Rename beacons.height_m to height_ft; migration converts existing
  values (m * 3.28084). Form/list/show labels updated.
- Beacon changeset now fills lat/lon from the grid when they're blank
  (previously only the reverse worked). Lat/lon and grid inputs are no
  longer marked required in the form — enter one side and the other
  populates on blur via phx-change.
2026-04-08 12:55:10 -05:00
b656e015bf Beacon power in mW, settings tweaks
- Rename beacons.power_watts to power_mw (migration multiplies existing
  values by 1000); form/list/show all relabeled to "Power (mW)"
- Fix Add-monitor button alignment on /users/settings by rendering the
  label above and putting the input and button in a plain flex row so
  the input wrapper margin no longer offsets the button
- Extend the settings page re-auth window from 10 minutes to 24 hours
2026-04-08 12:45:35 -05:00
e4fb422402 Dock map controls to right sidebar with collapsible panel
- Move controls (band selector, grid toggle, nav links) to a docked
  right sidebar on desktop, keep floating overlay on mobile
- Point detail panel floats over the map bottom-left instead of inside
  the narrow sidebar
- Add collapse/expand toggle for the sidebar
- Add auth links (login/logout/settings) to sidebar
- Wrap public live routes in live_session with UserAuth on_mount
- Use daisyUI menu lists for sidebar navigation
2026-04-08 12:35:00 -05:00
ee9275e0b9 Add /beacons CRUD and /users admin page
Beacons:
- Scaffolded with phx.gen.live then reworked so reads are public
  and mutations go through a live_session gated by the new
  :require_admin on_mount hook in UserAuth
- Beacon schema stores frequency (MHz), callsign, grid, lat/lon,
  power (W), and height above ground (m); grid auto-derives from
  lat/lon when left blank via Maidenhead.from_latlon
- Adds Maidenhead.from_latlon/3 so we can compute grids locally
  instead of hitting an external API

Users admin page:
- /users and /users/:id/edit (admin-only) for listing, editing
  (callsign/name/email/is_admin), and deleting other users
- Adds Accounts.list_users, admin_update_user, delete_user, and
  a dedicated admin_changeset on the User schema
- Nav gains a "Users" link for admins and a "Beacons" link for
  everyone
2026-04-08 12:01:45 -05:00
eaef5b0178 Add is_admin flag, auto-grant to graham@mcintire.me
Adds a boolean is_admin column (default false) and has the
registration changeset auto-set it to true when the email matches
graham@mcintire.me (case-insensitive). The migration also backfills
the flag for an existing row with that email.
2026-04-08 11:43:52 -05:00
355bed178d Add beacon monitor registration and fix prod SMTP
Users can now register beacon monitors on the settings page. Each
monitor gets a unique random token the remote program will use to
authenticate its reports. Name is the only user-supplied field for
now; more will be added as the monitor protocol is defined.

Also adds :gen_smtp to deps. Swoosh's SMTP adapter depends on
:mimemail which lives in that package, and production was 500'ing
when trying to send confirmation emails without it.
2026-04-08 11:30:28 -05:00
27170b5139 Merge auth links into main nav and lower password min to 8
- Register/Log in (or callsign/Settings/Log out) now live next to
  the Map/Path/Rover links in the main header instead of a separate
  top menu bar
- Add on_mount in UserAuth to assign current_scope on LiveView mount,
  and pass current_scope through to Layouts.app from every LiveView
- Drop the old top <ul> from root.html.heex
- Password minimum lowered from 12 to 8 characters
2026-04-08 10:39:51 -05:00
1d86e287b2 Add password auth with callsign + email confirmation
Generated Accounts context, User schema, and controllers via
phx.gen.auth. Adapted it for password-only login with required
email confirmation:

- Users have callsign (unique, uppercased), name, email, password
- Registration form fields: callsign, name, email, password, confirm
- Magic-link login path removed; login is email + password only
- After register, a confirmation email is sent and login is blocked
  until the account is confirmed via the token URL
- Confirmation link logs the user in on first use
- SMTP2GO configured as the outgoing mailer in k8s prod
2026-04-08 10:21:40 -05:00
e6aab62ea8 Show grid squares by default on rover planner 2026-04-08 10:21:40 -05:00
27deb02bc7 Add grid squares toggle to rover planner
Extract shared Maidenhead grid overlay into maidenhead_grid.js so both
the main map and rover planner use the same implementation. Wire up the
toggle_grid event and checkbox on the rover page matching the main map.
2026-04-08 09:06:09 -05:00
75fd05a6d1 Simplify rover planner: remove grid toggle and coverage calculation 2026-04-08 09:06:09 -05:00
1bee565d04 Add HRRR propagation overlay to rover planner map 2026-04-07 17:39:03 -05:00
0f885f5c13 Match rover grid toggle to main map's checkbox style 2026-04-07 17:14:11 -05:00
cfaef287e4 Add 902-5760 MHz bands, centralize band options
New BandConfig entries for 902, 1296, 2304, 3456, 5760 MHz:
- All beneficial humidity effect (like 10 GHz)
- Near-zero gaseous absorption and rain attenuation
- Ranges: 902 MHz typical 400 km, 5760 MHz typical 220 km
- Same seasonal curves as 10 GHz (ducting-driven)

BandConfig.band_options/0 generates dropdown options from configs.
All pages (path, rover, submit) use centralized band_options instead
of hardcoded lists. Map page already used BandConfig.all_bands().
10 GHz remains the default on all pages.
2026-04-07 16:39:25 -05:00
caa03fed53 Add grid toggle, use 0.25deg resolution for coverage scoring
- Toggle button for Maidenhead grid overlay (on by default)
- Coverage candidates now at 0.25° resolution (~28 km) instead of
  2°×1° Maidenhead grids, matching HRRR propagation grid density
- Coverage rectangles render as 0.25° cells for finer detail
- Grid overlay is separate from coverage coloring
2026-04-07 16:31:39 -05:00
b54de258f7 Fix rover: use length(@stations) instead of missing total_stations key 2026-04-07 16:21:47 -05:00
6aec3eeea4 Extract Coverage module, fix computation, add tests
- Extract scoring logic to Microwaveprop.Rover.Coverage for testability
- Two-phase computation: fast pass (distance + propagation) for all grids,
  then SRTM terrain analysis for top 20 candidates only
- Cap search radius to 300 km to keep candidate count reasonable
- Run terrain analysis in Task with rescue/fallback for resilience
- Background Task doesn't block LiveView process
- 5 tests covering: ranked results, empty inputs, station details,
  band range differences, field presence
2026-04-07 16:08:03 -05:00
1ccb83f2ef Rename button to Find Best Roving Locations 2026-04-07 15:53:53 -05:00
a13d5a47f2 Fix rover planner: manual Calculate button, non-blocking computation
- Add explicit "Calculate Coverage" button instead of auto-triggering
- Move coverage computation to background Task so LV stays responsive
  (users can keep adding/removing stations while it computes)
- Remove all auto-compute triggers from station add/remove/band change
- URL loading resolves stations but doesn't auto-calculate
- Button shows "Finding best roving locations..." spinner while computing
2026-04-07 15:53:09 -05:00
70521c37f2 Rover planner: auto-compute coverage areas with terrain + propagation
Rewrites rover planner to automatically color the map after stations
are entered. No manual stop selection needed — the system evaluates
every Maidenhead grid in range and scores them by:

- SRTM terrain analysis to each station (diffraction loss, LOS verdict)
- HRRR propagation forecast (ducting, refractivity, atmospheric conditions)
- Distance to stations vs band range limits
- Propagation boost: good conditions (score 80+) make blocked paths
  viable through ducting, matching the app's core beyond-LOS prediction

Coverage score formula: boosted path quality 30% + propagation 30%
+ distance 20% + station count 20%. Colored grid squares on the map
show best (green) to worst (red) operating positions.

Sidebar shows top 10 grids ranked with: workable station count,
propagation score, best operating hour, 18-hour forecast sparkline.
Click a grid for per-station detail: distance, terrain verdict,
diffraction loss.

URL stores stations + band for sharing: /rover?stations=W5ISP,K5TR&band=10000
2026-04-07 15:43:26 -05:00
43b51fdb98 Throttle GPS updates on path page to once per minute
Switch from getCurrentPosition (which looped via push_patch) to
watchPosition with a 60-second throttle. Break the server-side
request_gps loop by skipping re-request when coords already exist.
2026-04-07 14:44:34 -05:00
8a7719b953 Add /rover planner for microwave contest rovers
Interactive map-based tool for planning rover operating positions.
Rovers enter stationary stations they want to work, then click the
map to evaluate candidate operating grids.

Features:
- Add stationary stations by callsign or grid square
- Click map to add operating positions (snaps to Maidenhead grid)
- Async SRTM terrain analysis: each stop computes terrain profile
  to every station, shows CLEAR/BLOCKED verdict + diffraction loss
- Terrain lines on map: green=workable, red=blocked, dashed=marginal
- Propagation score and 18-hour forecast sparkline per stop
- Route summary: driving distance, unique grid-station pairs
- Band selector affects range limits and propagation scores
- Numbered waypoint markers with score-colored backgrounds

New files:
- Geo module (shared haversine/bearing, used by PathLive too)
- RoverLive with fullscreen map + sidebar
- RoverMap JS hook with grid overlay, station/stop markers, route line
- Nav links in header and map control panel
2026-04-07 14:17:48 -05:00
15ff056a2a Keep source=gps in URL when GPS is used, single copy link, fix unicode
- GPS button sets source_is_gps flag, URL stays source=gps
- Form shows resolved coordinates but URL preserves gps intent
- Single "Copy Link" button copies current URL (includes gps if used)
- Fix unicode escapes in HEEx: use Elixir interpolation for subscripts
2026-04-07 13:56:37 -05:00
1b0b55358b Improve path page UX: prominent GPS button, share links
- "My Location" button with text label (was just an icon)
- Two share buttons after results:
  - "Copy Link" — fixed coordinates, exact result
  - "Copy Link (use viewer's GPS)" — source=gps, dynamic per viewer
- CopyLink JS hook copies full URL to clipboard with "Copied!" feedback
2026-04-07 13:52:43 -05:00
f1e51d679a Support source=gps in URL for dynamic GPS-based bookmarks
Bookmarking /path?source=gps&destination=K5TR&band=10000 will use
the viewer's device GPS to resolve the source location on load.
The LocateMe hook handles both button clicks and server-pushed
request_gps events. Once coords are resolved, the URL is updated
with actual lat,lon and calculation runs automatically.
2026-04-07 13:48:01 -05:00
4ec4cd348b Add forecast timeline, shareable URLs, and real-time form updates to /path
- 18-hour propagation forecast bar chart from grid scores at path midpoint
  Shows best/worst score and time, color-coded bars
- URL query params: all form fields reflected in URL, auto-calculates
  when source+destination present. Bookmarkable/shareable links.
- phx-change on form updates assigns in real-time as user types
- Removed LiveStash in favor of URL-based state persistence
2026-04-07 13:45:57 -05:00
1bd893992b Add station parameters, GPS locate, and detailed budgets to /path
- TX/RX height in feet (converted to meters for terrain analysis)
- TX power in dBm with mW equivalent display
- TX gain and RX gain (dBi) inputs
- Power budget: TX power + TX gain - feed loss = EIRP - path loss
  + RX gain = RX power vs CW sensitivity = margin
- Loss budget: FSPL + gaseous (O2+H2O) + diffraction + rain
  with per-km gas rate detail
- GPS "Locate Me" button using browser geolocation API
- Lat,lon coordinate pairs accepted as input
- Weather displayed in F with C below
- LiveStash preserves form inputs across page reloads
- Terrain cross-section with beam line and Fresnel zone
2026-04-07 13:38:59 -05:00
95351a2358 Add /path calculator page
LiveView page for analyzing microwave propagation between two points.
Features:
- Source/destination input: callsign (via gridmap.org API) or Maidenhead grid
- Band selector for all 8 microwave bands (10-241 GHz)
- Leaflet map showing path line (reuses ContactMap hook)
- Terrain elevation cross-section (reuses ElevationProfile hook)
- Link summary: distance, bearing, terrain verdict
- Loss budget: FSPL + O2 + H2O + rain + diffraction
- Propagation score with 10-factor breakdown
- Current atmospheric conditions from path-integrated HRRR

New CallsignClient for gridmap.org location lookup.
Navigation links added to main navbar and map control panel.
2026-04-07 13:25:04 -05:00
fbf24622fa Fix ERA5 CDS API endpoint for new v1 API
- Submit: /api/retrieve/v1/processes/{dataset}/execution
- Request body wrapped in {"inputs": ...}
- Poll: /api/retrieve/v1/jobs/{jobId}
- Status field: "status" not "state", values: accepted/running/successful/failed
- Download: GET {job_url}/results, follow asset href link
- Add ERA5 credentials to k8s secret manifest
2026-04-07 12:32:21 -05:00
e79e65db43 Add ERA5 to backfill dashboard and enrichment pipeline
- ERA5 checkbox on /backfill page alongside HRRR/weather/terrain/IEMRE
- BackfillEnqueueWorker handles era5 type (targets hrrr_status=unavailable)
- ContactWeatherEnqueueWorker.build_era5_jobs/1 creates ERA5 fetch jobs
  for path points without existing ERA5 profiles
- Progress bar shows ERA5 candidate count (HRRR-unavailable contacts)
- Status panel shows ERA5 candidates and fetched profile count
2026-04-07 12:24:18 -05:00
ddf86ed806 Add mix era5_backfill task and gitignore .envrc
Enqueues ERA5 fetch jobs for contacts where HRRR is unavailable.
Deduplicates by rounded grid point and hour, skips existing profiles.

Usage: mix era5_backfill [limit]
2026-04-07 12:24:18 -05:00
dea407c8ca Add ERA5 reanalysis and RTMA data sources
ERA5 (Copernicus CDS API):
- Era5Profile schema matching HRRR profile structure for interop
- Era5Client with async job submission, polling, GRIB2 download
- Era5FetchWorker (Oban queue: era5, max_attempts: 5)
- Unified lookup: Weather.best_profile_for_contact/1 tries HRRR
  first, falls back to ERA5 for pre-2014 contacts

RTMA (NOAA S3, 2.5km/15-min):
- RtmaObservation schema for surface-only fields
- RtmaClient with byte-range GRIB2 requests (same pattern as HRRR)
- RtmaFetchWorker (Oban queue: rtma, max_attempts: 10)
- Weather.find_nearest_rtma/3 for surface condition lookup

Both queues added to production Oban config (2 workers each).
2026-04-07 12:04:16 -05:00
7652e41971 Path-integrated HRRR scoring across pos1/mid/pos2
Contact scoring now uses all HRRR profiles along the path instead of
just pos1. Aggregation strategy:
- Best along path for beneficial factors (refractivity, pressure)
- Worst along path for harmful factors (rain, wind)
- Average for neutral factors (temp, dewpoint, PWAT, BL depth)

Scorer.path_integrated_conditions/2 merges multiple profiles into a
single conditions map. Falls back gracefully to single-profile scoring
when only one profile is available.
2026-04-07 12:04:16 -05:00
2eefe64c9b Update seasonal scores from sounding data, document commercial link patterns
Soundings (10,935): October has strongest gradients (-350 N/km) and
65.7% ducting, raised seasonal score 82→88. February ducting (20.2%)
exceeds January (14.8%), raised 32→40.

Commercial links (12,068 samples): 68 GHz shows 3.9 dB diurnal swing,
11 GHz shows 1.7 dB with inverted multipath pattern, 24 GHz remarkably
stable at 0.9 dB. Diurnal sensitivity is non-monotonic with frequency.

RAOB gradients avg -265 N/km vs HRRR -107 (2.5x stronger) — confirms
HRRR misses thin surface ducts that soundings resolve.
2026-04-07 11:45:44 -05:00