Commit graph

387 commits

Author SHA1 Message Date
FluxCD
d11ffd218d chore: update prop image to git.mcintire.me/graham/prop:main-1775588205-54b9a1f [skip ci] 2026-04-07 18:58:17 +00:00
15ff056a2a Keep source=gps in URL when GPS is used, single copy link, fix unicode
- GPS button sets source_is_gps flag, URL stays source=gps
- Form shows resolved coordinates but URL preserves gps intent
- Single "Copy Link" button copies current URL (includes gps if used)
- Fix unicode escapes in HEEx: use Elixir interpolation for subscripts
2026-04-07 13:56:37 -05:00
FluxCD
44d0c3d1ed chore: update prop image to git.mcintire.me/graham/prop:main-1775587971-67fe71f [skip ci] 2026-04-07 18:54:16 +00:00
1b0b55358b Improve path page UX: prominent GPS button, share links
- "My Location" button with text label (was just an icon)
- Two share buttons after results:
  - "Copy Link" — fixed coordinates, exact result
  - "Copy Link (use viewer's GPS)" — source=gps, dynamic per viewer
- CopyLink JS hook copies full URL to clipboard with "Copied!" feedback
2026-04-07 13:52:43 -05:00
FluxCD
ed6a378f3e chore: update prop image to git.mcintire.me/graham/prop:main-1775587689-be8e23c [skip ci] 2026-04-07 18:49:14 +00:00
f1e51d679a Support source=gps in URL for dynamic GPS-based bookmarks
Bookmarking /path?source=gps&destination=K5TR&band=10000 will use
the viewer's device GPS to resolve the source location on load.
The LocateMe hook handles both button clicks and server-pushed
request_gps events. Once coords are resolved, the URL is updated
with actual lat,lon and calculation runs automatically.
2026-04-07 13:48:01 -05:00
FluxCD
c647221a45 chore: update prop image to git.mcintire.me/graham/prop:main-1775587563-40c1bb3 [skip ci] 2026-04-07 18:47:13 +00:00
4ec4cd348b Add forecast timeline, shareable URLs, and real-time form updates to /path
- 18-hour propagation forecast bar chart from grid scores at path midpoint
  Shows best/worst score and time, color-coded bars
- URL query params: all form fields reflected in URL, auto-calculates
  when source+destination present. Bookmarkable/shareable links.
- phx-change on form updates assigns in real-time as user types
- Removed LiveStash in favor of URL-based state persistence
2026-04-07 13:45:57 -05:00
FluxCD
069af0103b chore: update prop image to git.mcintire.me/graham/prop:main-1775587147-44fef0d [skip ci] 2026-04-07 18:40:12 +00:00
1bd893992b Add station parameters, GPS locate, and detailed budgets to /path
- TX/RX height in feet (converted to meters for terrain analysis)
- TX power in dBm with mW equivalent display
- TX gain and RX gain (dBi) inputs
- Power budget: TX power + TX gain - feed loss = EIRP - path loss
  + RX gain = RX power vs CW sensitivity = margin
- Loss budget: FSPL + gaseous (O2+H2O) + diffraction + rain
  with per-km gas rate detail
- GPS "Locate Me" button using browser geolocation API
- Lat,lon coordinate pairs accepted as input
- Weather displayed in F with C below
- LiveStash preserves form inputs across page reloads
- Terrain cross-section with beam line and Fresnel zone
2026-04-07 13:38:59 -05:00
FluxCD
942794149d chore: update prop image to git.mcintire.me/graham/prop:main-1775586313-259b9cd [skip ci] 2026-04-07 18:26:04 +00:00
95351a2358 Add /path calculator page
LiveView page for analyzing microwave propagation between two points.
Features:
- Source/destination input: callsign (via gridmap.org API) or Maidenhead grid
- Band selector for all 8 microwave bands (10-241 GHz)
- Leaflet map showing path line (reuses ContactMap hook)
- Terrain elevation cross-section (reuses ElevationProfile hook)
- Link summary: distance, bearing, terrain verdict
- Loss budget: FSPL + O2 + H2O + rain + diffraction
- Propagation score with 10-factor breakdown
- Current atmospheric conditions from path-integrated HRRR

New CallsignClient for gridmap.org location lookup.
Navigation links added to main navbar and map control panel.
2026-04-07 13:25:04 -05:00
FluxCD
577313a39e chore: update prop image to git.mcintire.me/graham/prop:main-1775583477-ad44acb [skip ci] 2026-04-07 17:38:47 +00:00
0977d041b8 Document ERA5, RTMA, and path-integrated HRRR in algo.md
Add sections for all three new data sources in Meteorological
Foundations: path-integrated scoring strategy, ERA5 reanalysis
for pre-2014 contacts, RTMA 15-minute surface analysis. Update
calibration dataset summary with ERA5/RAOB/RTMA coverage.
2026-04-07 12:37:51 -05:00
FluxCD
9b053c15d2 chore: update prop image to git.mcintire.me/graham/prop:main-1775583149-7ccbe77 [skip ci] 2026-04-07 17:33:46 +00:00
fbf24622fa Fix ERA5 CDS API endpoint for new v1 API
- Submit: /api/retrieve/v1/processes/{dataset}/execution
- Request body wrapped in {"inputs": ...}
- Poll: /api/retrieve/v1/jobs/{jobId}
- Status field: "status" not "state", values: accepted/running/successful/failed
- Download: GET {job_url}/results, follow asset href link
- Add ERA5 credentials to k8s secret manifest
2026-04-07 12:32:21 -05:00
FluxCD
fc65e2f731 chore: update prop image to git.mcintire.me/graham/prop:main-1775582667-e47c6ac [skip ci] 2026-04-07 17:25:45 +00:00
e79e65db43 Add ERA5 to backfill dashboard and enrichment pipeline
- ERA5 checkbox on /backfill page alongside HRRR/weather/terrain/IEMRE
- BackfillEnqueueWorker handles era5 type (targets hrrr_status=unavailable)
- ContactWeatherEnqueueWorker.build_era5_jobs/1 creates ERA5 fetch jobs
  for path points without existing ERA5 profiles
- Progress bar shows ERA5 candidate count (HRRR-unavailable contacts)
- Status panel shows ERA5 candidates and fetched profile count
2026-04-07 12:24:18 -05:00
ddf86ed806 Add mix era5_backfill task and gitignore .envrc
Enqueues ERA5 fetch jobs for contacts where HRRR is unavailable.
Deduplicates by rounded grid point and hour, skips existing profiles.

Usage: mix era5_backfill [limit]
2026-04-07 12:24:18 -05:00
FluxCD
7df636147a chore: update prop image to git.mcintire.me/graham/prop:main-1775581463-2286725 [skip ci] 2026-04-07 17:05:41 +00:00
dea407c8ca Add ERA5 reanalysis and RTMA data sources
ERA5 (Copernicus CDS API):
- Era5Profile schema matching HRRR profile structure for interop
- Era5Client with async job submission, polling, GRIB2 download
- Era5FetchWorker (Oban queue: era5, max_attempts: 5)
- Unified lookup: Weather.best_profile_for_contact/1 tries HRRR
  first, falls back to ERA5 for pre-2014 contacts

RTMA (NOAA S3, 2.5km/15-min):
- RtmaObservation schema for surface-only fields
- RtmaClient with byte-range GRIB2 requests (same pattern as HRRR)
- RtmaFetchWorker (Oban queue: rtma, max_attempts: 10)
- Weather.find_nearest_rtma/3 for surface condition lookup

Both queues added to production Oban config (2 workers each).
2026-04-07 12:04:16 -05:00
7652e41971 Path-integrated HRRR scoring across pos1/mid/pos2
Contact scoring now uses all HRRR profiles along the path instead of
just pos1. Aggregation strategy:
- Best along path for beneficial factors (refractivity, pressure)
- Worst along path for harmful factors (rain, wind)
- Average for neutral factors (temp, dewpoint, PWAT, BL depth)

Scorer.path_integrated_conditions/2 merges multiple profiles into a
single conditions map. Falls back gracefully to single-profile scoring
when only one profile is available.
2026-04-07 12:04:16 -05:00
FluxCD
4c848263de chore: update prop image to git.mcintire.me/graham/prop:main-1775580351-12b8bdb [skip ci] 2026-04-07 16:47:37 +00:00
2eefe64c9b Update seasonal scores from sounding data, document commercial link patterns
Soundings (10,935): October has strongest gradients (-350 N/km) and
65.7% ducting, raised seasonal score 82→88. February ducting (20.2%)
exceeds January (14.8%), raised 32→40.

Commercial links (12,068 samples): 68 GHz shows 3.9 dB diurnal swing,
11 GHz shows 1.7 dB with inverted multipath pattern, 24 GHz remarkably
stable at 0.9 dB. Diurnal sensitivity is non-monotonic with frequency.

RAOB gradients avg -265 N/km vs HRRR -107 (2.5x stronger) — confirms
HRRR misses thin surface ducts that soundings resolve.
2026-04-07 11:45:44 -05:00
FluxCD
b86e0359ea chore: update prop image to git.mcintire.me/graham/prop:main-1775578557-be70d39 [skip ci] 2026-04-07 16:18:31 +00:00
3cdd132f24 Data-driven algorithm refinements from full dataset analysis
Analyzed 37,925 HRRR-matched contacts from 58,367 total.

- Remove shallow BL bonus (score 82 for HPBL < 300m): data shows
  medium BL (1000-2000m) produces longest contacts (222 km avg),
  not shallow (210 km). Refractivity fallback now uses default score.

- Refine pressure scoring bins: add <980 mb tier (score 88), steeper
  gradient from low to high. Contacts at <970 mb avg 242.7 km vs
  184.1 km at 990-1000 mb.

- Update algo.md calibration stats (58,367 contacts, 41M HRRR
  profiles, 58,361 terrain profiles) and document dataset bias
  (99.5% Aug-Sep).

- Create updates.md with full binned analysis tables for all factors.
2026-04-07 11:15:49 -05:00
511ae09fea Fix livebook path resolution in bin/notebook
Falls back to mix escripts directory when livebook isn't in PATH.
2026-04-07 11:15:49 -05:00
3cc4301855 Auto-deploy analysis notebook as Livebook app
Add app_settings metadata to notebook and set LIVEBOOK_APPS_PATH
so it shows up under Apps on the Livebook home page.
2026-04-07 11:15:49 -05:00
548e277bec Add bin/notebook script to start Phoenix + Livebook together
Starts Phoenix with a named node and Livebook attached to it on
port 8081. Single command, Ctrl-C stops both.
2026-04-07 11:15:49 -05:00
44f0ff26e5 Add mix notebook task for Livebook attached to dev node
Run `mix notebook` for instructions to start Livebook at localhost:8081
connected to the running Phoenix node with full Repo/Scorer access.
2026-04-07 11:15:49 -05:00
6252aff728 Rewrite analysis notebook to attach to running node
Uses Repo, Scorer, BandConfig, Weather directly instead of raw SQL.
Scores all historical contacts with the real composite_score/2 and
includes factor heatmaps and weight sensitivity analysis.
2026-04-07 11:15:48 -05:00
7f0e1f4182 Add Livebook notebook for propagation algorithm analysis
Connects to local dev DB to analyze contacts, HRRR conditions,
terrain impact, commercial link correlation, and score sensitivity.
Ready for use once prod data is imported locally.
2026-04-07 11:15:48 -05:00
FluxCD
f8f2450ec1 chore: update prop image to git.mcintire.me/graham/prop:main-1775566243-7f457e4 [skip ci] 2026-04-07 12:51:39 +00:00
976cd1a4c5 Remove icons from navigation links in map control panels 2026-04-07 07:50:36 -05:00
a774e98350 Smooth viewshed boundary to remove SRTM elevation spikes
Apply 5-point circular moving average to reach_km values before
rendering the boundary polygon. Eliminates sharp spikes caused by
isolated terrain anomalies in SRTM data.
2026-04-07 07:42:32 -05:00
FluxCD
05fcb6f5be chore: update prop image to git.mcintire.me/graham/prop:main-1775565490-61e4196 [skip ci] 2026-04-07 12:39:36 +00:00
FluxCD
d409c190d5 chore: update prop image to git.mcintire.me/graham/prop:main-1775565429-f0340a2 [skip ci] 2026-04-07 12:38:36 +00:00
721d59ed77 Remove stale propagation scores maintenance banner 2026-04-07 07:38:03 -05:00
e03b9fbc0e Add periodic enrichment backfill cron and terrain reconciliation
Runs BackfillEnqueueWorker every 30 minutes to pick up contacts with
pending/queued/failed enrichment status and enqueue missing jobs.
Before enqueueing, reconciles terrain contacts stuck in "queued" by
checking the terrain_profiles FK directly.
2026-04-07 07:36:57 -05:00
FluxCD
8e39cfcc4c chore: update prop image to git.mcintire.me/graham/prop:main-1775513079-b5a87e0 [skip ci] 2026-04-06 22:05:15 +00:00
d63653e9a1 Bump terrain queue concurrency to 3 per pod 2026-04-06 17:04:26 -05:00
FluxCD
7c7dc80acf chore: update prop image to git.mcintire.me/graham/prop:main-1775512945-6ab4ba7 [skip ci] 2026-04-06 22:03:25 +00:00
f2e6512974 Re-enable hourly propagation grid scoring in production 2026-04-06 17:02:18 -05:00
FluxCD
c6c4214a91 chore: update prop image to git.mcintire.me/graham/prop:main-1775512477-be9b890 [skip ci] 2026-04-06 21:56:05 +00:00
79130a39e8 Mark terrain status complete when profile already exists
TerrainProfileWorker early-exited with :ok when a profile existed
but never updated the contact status, leaving contacts stuck at queued.
2026-04-06 16:54:29 -05:00
FluxCD
366258e329 chore: update prop image to git.mcintire.me/graham/prop:main-1775509847-dcf5190 [skip ci] 2026-04-06 21:11:31 +00:00
646467cf4d Store stub records on empty weather fetches to prevent infinite backfill retries
When ASOS/RAOB/IEMRE APIs return empty data, store a stub record so
dedup checks (has_surface_observations?, has_sounding?, has_iemre_observation?)
see coverage and don't recreate the same jobs on future backfill runs.
2026-04-06 16:10:39 -05:00
FluxCD
a6ae6ce973 chore: update prop image to git.mcintire.me/graham/prop:main-1775499376-15ec8af [skip ci] 2026-04-06 18:17:26 +00:00
c3819112fe Persist checkbox and limit state across backfill enqueue submissions 2026-04-06 12:46:08 -05:00
FluxCD
ef44109351 chore: update prop image to git.mcintire.me/graham/prop:main-1775494893-2fd0175 [skip ci] 2026-04-06 17:02:37 +00:00