Commit graph

139 commits

Author SHA1 Message Date
6dff6372ab
Default forecast timeline selection to closest hour to now 2026-03-31 17:45:54 -05:00
2b1fd6129d
Disable propagation grid worker and freshness monitor in dev for backfill 2026-03-31 17:27:59 -05:00
e028de51ae
Tighten HRRR grid profile retention from 48h to 24h 2026-03-31 17:20:46 -05:00
fe5228b306
Prevent duplicate PropagationGridWorker jobs via Oban unique constraint
Worker now uses unique: [period: 3600, states: [:available, :scheduled,
:executing, :retryable]] so Oban atomically prevents duplicates.
FreshnessMonitor simplified — no longer does manual oban_jobs query.
2026-03-31 17:16:40 -05:00
09a26f1167
Add Y-axis (score) and X-axis (time) labels to forecast sparkline 2026-03-31 17:13:11 -05:00
88655e40a6
Add meteorological foundations section to algo.md
Comprehensive description of atmospheric science behind the predictions
for meteorologist review: refractivity-propagation connection, four
ducting mechanisms, HRRR model usage, frequency-dependent effects,
diurnal/seasonal cycles, and known limitations.
2026-03-31 17:00:30 -05:00
741534610d
Change timeline label to 'Propagation Forecast' 2026-03-31 16:56:24 -05:00
fe150adf77
Change antenna label to 'Antenna height' 2026-03-31 16:55:40 -05:00
583bcb2b0b
Hide forecast timeline when time spread is under 1 hour 2026-03-31 16:49:30 -05:00
6145f80c22
Add forecast sparkline graph to point detail panel
- point_detail response includes forecast array (score per valid_time)
- SVG sparkline shows score trend across all forecast hours
- Trend indicator: Improving/Declining/Steady based on first vs last score
- Add covering index (band_mhz, lat, lon, valid_time) INCLUDE (score) for
  point_forecast queries
2026-03-31 16:47:52 -05:00
11af9b966f
Fix timeline labels: only closest hour shows "Now", others show offset 2026-03-31 16:46:03 -05:00
e82e631135
HRRR forecast hours (f00-f18) with timeline map UI
- HrrrClient.hrrr_url accepts forecast_hour param (wrfsfcfHH.grib2)
- PropagationGridWorker fetches all 19 forecast hours per run
- Propagation.scores_at/3 queries scores at specific valid_time
- Propagation.available_valid_times/1 returns all forecast times for timeline
- Pruning keeps scores with valid_time >= now - 2h (forecast-aware)
- MapLive: select_time event, timeline data pushed to JS
- JS: forecast timeline bar at bottom of map with clickable hour buttons
- PubSub broadcast sends list of valid_times instead of single time
2026-03-31 16:44:47 -05:00
c775eb2611
Preserve detail panel and range circle across score updates
- Add phx-update="ignore" to detail-panel so LiveView patches don't wipe it
- Re-request point_detail after scores update to refresh with new data
- Range circles persist across updates since they're on a separate layer
2026-03-31 16:28:38 -05:00
8949920b7f
Add Nx/Axon/EXLA ML model skeleton for propagation prediction
13-feature feed-forward network (atmospheric + temporal + frequency).
Includes build, init, predict, encode_features, save/load to disk.
Model weights saved to priv/models/propagation_v1.nx (gitignored).
Not yet trained — scaffolding only.
2026-03-31 16:26:34 -05:00
77db483a27
Add timing instrumentation to PropagationGridWorker
Logs duration for each phase: HRRR fetch, profile storage, score
computation, and upsert, plus total elapsed time.
2026-03-31 16:18:10 -05:00
c9112b9280
Recalibrate refractivity thresholds for HRRR gradient distribution
Previous thresholds (-500 to -60) were calibrated for radiosonde data.
HRRR profiles have coarser vertical resolution, with gradients clustering
between -40 and -130 N/km (median -70). Nearly all grid points were
falling through to the default score of 42, wasting the refractivity
factor. New thresholds (-200 to -40) spread across HRRR percentiles.
2026-03-31 16:08:15 -05:00
49cbe6789c
Implement ITU-R P.526-16 terrain diffraction model
- Replace piecewise knife-edge loss with P.526-16 Eq. 31 single formula
- Fix diffraction parameter ν to use standard formula instead of ad-hoc approximation
- Implement Deygout 3-edge method for multiple obstacle diffraction
- Add dynamic k-factor from HRRR refractivity gradient (falls back to 4/3)
- Terrain worker now looks up nearest HRRR profile for atmospheric correction
- Update algo.md with P.526-16 methods and k-factor table
- Fix pre-existing map_live_test antenna height default (33 ft, not 8)
2026-03-31 16:04:23 -05:00
6d103d7cc4
Prune HRRR grid profiles older than 48h after each scoring run
Deletes grid-aligned profiles (0.125 degree) older than 48 hours
while preserving QSO-linked profiles at arbitrary positions.
Called after each PropagationGridWorker run.
2026-03-31 16:03:14 -05:00
2da061201d
Disable ASOS adjustment cron, rely solely on HRRR hourly scoring
ASOS re-scoring was upserting 458K records every 10 min while losing
the refractivity gradient (the key ducting indicator). HRRR hourly
provides all surface variables plus vertical profile data.
2026-03-31 16:00:04 -05:00
ee6975bc39
Fix ASOS upsert timeout: infinite tx timeout, bump prod pool to 20
458K-record upsert held a connection for the entire transaction,
exceeding the 15s Postgrex timeout on prod. Set transaction timeout
to infinity and increase prod pool from 10 to 20.
2026-03-31 15:54:19 -05:00
ffae4b52e7
Fix HRRR fetch timeouts: 60s receive_timeout, reduce queue to 5
Req default 15s receive_timeout too short for GRIB2 byte-range
downloads on prod. Also reduce HRRR queue concurrency from 20 to 5
to avoid NOAA rate limiting (was causing burst-then-stall pattern).
2026-03-31 15:37:21 -05:00
ff3a879dff
Rework viewshed: score-aware NLOS range, sidebar detail panel, factor explanations
- Viewshed uses propagation score to set max range (ducting/NLOS distance)
- Terrain checked within radio horizon; clear paths get full atmospheric range
- Detail panel moves from map popup to sidebar for better visibility
- Panel shows instantly on click with loading state
- Dark opaque background for readability over map
- Factor analysis section explains each score with contextual descriptions
2026-03-31 13:49:42 -05:00
6e5409bcd9
Update esbuild 0.27.4 and tailwind 4.2.2, add viewshed logging 2026-03-31 13:15:56 -05:00
bfd9ed6339
Copy algo.md into Docker build before mix compile 2026-03-31 13:09:27 -05:00
8f5b53180b
Use propagation tier color as popup header background 2026-03-31 13:08:21 -05:00
5eaa55448e
Add terrain-aware viewshed on map click
Replace generic range circles with actual LOS coverage polygon computed
from SRTM elevation data. Casts 180 rays (every 2 degrees) from the
clicked point, runs Fresnel/diffraction analysis on each, and renders
the reachable area as a Leaflet polygon.

- Viewshed module with haversine forward, terrain sweep, async compute
- Antenna height control (default 8 ft) in map panel
- LiveView start_async/handle_async for non-blocking computation
- Remove signal icon from band selector
2026-03-31 13:06:01 -05:00
166f95e29d
Embed algo.md at compile time to fix prod 500
algo.md was read at runtime via File.read! but isn't included in
the release. Use @external_resource + compile-time module attribute
to bake the HTML into the module.
2026-03-31 12:51:40 -05:00
40d1fa03aa
QSO submission triggers enrichment directly, fix prod badarith crash
- Add enqueue_for_qso/1 to directly enqueue weather/HRRR/terrain/IEMRE
  jobs for a single user-submitted QSO (no cron, no bulk processing)
- Submit flow calls enqueue_for_qso instead of generic enqueue worker
- Add enrichment queues to prod config for on-demand processing
- Guard against HRRR fill values in store_hrrr_profiles (fixes badarith)
- Filter QSOs without pos2 in build_terrain_jobs
2026-03-31 12:29:39 -05:00
105298fc3f
Add FreshnessMonitor GenServer to detect and fill stale propagation data
Replaces the one-shot startup Task with a GenServer that checks every
5 minutes whether propagation scores are older than 2 hours. If stale,
enqueues a PropagationGridWorker job (with dedup check to avoid double
queuing). Disabled in test env to avoid SQL Sandbox conflicts.
2026-03-31 12:08:51 -05:00
45fb9736fa
Log client IP from X-Forwarded-For in request logs
Add RemoteIp plug that extracts real client IP from X-Forwarded-For
header (set by nginx/dokku proxy) and adds it to Logger metadata.
2026-03-31 12:05:05 -05:00
c709fb5c32
Enqueue propagation grid worker on startup if scores are stale
On app start, check if latest scores are older than 2 hours. If so,
immediately enqueue a PropagationGridWorker job. Covers app restarts,
deploys, and missed cron ticks.
2026-03-31 11:56:41 -05:00
4301f48d27
Guard against HRRR fill values in grid scoring
Skip grid points where surface_temp_c or surface_dewpoint_c are
physically impossible (< -80°C or > 60°C). HRRR returns -273.15
(absolute zero) for ocean/missing points which caused division by
zero in absolute_humidity calculation.
2026-03-31 11:07:26 -05:00
187fdf085e
Disable historical backfill queues and cron in production
Only run live scoring (propagation, commercial, solar) in prod.
Backfill queues (weather, hrrr, iemre, terrain, enqueue) and the
QsoWeatherEnqueueWorker cron are dev-only for historical enrichment.
2026-03-31 11:05:33 -05:00
d458c23782
Expand algo.md sources: full data provenance, APIs, and ITU-R references
- Expand ITU-R section with recommendation numbers, titles, and usage
- Document all 8 external data sources with API URLs, volumes, fields
- Add derived parameter documentation (sounding params, HRRR profiles)
- Document live scoring pipeline (grid resolution, update cadence, retention)
- Add contest bias and rain model caveats to known issues
2026-03-31 10:52:45 -05:00
a18cf48493
Add regional analysis to algo.md, update site title to NTMS
- Document Finding 11: regional QSO performance by Maidenhead field
- Explain why regional weight adjustments are not justified
- Fix algo.md table formatting (blank lines before tables for Earmark)
- Change page title from Phoenix Framework to North Texas Microwave Society
2026-03-31 10:47:39 -05:00
a6b4ac1965
Increase weather queue to 20 and iemre to 10 for faster backfill 2026-03-31 10:41:49 -05:00
3874fc173d
Atomic score upserts and auto-prune old propagation data
- Wrap upsert_scores in Repo.transaction for all-or-nothing visibility
- Prune scores older than the 2 most recent valid_times after each upsert
- Add band-specific latest_valid_time/1 to eliminate N+1 query
- Add require Logger to Propagation module
2026-03-31 10:35:59 -05:00
1aebd7f1cc
Optimize map performance: reduce payload 75%, faster rendering
- Remove factors from heatmap scores, fetch on-demand via point_detail
- Pre-encode initial scores JSON once instead of per-render
- Don't store scores in socket assigns (reduce memory)
- Pre-calculate RGBA color lookup table for canvas rendering
- Batch fillStyle changes, hoist math out of inner loop
- Show topbar immediately on data requests (no 300ms delay)
2026-03-31 10:26:20 -05:00
2f2f3388cd
Rename 'How this works' to 'Scoring Algorithm' 2026-03-31 10:24:04 -05:00
ccac7a5020
Add NTMS title to map panel, make range circles non-interactive 2026-03-31 10:17:16 -05:00
c3a5cf4b1a
Add Maidenhead grid overlay, consolidate map controls, improve UX
- Port grid square rendering from gridmap-web with toggle control
- Consolidate band selector, grid toggle, data timestamp, and links
  into unified control panel
- Add "How this works" (/algo) and "Submit a QSO" (/submit) links
- Add back-to-map link on algo page
- Darken range circles for visibility over propagation overlay
- Fix popup blocking double-click zoom with pointer-events passthrough
- Prevent control panel from intercepting map click/scroll events
2026-03-31 10:15:21 -05:00
8f89bbdc02
Exclude slow GRIB2 fixture tests by default (81s -> 16s) 2026-03-31 10:00:13 -05:00
14e1078bf9
Fall back to 1000mb level data when surface fields are missing (fills coastal gaps) 2026-03-31 09:58:16 -05:00
aa694c1771
Remove auto zoom-out on range circle click 2026-03-31 09:53:31 -05:00
69b9128c3f
Fix range circles: zoom to fit, increase opacity, add distance tooltips 2026-03-31 09:52:13 -05:00
2e216d4244
Make point detail popup instant with client-side lookup and range circles
- Scores now include factors and valid_time in the viewport query,
  eliminating the server round-trip for popups
- Click shows two range circles: solid inner (typical range) and
  dashed outer (max estimated range), colored by score tier
- Circles disappear when popup closes
- Band info pushed to client on band switch for accurate range estimates
2026-03-31 09:50:48 -05:00
0c295c1558
Add click-to-inspect factor breakdown popup on map
Click anywhere on the propagation map to see a detailed popup with:
- Overall score and tier label with color
- Estimated range for the selected band (CW mode)
- All 9 scoring factors with visual bar charts, individual scores,
  and weight percentages
- Grid point coordinates and data timestamp

Factors are displayed in weight order so users can immediately see
which atmospheric conditions are driving the prediction.
2026-03-31 09:45:50 -05:00
d7fc120c28
Update algo link data location and fix markdown dark mode styling 2026-03-31 09:39:12 -05:00
e3c9207f57
Add covering index for map viewport query (6s -> 25ms) 2026-03-31 09:36:36 -05:00
d5140267b1
Show topbar loading indicator while scores are loading 2026-03-31 09:30:20 -05:00