Commit graph

105 commits

Author SHA1 Message Date
a18cf48493
Add regional analysis to algo.md, update site title to NTMS
- Document Finding 11: regional QSO performance by Maidenhead field
- Explain why regional weight adjustments are not justified
- Fix algo.md table formatting (blank lines before tables for Earmark)
- Change page title from Phoenix Framework to North Texas Microwave Society
2026-03-31 10:47:39 -05:00
a6b4ac1965
Increase weather queue to 20 and iemre to 10 for faster backfill 2026-03-31 10:41:49 -05:00
3874fc173d
Atomic score upserts and auto-prune old propagation data
- Wrap upsert_scores in Repo.transaction for all-or-nothing visibility
- Prune scores older than the 2 most recent valid_times after each upsert
- Add band-specific latest_valid_time/1 to eliminate N+1 query
- Add require Logger to Propagation module
2026-03-31 10:35:59 -05:00
1aebd7f1cc
Optimize map performance: reduce payload 75%, faster rendering
- Remove factors from heatmap scores, fetch on-demand via point_detail
- Pre-encode initial scores JSON once instead of per-render
- Don't store scores in socket assigns (reduce memory)
- Pre-calculate RGBA color lookup table for canvas rendering
- Batch fillStyle changes, hoist math out of inner loop
- Show topbar immediately on data requests (no 300ms delay)
2026-03-31 10:26:20 -05:00
2f2f3388cd
Rename 'How this works' to 'Scoring Algorithm' 2026-03-31 10:24:04 -05:00
ccac7a5020
Add NTMS title to map panel, make range circles non-interactive 2026-03-31 10:17:16 -05:00
c3a5cf4b1a
Add Maidenhead grid overlay, consolidate map controls, improve UX
- Port grid square rendering from gridmap-web with toggle control
- Consolidate band selector, grid toggle, data timestamp, and links
  into unified control panel
- Add "How this works" (/algo) and "Submit a QSO" (/submit) links
- Add back-to-map link on algo page
- Darken range circles for visibility over propagation overlay
- Fix popup blocking double-click zoom with pointer-events passthrough
- Prevent control panel from intercepting map click/scroll events
2026-03-31 10:15:21 -05:00
8f89bbdc02
Exclude slow GRIB2 fixture tests by default (81s -> 16s) 2026-03-31 10:00:13 -05:00
14e1078bf9
Fall back to 1000mb level data when surface fields are missing (fills coastal gaps) 2026-03-31 09:58:16 -05:00
aa694c1771
Remove auto zoom-out on range circle click 2026-03-31 09:53:31 -05:00
69b9128c3f
Fix range circles: zoom to fit, increase opacity, add distance tooltips 2026-03-31 09:52:13 -05:00
2e216d4244
Make point detail popup instant with client-side lookup and range circles
- Scores now include factors and valid_time in the viewport query,
  eliminating the server round-trip for popups
- Click shows two range circles: solid inner (typical range) and
  dashed outer (max estimated range), colored by score tier
- Circles disappear when popup closes
- Band info pushed to client on band switch for accurate range estimates
2026-03-31 09:50:48 -05:00
0c295c1558
Add click-to-inspect factor breakdown popup on map
Click anywhere on the propagation map to see a detailed popup with:
- Overall score and tier label with color
- Estimated range for the selected band (CW mode)
- All 9 scoring factors with visual bar charts, individual scores,
  and weight percentages
- Grid point coordinates and data timestamp

Factors are displayed in weight order so users can immediately see
which atmospheric conditions are driving the prediction.
2026-03-31 09:45:50 -05:00
d7fc120c28
Update algo link data location and fix markdown dark mode styling 2026-03-31 09:39:12 -05:00
e3c9207f57
Add covering index for map viewport query (6s -> 25ms) 2026-03-31 09:36:36 -05:00
d5140267b1
Show topbar loading indicator while scores are loading 2026-03-31 09:30:20 -05:00
d97de91978
Add meteorologist feedback on model choice to RRFS doc 2026-03-31 09:27:23 -05:00
0ed10e695c
Add RRFS data assessment — not yet suitable, continue with HRRR 2026-03-31 09:25:45 -05:00
6e3b8d62de
Fix wgrib2: use v3.6.0 (v3.6.1 doesn't exist) and follow redirects 2026-03-31 09:20:26 -05:00
bb514112bf
Clean up Dockerfile: remove unused deps, comments, and redundant lines 2026-03-31 09:18:30 -05:00
9aef372985
Build wgrib2 v3.6.1 from GitHub source with cmake in Docker 2026-03-31 09:17:10 -05:00
24840c48d4
Show 'Latest data: date time (Xm ago)' in map badge 2026-03-31 09:12:32 -05:00
e0d9900608
Add real-time ASOS adjustments between hourly HRRR updates
New AsosAdjustmentWorker runs every 10 minutes:
- Fetches latest ASOS observations from all ~2900 US stations via
  IEM bulk currents API (parallel fetch across 51 state networks)
- For each grid point within 75km of a reporting station, re-scores
  using fresh ASOS data (temp, dewpoint, wind, sky, pressure, precip)
  with HRRR refractivity gradient from the last hourly computation
- Pushes updated scores to the map via PubSub

Also stores HRRR profiles in the database during grid computation
so the data persists for reference and ASOS blending.
2026-03-31 09:10:12 -05:00
b87572aa81
Close band dropdown after selection by shifting focus to map 2026-03-31 09:04:31 -05:00
97d4bd9372
Only send visible map scores based on viewport bounds
The JS hook sends map bounds on load and on pan/zoom. The server
queries only scores within those bounds, dramatically reducing the
payload for band switches and map updates. At zoom 7 (DFW area)
this sends ~2k scores instead of ~95k.
2026-03-31 09:01:48 -05:00
321133c3cb
Push map updates via PubSub when propagation scores are computed 2026-03-31 08:57:51 -05:00
4e2d74e21c
Build wgrib2 in Docker with cached build stage
Add a separate wgrib2-builder stage that compiles wgrib2 from source.
Docker layer caching means this only builds once — subsequent deploys
reuse the cached layer. The binary is copied into the final runtime
image. Also fixed wgrib2 path lookup to be runtime instead of
compile-time so it works in the Docker build pipeline.
2026-03-31 08:55:09 -05:00
66639e3717
Increase grid to 0.125 degrees with smooth canvas overlay
Replace circle markers with a canvas tile layer that renders smooth,
flowing colored regions using bilinear interpolation between grid
points. Colors interpolate between tiers for gradients. ~95k grid
points at 0.125 degree resolution with wgrib2 extraction.
2026-03-31 08:52:07 -05:00
becdf89978
Truncate valid_time microseconds for Ecto utc_datetime 2026-03-31 08:48:32 -05:00
22cb055269
Skip grid points with missing surface data (ocean/outside-domain) 2026-03-31 08:36:17 -05:00
b7446e83f5
Use wgrib2 for HRRR grid extraction (7s vs 80+ minutes)
Add Wgrib2 module that shells out to wgrib2 binary for fast GRIB2
grid extraction using -lola (nearest-neighbor to regular lat-lon grid).
Falls back to pure-Elixir decoder if wgrib2 is not installed.

Also: parallel GRIB2 range downloads, merge adjacent byte ranges,
skip corrupt messages instead of failing, pressure fetch is optional.
2026-03-31 08:33:29 -05:00
81cb735d6e
Skip corrupt GRIB2 messages instead of failing entire fetch
Both extract_points and extract_grid now skip messages with missing
data sections instead of halting. Pressure product fetch is now
optional for both grid and single-point modes — if pressure data
has corrupt messages, surface-only profiles are still stored. Fixes
historical HRRR backfill failing on old data with incomplete files.
2026-03-31 08:12:10 -05:00
c4de1dd77c
Center map on DFW with ~500 mile radius default view 2026-03-31 08:07:16 -05:00
c56f349c86
Reduce grid resolution to 0.5 degrees (~6k points)
95k points at 0.125 degree resolution caused the GRIB2 extraction to
take too long. 0.5 degree (~55 km) resolution gives 6k points which
completes in under a minute. Can increase resolution later once the
extraction is optimized.
2026-03-31 07:22:42 -05:00
6d4a3e8738
Remove unique constraint from propagation grid worker that blocked execution 2026-03-31 07:02:19 -05:00
7f6905645f
Pause backfill queues during propagation grid fetch
The hrrr/weather/iemre/terrain backfill queues (20+ concurrent jobs)
were competing for bandwidth with the propagation grid HRRR download.
Now pauses those queues before the grid fetch and resumes them after,
ensuring the hourly propagation update completes reliably.
2026-03-30 17:42:05 -05:00
9c504acb67
Disable Oban queues in mix propagation_grid to prevent backfill noise 2026-03-30 17:36:32 -05:00
f6b1f751d4
Use 2-hour lag for HRRR grid fetch to ensure data availability 2026-03-30 17:32:39 -05:00
86e01748ef
Fix legend readability with explicit text color 2026-03-30 17:30:58 -05:00
8cbce9565d
Fix Leaflet hook registration and full-page map
Move PropagationMap hook from colocated .hooks.js to assets/js and
register it directly in app.js hooks config. Fixes "unknown hook"
error at runtime.
2026-03-30 17:29:05 -05:00
97bd93ed5c
Make /map full-page Leaflet with band dropdown
- Remove Layouts.app wrapper so map fills entire viewport
- Replace band buttons with daisyUI dropdown selector
- Overlay controls (home button, band picker, timestamp) on top-left
- Keep Leaflet's standard zoom controls
2026-03-30 17:27:21 -05:00
a283ad9c66
Refactor HRRR fetch to batch points per hour
QSO enrichment now groups all path points by HRRR hour and creates
one batch job per hour instead of one job per point. The batch job
downloads the GRIB2 data once and extracts all needed points from
the same binary. Legacy single-point jobs are still supported for
backward compatibility.
2026-03-30 17:21:47 -05:00
1e205cb471
Add propagation context, grid worker, and fix merge issues
- Replace stub propagation.ex with real implementation (score_grid_point,
  upsert_scores, latest_scores, latest_valid_time)
- Add PropagationGridWorker (hourly Oban cron) for CONUS grid scoring
- Add mix propagation_grid task for manual triggering
- Fix duplicate extract_grid/2 from parallel merges
- Fix extract_grid to skip outside-grid points instead of erroring
- Add propagation queue to Oban config
2026-03-30 17:18:05 -05:00
05e14c9f08
Merge branch 'worktree-agent-a212f735' into feature/propagation-map
# Conflicts:
#	lib/microwaveprop/propagation/band_config.ex
#	test/microwaveprop/propagation/band_config_test.exs
2026-03-30 17:13:53 -05:00
61e7bbdb3a
Merge branch 'worktree-agent-a791a51b' into feature/propagation-map 2026-03-30 17:13:43 -05:00
5a0113aac2
Add Leaflet map integration with propagation score display
Vendor Leaflet 1.9.4 (JS, CSS, marker images) and wire it into
the esbuild/Tailwind asset pipeline. Create MapLive with band
selector buttons, auto-refresh, and a colocated JS hook that
renders propagation scores as color-coded circle markers with a
legend. Stub Propagation context and BandConfig modules provide
the data interface for the scoring pipeline.

Add nav bar with links to Map, QSOs, and Submit pages.
2026-03-30 17:11:00 -05:00
663f1b7b46
Add HRRR batch fetch client with multi-point grid extraction
Add fetch_grid/3 for batch HRRR data retrieval across multiple lat/lon
points in a single download pass. Expand surface messages to include
wind (UGRD/VGRD), cloud cover (TCDC), and precipitation (APCP). Add
extract_grid/2 to the GRIB2 extractor for multi-point extraction from
a single GRIB binary, and add GRIB2 variable identifiers for the new
surface fields.
2026-03-30 17:08:21 -05:00
8d75f188f6
Merge branch 'worktree-agent-a3cdfb99' into feature/propagation-map 2026-03-30 17:00:40 -05:00
2698f33cd3
Add propagation scoring algorithm with 9 weighted factors
Implements BandConfig (data-driven thresholds for 8 bands) and Scorer
module with humidity, time-of-day, TD depression, refractivity, sky
cover, season, wind, rain attenuation, and pressure trend scoring.
Composite score applies BandConfig weights (sum to 1.0) across all
factors, producing 0-100 score per band. 10 GHz uses beneficial
humidity/ducting model; 24 GHz+ uses harmful absorption model.
2026-03-30 16:57:38 -05:00
8dd289580d
Merge branch 'worktree-agent-ae6c8945' into feature/propagation-map 2026-03-30 16:50:42 -05:00