Commit graph

18 commits

Author SHA1 Message Date
1e205cb471
Add propagation context, grid worker, and fix merge issues
- Replace stub propagation.ex with real implementation (score_grid_point,
  upsert_scores, latest_scores, latest_valid_time)
- Add PropagationGridWorker (hourly Oban cron) for CONUS grid scoring
- Add mix propagation_grid task for manual triggering
- Fix duplicate extract_grid/2 from parallel merges
- Fix extract_grid to skip outside-grid points instead of erroring
- Add propagation queue to Oban config
2026-03-30 17:18:05 -05:00
7f97c72f98
Increase weather enqueue frequency to every 30 minutes 2026-03-30 13:28:21 -05:00
3228835636
Add IEM precipitation data to QSO weather pipeline
- Add precip_1h_in and wx_codes fields to ASOS surface observations
- Add IEMRE reanalysis schema for radar-derived hourly precipitation
- Add IemreFetchWorker with exponential backoff and idempotency
- Integrate IEMRE enqueue into cron weather backfill pipeline
- All existing QSOs marked iemre_queued=false for automatic backfill
2026-03-30 13:18:11 -05:00
10e8feb486
Add commercial link monitoring via SNMP polling
Poll UBNT AirFiber radios (AF11X + AF60-LR) every 5 minutes via
net-snmp CLI, storing signal metrics in commercial_samples. Fetches
ASOS weather alongside each cycle for propagation correlation.

Includes 7 seeded link definitions, Oban cron worker, and net-snmp
in the Docker image.
2026-03-30 13:02:59 -05:00
8ccd778623
bump hrr downloads 2026-03-30 12:11:45 -05:00
9c25cff935
bump hrrr downloads 2026-03-30 10:37:55 -05:00
25bbaeefca
increase terrain workers 2026-03-30 10:28:33 -05:00
ead34990b0
Fix HRRR pipeline: single concurrency, cancel bad GRIB, rescue stuck jobs
- Reduce HRRR queue to 1 worker to avoid overwhelming upstream
- Add Oban Lifeline plugin to rescue orphaned executing jobs after 30m
- Treat GRIB2 decode errors (malformed section, missing sections, etc.)
  as permanent failures instead of retrying indefinitely
2026-03-30 09:16:07 -05:00
2ae60d036b
Add SRTM auto-download on tile miss from AWS Terrain Tiles
When a local .hgt tile is missing, download it from the public AWS S3
skadi bucket, decompress with zlib, and write to the tiles directory
before retrying the lookup. Falls back to Open-Meteo/OpenTopo APIs if
the download fails.
2026-03-30 08:36:21 -05:00
653efa450a
Reduce terrain queue concurrency to 1 to avoid 429 rate limits
Open-Meteo elevation API rate-limits when two terrain workers
fire concurrent requests.
2026-03-30 08:08:33 -05:00
5a9359191f
Replace wgrib2 with pure Elixir GRIB2 decoder for HRRR pipeline
Eliminates the external wgrib2 C tool dependency that blocked HRRR
processing. Implements Lambert Conformal projection, simple packing
(Template 5.0), complex packing with spatial differencing (Template 5.3),
and GRIB2 section parsing — enough to extract point values from HRRR
grid data using only Elixir.
2026-03-29 18:43:43 -05:00
45e2e69361
Add SRTM terrain path profiles for QSOs
Fetch elevation data along the path between two stations via the
Open-Meteo Elevation API (with Open-Topo-Data fallback), compute
Fresnel zone clearance, earth bulge, and knife-edge diffraction loss,
and store the results per QSO.

- terrain_profiles table with migration and TerrainProfile schema
- ElevationClient with batched API calls and fallback
- TerrainAnalysis with Fresnel/diffraction physics (ITU-R P.526-15)
- TerrainProfileWorker on Oban :terrain queue
- QsoWeatherEnqueueWorker enqueues terrain jobs automatically
- QSO show page displays verdict badge and collapsible elevation table
- Reorder show page: terrain, soundings, solar, HRRR, surface obs
- Fix Dockerfile wgrib2 build (add cmake dependency)
2026-03-29 16:22:29 -05:00
8bd0989fbe
Add HRRR model profile fetching for QSOs
HRRR provides hourly 3km-resolution atmospheric profiles, filling the
temporal gaps (12-hourly soundings) and spatial gaps (only 9 sounding
stations) in our current weather data.

- Add hrrr_profiles table and hrrr_queued flag on QSOs
- HrrrClient fetches GRIB2 data via HTTP Range requests + wgrib2
- HrrrFetchWorker derives refractivity/ducting params via SoundingParams
- QsoWeatherEnqueueWorker now also enqueues HRRR jobs
- QSO show page displays HRRR section with collapsible profile
- Dockerfile builds wgrib2 from source for production
2026-03-29 15:54:22 -05:00
c38596cc36
Add QsoWeatherEnqueueWorker cron job to auto-fetch weather for new QSOs
Oban cron worker runs every 4 hours, finds QSOs without weather data,
discovers nearby ASOS/sounding stations, and enqueues WeatherFetchWorker
jobs. Migrations run automatically on app start in production.
2026-03-29 14:11:03 -05:00
1008ce7fd2
Add Oban Pruner plugin to clean up completed jobs after 24 hours 2026-03-29 13:35:04 -05:00
5124cc09c1
Add IEM rate limiting, Req retry, and Oban retry for failed weather fetches
IEM was returning 503s under 10 concurrent workers. Three fixes:
- Req transient retry with exponential backoff on IEM requests
- WeatherFetchWorker Oban job retries failed ASOS/RAOB fetches 2-3h later
  with random jitter to avoid thundering herd
- Import script concurrency reduced 10→5, weather queue capped at 3
2026-03-29 13:24:52 -05:00
6f16395f44
Add QSO import, solar indices, Oban workers, LiveView UI, and parallel weather import
- Radio context with QSO schema and CSV import script (58K contacts)
- Solar index schema, GFZ client, and daily Oban cron worker
- LiveView dashboard with QSO table and weather correlation views
- Parallelize weather import script using Task.async_stream (10 concurrent workers)
- Replace sequential rate_limit sleeps with concurrency-based backpressure
- Atomic progress counter for interleave-safe reporting
2026-03-29 13:04:55 -05:00
bc674f0d54
initial 2026-03-28 11:28:47 -05:00