Commit graph

8 commits

Author SHA1 Message Date
3749360a28
Fix complex packing decode crash on trailing padding bits
consume_bits would crash with a MatchError when the remaining
bitstring had fewer bits than the group width. Added a guard
clause to stop consuming when insufficient bits remain, matching
how wgrib2 handles trailing padding in packed data sections.
2026-03-30 07:52:06 -05:00
5a9359191f
Replace wgrib2 with pure Elixir GRIB2 decoder for HRRR pipeline
Eliminates the external wgrib2 C tool dependency that blocked HRRR
processing. Implements Lambert Conformal projection, simple packing
(Template 5.0), complex packing with spatial differencing (Template 5.3),
and GRIB2 section parsing — enough to extract point values from HRRR
grid data using only Elixir.
2026-03-29 18:43:43 -05:00
9334539442
Retry on 429/5xx responses across all HTTP clients
Previously retry: :transient only retried connection errors, not HTTP
429 rate limits. All clients now use a custom retry function that
handles 429, 500, 502, 503, 504 with 5 retries and jittered backoff.
2026-03-29 17:47:08 -05:00
3534b282af
Fix wgrib2 error message — no homebrew formula exists 2026-03-29 17:44:58 -05:00
8bd0989fbe
Add HRRR model profile fetching for QSOs
HRRR provides hourly 3km-resolution atmospheric profiles, filling the
temporal gaps (12-hourly soundings) and spatial gaps (only 9 sounding
stations) in our current weather data.

- Add hrrr_profiles table and hrrr_queued flag on QSOs
- HrrrClient fetches GRIB2 data via HTTP Range requests + wgrib2
- HrrrFetchWorker derives refractivity/ducting params via SoundingParams
- QsoWeatherEnqueueWorker now also enqueues HRRR jobs
- QSO show page displays HRRR section with collapsible profile
- Dockerfile builds wgrib2 from source for production
2026-03-29 15:54:22 -05:00
5124cc09c1
Add IEM rate limiting, Req retry, and Oban retry for failed weather fetches
IEM was returning 503s under 10 concurrent workers. Three fixes:
- Req transient retry with exponential backoff on IEM requests
- WeatherFetchWorker Oban job retries failed ASOS/RAOB fetches 2-3h later
  with random jitter to avoid thundering herd
- Import script concurrency reduced 10→5, weather queue capped at 3
2026-03-29 13:24:52 -05:00
6f16395f44
Add QSO import, solar indices, Oban workers, LiveView UI, and parallel weather import
- Radio context with QSO schema and CSV import script (58K contacts)
- Solar index schema, GFZ client, and daily Oban cron worker
- LiveView dashboard with QSO table and weather correlation views
- Parallelize weather import script using Task.async_stream (10 concurrent workers)
- Replace sequential rate_limit sleeps with concurrency-based backpressure
- Atomic progress counter for interleave-safe reporting
2026-03-29 13:04:55 -05:00
6489f08138
Add weather data schema, context, sounding params, and IEM ingestion
Store surface observations (ASOS) and upper-air soundings (RAOB) alongside
QSOs for atmospheric propagation correlation. Three new tables: weather_stations,
surface_observations, and soundings with JSONB profiles and pre-computed derived
parameters (refractivity, gradients, duct detection, stability indices).

Includes IEM API client for historical data import and import script seeded
with 95 ASOS + 9 sounding stations from PropCast coverage area.
2026-03-28 15:57:19 -05:00