Commit graph

7 commits

Author SHA1 Message Date
1008ce7fd2
Add Oban Pruner plugin to clean up completed jobs after 24 hours 2026-03-29 13:35:04 -05:00
ef9aac60c0
Move all weather fetching to Oban, import script only enqueues jobs
Instead of fetching inline with Task.async_stream (which hammered IEM
with concurrent requests), the import script now just checks what data
is missing and enqueues WeatherFetchWorker jobs. Oban handles all HTTP
with concurrency 3 and automatic backoff on failures.
2026-03-29 13:29:29 -05:00
5124cc09c1
Add IEM rate limiting, Req retry, and Oban retry for failed weather fetches
IEM was returning 503s under 10 concurrent workers. Three fixes:
- Req transient retry with exponential backoff on IEM requests
- WeatherFetchWorker Oban job retries failed ASOS/RAOB fetches 2-3h later
  with random jitter to avoid thundering herd
- Import script concurrency reduced 10→5, weather queue capped at 3
2026-03-29 13:24:52 -05:00
6f16395f44
Add QSO import, solar indices, Oban workers, LiveView UI, and parallel weather import
- Radio context with QSO schema and CSV import script (58K contacts)
- Solar index schema, GFZ client, and daily Oban cron worker
- LiveView dashboard with QSO table and weather correlation views
- Parallelize weather import script using Task.async_stream (10 concurrent workers)
- Replace sequential rate_limit sleeps with concurrency-based backpressure
- Atomic progress counter for interleave-safe reporting
2026-03-29 13:04:55 -05:00
6489f08138
Add weather data schema, context, sounding params, and IEM ingestion
Store surface observations (ASOS) and upper-air soundings (RAOB) alongside
QSOs for atmospheric propagation correlation. Three new tables: weather_stations,
surface_observations, and soundings with JSONB profiles and pre-computed derived
parameters (refractivity, gradients, duct detection, stability indices).

Includes IEM API client for historical data import and import script seeded
with 95 ASOS + 9 sounding stations from PropCast coverage area.
2026-03-28 15:57:19 -05:00
e0ab6560d0
docker 2026-03-28 11:41:31 -05:00
bc674f0d54
initial 2026-03-28 11:28:47 -05:00