Commit graph

9 commits

Author SHA1 Message Date
30c1018400
Extend skew-T plot to cover the full troposphere
Historical contacts showed a skew-T log-P diagram that stopped at 700 mb
because HrrrClient and Era5Client only fetched pressure-level data down
to the top of the boundary layer. The chart canvas already ran up to
100 mb, so the trace clipped mid-atmosphere.

Two complementary fixes:

1. Extend @pressure_levels in HrrrClient and Era5Client with
   650/600/550/500/450/400/350/300/250/200/150/100 mb so new fetches
   cover the full troposphere + lower stratosphere.

2. Prefer the native hybrid-sigma profile for the contact-detail
   skew-T when one has been backfilled for the contact's hour. The
   native profile already stores all 50 hybrid levels up to ~19 km,
   so historical contacts covered by the native backfill get a full
   trace without re-hitting S3. A new HrrrNativeProfile.to_skew_t_profile/1
   converts the parallel arrays into the %{"pres","tmpc","dwpc","hght"}
   list shape the renderer expects, deriving dewpoint from SPFH via the
   Magnus inverse. Weather.find_nearest_native_profile/3 mirrors
   find_nearest_hrrr/3 for the lookup.
2026-04-14 17:25:58 -05:00
dc66252fdc
Reduce dead tuples by using conditional upserts and skip-on-conflict
- HRRR, IEMRE, terrain: on_conflict: :nothing (immutable data)
- Surface obs, soundings, solar: conditional WHERE (only update when values differ)
- set_enrichment_status!: skip rows already at target status
2026-04-04 17:32:57 -05:00
254e64dedc
Rename qsos to contacts throughout codebase, keep DB table name
Rename all modules, functions, variables, routes, and UI text from
qso/qsos to contact/contacts. Database table stays as "qsos" to avoid
migration. Add /qsos -> /contacts redirects for old URLs.
2026-04-01 11:25:04 -05:00
ded9c054ac
Multi-point path enrichment for HRRR, IEMRE, and weather data
Enqueue worker now gathers atmospheric data at pos1, midpoint, and pos2
along each QSO path instead of only pos1. Existing has_* guards prevent
duplicate fetches at each grid point.

- Add Radio.qso_path_points/1 for path point extraction
- Update hrrr_job_for_qso, iemre_job_for_qso, jobs_for_qso to iterate path points
- Refactor Weather into find_nearest_hrrr/3 and find_nearest_iemre/3
- Add hrrr_profiles_for_path/1 and iemre_for_path/1 query functions
- Add mix reset_enrichment task to trigger re-processing
2026-03-30 16:09:01 -05:00
3228835636
Add IEM precipitation data to QSO weather pipeline
- Add precip_1h_in and wx_codes fields to ASOS surface observations
- Add IEMRE reanalysis schema for radar-derived hourly precipitation
- Add IemreFetchWorker with exponential backoff and idempotency
- Integrate IEMRE enqueue into cron weather backfill pipeline
- All existing QSOs marked iemre_queued=false for automatic backfill
2026-03-30 13:18:11 -05:00
8bd0989fbe
Add HRRR model profile fetching for QSOs
HRRR provides hourly 3km-resolution atmospheric profiles, filling the
temporal gaps (12-hourly soundings) and spatial gaps (only 9 sounding
stations) in our current weather data.

- Add hrrr_profiles table and hrrr_queued flag on QSOs
- HrrrClient fetches GRIB2 data via HTTP Range requests + wgrib2
- HrrrFetchWorker derives refractivity/ducting params via SoundingParams
- QsoWeatherEnqueueWorker now also enqueues HRRR jobs
- QSO show page displays HRRR section with collapsible profile
- Dockerfile builds wgrib2 from source for production
2026-03-29 15:54:22 -05:00
c38596cc36
Add QsoWeatherEnqueueWorker cron job to auto-fetch weather for new QSOs
Oban cron worker runs every 4 hours, finds QSOs without weather data,
discovers nearby ASOS/sounding stations, and enqueues WeatherFetchWorker
jobs. Migrations run automatically on app start in production.
2026-03-29 14:11:03 -05:00
6f16395f44
Add QSO import, solar indices, Oban workers, LiveView UI, and parallel weather import
- Radio context with QSO schema and CSV import script (58K contacts)
- Solar index schema, GFZ client, and daily Oban cron worker
- LiveView dashboard with QSO table and weather correlation views
- Parallelize weather import script using Task.async_stream (10 concurrent workers)
- Replace sequential rate_limit sleeps with concurrency-based backpressure
- Atomic progress counter for interleave-safe reporting
2026-03-29 13:04:55 -05:00
6489f08138
Add weather data schema, context, sounding params, and IEM ingestion
Store surface observations (ASOS) and upper-air soundings (RAOB) alongside
QSOs for atmospheric propagation correlation. Three new tables: weather_stations,
surface_observations, and soundings with JSONB profiles and pre-computed derived
parameters (refractivity, gradients, duct detection, stability indices).

Includes IEM API client for historical data import and import script seeded
with 95 ASOS + 9 sounding stations from PropCast coverage area.
2026-03-28 15:57:19 -05:00