prop/lib/microwaveprop/weather
Graham McIntire 900685aa06 Phase 1 tasks 1.1-1.5: HRRR native hybrid-sigma ingestion
- Spike docs at docs/research/hrrr_native_levels.md confirming files
  are on AWS S3 for 5+ years, 50 hybrid levels, and include TKE and
  SPFH needed for Phase 2 turbulence features. Architectural finding:
  per-point on-demand fetching is impractical (~530 MB/file), so
  the ingestion worker batches per (date, hour) instead.
- hrrr_native_profiles schema: arrays per level plus cached surface
  scalars and placeholder columns for Phase 2/4 derived fields.
  Strictly additive — the existing hrrr_profiles table is untouched.
- HrrrNativeClient: pure URL/message-list helpers, build_native_profile/1
  that turns a parsed wgrib2 map into the schema shape (TDD'd).
- Exposed HrrrClient.download_grib_ranges/2 so the native client
  reuses the existing parallel byte-range download + disk cache.
- HrrrNativeGridWorker: Oban worker keyed on {year, month, day, hour},
  unique at :infinity, pulls distinct (lat, lon) points from contacts
  in the ±30 min window, downloads the native grib2, extracts per
  point, bulk-upserts.
- mix hrrr_native_backfill --limit N enqueues the top-N hours by
  contact count.

Phase 1 gate still pending Task 1.6 (sanity-check backtest after
live data lands).
2026-04-09 16:23:51 -05:00
..
grib2 Batch ERA5 fetches by month and 2° tile 2026-04-09 13:01:49 -05:00
era5_batch_client.ex Batch ERA5 fetches by month and 2° tile 2026-04-09 13:01:49 -05:00
era5_client.ex Batch ERA5 fetches by month and 2° tile 2026-04-09 13:01:49 -05:00
era5_profile.ex Add ERA5 reanalysis and RTMA data sources 2026-04-07 12:04:16 -05:00
hrrr_client.ex Phase 1 tasks 1.1-1.5: HRRR native hybrid-sigma ingestion 2026-04-09 16:23:51 -05:00
hrrr_native_client.ex Phase 1 tasks 1.1-1.5: HRRR native hybrid-sigma ingestion 2026-04-09 16:23:51 -05:00
hrrr_native_profile.ex Phase 1 tasks 1.1-1.5: HRRR native hybrid-sigma ingestion 2026-04-09 16:23:51 -05:00
hrrr_profile.ex Database performance fixes and async backfill enqueue 2026-04-04 19:19:18 -05:00
iem_client.ex Add real-time ASOS adjustments between hourly HRRR updates 2026-03-31 09:10:12 -05:00
iemre_observation.ex Add IEM precipitation data to QSO weather pipeline 2026-03-30 13:18:11 -05:00
rtma_client.ex Add ERA5 reanalysis and RTMA data sources 2026-04-07 12:04:16 -05:00
rtma_observation.ex Add ERA5 reanalysis and RTMA data sources 2026-04-07 12:04:16 -05:00
solar_client.ex Add QSO import, solar indices, Oban workers, LiveView UI, and parallel weather import 2026-03-29 13:04:55 -05:00
solar_index.ex Add QSO import, solar indices, Oban workers, LiveView UI, and parallel weather import 2026-03-29 13:04:55 -05:00
sounding.ex Add weather data schema, context, sounding params, and IEM ingestion 2026-03-28 15:57:19 -05:00
sounding_params.ex Add weather data schema, context, sounding params, and IEM ingestion 2026-03-28 15:57:19 -05:00
station.ex Add weather data schema, context, sounding params, and IEM ingestion 2026-03-28 15:57:19 -05:00
surface_observation.ex Add IEM precipitation data to QSO weather pipeline 2026-03-30 13:18:11 -05:00
weather_layers.ex Add WeatherLayers module for deriving upper-air map fields from HRRR profiles 2026-04-03 16:49:17 -05:00