Commit graph

10 commits

Author SHA1 Message Date
77db483a27
Add timing instrumentation to PropagationGridWorker
Logs duration for each phase: HRRR fetch, profile storage, score
computation, and upsert, plus total elapsed time.
2026-03-31 16:18:10 -05:00
6d103d7cc4
Prune HRRR grid profiles older than 48h after each scoring run
Deletes grid-aligned profiles (0.125 degree) older than 48 hours
while preserving QSO-linked profiles at arbitrary positions.
Called after each PropagationGridWorker run.
2026-03-31 16:03:14 -05:00
40d1fa03aa
QSO submission triggers enrichment directly, fix prod badarith crash
- Add enqueue_for_qso/1 to directly enqueue weather/HRRR/terrain/IEMRE
  jobs for a single user-submitted QSO (no cron, no bulk processing)
- Submit flow calls enqueue_for_qso instead of generic enqueue worker
- Add enrichment queues to prod config for on-demand processing
- Guard against HRRR fill values in store_hrrr_profiles (fixes badarith)
- Filter QSOs without pos2 in build_terrain_jobs
2026-03-31 12:29:39 -05:00
e0d9900608
Add real-time ASOS adjustments between hourly HRRR updates
New AsosAdjustmentWorker runs every 10 minutes:
- Fetches latest ASOS observations from all ~2900 US stations via
  IEM bulk currents API (parallel fetch across 51 state networks)
- For each grid point within 75km of a reporting station, re-scores
  using fresh ASOS data (temp, dewpoint, wind, sky, pressure, precip)
  with HRRR refractivity gradient from the last hourly computation
- Pushes updated scores to the map via PubSub

Also stores HRRR profiles in the database during grid computation
so the data persists for reference and ASOS blending.
2026-03-31 09:10:12 -05:00
321133c3cb
Push map updates via PubSub when propagation scores are computed 2026-03-31 08:57:51 -05:00
becdf89978
Truncate valid_time microseconds for Ecto utc_datetime 2026-03-31 08:48:32 -05:00
6d4a3e8738
Remove unique constraint from propagation grid worker that blocked execution 2026-03-31 07:02:19 -05:00
7f6905645f
Pause backfill queues during propagation grid fetch
The hrrr/weather/iemre/terrain backfill queues (20+ concurrent jobs)
were competing for bandwidth with the propagation grid HRRR download.
Now pauses those queues before the grid fetch and resumes them after,
ensuring the hourly propagation update completes reliably.
2026-03-30 17:42:05 -05:00
f6b1f751d4
Use 2-hour lag for HRRR grid fetch to ensure data availability 2026-03-30 17:32:39 -05:00
1e205cb471
Add propagation context, grid worker, and fix merge issues
- Replace stub propagation.ex with real implementation (score_grid_point,
  upsert_scores, latest_scores, latest_valid_time)
- Add PropagationGridWorker (hourly Oban cron) for CONUS grid scoring
- Add mix propagation_grid task for manual triggering
- Fix duplicate extract_grid/2 from parallel merges
- Fix extract_grid to skip outside-grid points instead of erroring
- Add propagation queue to Oban config
2026-03-30 17:18:05 -05:00