Commit graph

5 commits

Author SHA1 Message Date
ac4e43c726
Remove 500-limit on BackfillEnqueueWorker cron
10K+ HRRR-pending contacts were starving the 200-ish NARR candidates
out of the 500 per-run slot each 30 min because NARR candidates sort
last by the :hrrr_status priority. Drop the cap so every eligible
contact gets enqueued on each cron run; queue concurrency still paces
the actual fetches. Worker keeps supporting an explicit "limit" arg
for ad-hoc runs and tests.
2026-04-16 12:46:56 -05:00
3604896726
Rename ERA5 → NARR across the codebase
- Schema: Era5Profile → NarrProfile; table renamed via migration
  rename_era5_profiles_to_narr_profiles (ALTER TABLE + rename indexes,
  atomic and instant, no row rewrite)
- Weather helpers: find_nearest_era5/era5_for_contact/era5_profiles_for_path
  → find_nearest_narr/narr_for_contact/narr_profiles_for_path
- BackfillEnqueueWorker: :era5 → :narr type (virtual, no narr_status column);
  reconcile_stale_queued_to_unavailable and status_priority_order now skip
  virtual types via @virtual_types. Fixes the prod crash where the cron tried
  to update a non-existent era5_status column.
- ContactWeatherEnqueueWorker: build_era5_jobs → build_narr_jobs,
  era5_jobs_for_contact → narr_jobs_for_contact
- ContactLive: @era5 → @narr, maybe_enqueue_era5 → maybe_enqueue_narr;
  UI label "ERA5 (0.25°)" → "NARR (32 km)"
- Cron (runtime.exs): args types list now "narr" instead of "era5"
- /status: progress row, status-by-type card, totals.narr_profiles, and
  table-count lookup all target narr_profiles
- Drop obsolete Era5CdsJob schema + era5_cds_jobs table (inspection
  artifact from the retired CDS pipeline; 34 orphan rows in prod)
- Misc docstring/comment cleanups (skew_t, about, wgrib2, propagation_train)

Includes a regression test for the virtual-type crash.
2026-04-16 09:22:23 -05:00
79e16ae9d3
Give up on stuck :queued backfill contacts after 3 days
Contacts whose weather/hrrr/terrain/iemre status has been :queued for
more than 3 days (144 cron cycles) are flipped to :unavailable. The
set_enrichment_status!/3 helper is a no-op when the status is unchanged,
so a stuck row retains a stale updated_at — which is exactly the signal
we need for 'the cron has already tried and the data doesnt exist
upstream' (dead ASOS station, pre-2014 HRRR, out-of-grid IEMRE). Clears
~1,492 pre-existing stuck contacts and stops them from churning the
backfill queue.
2026-04-13 17:20:18 -05:00
a6a71e8c94
Fix BackfillEnqueueWorker test timeouts and credo length warnings
The three BackfillEnqueueWorkerTest cases were timing out because the
default types list includes :era5, which cascades through inline Oban
into Era5Client.poll_and_download where Process.sleep blocks for the
full 60s test timeout. Era5Client uses bare Req.get with no plug hook,
so it can't be stubbed via Req.Test the way the other clients are. Pass
explicit non-ERA5 types in the three affected cases — ERA5 has its own
coverage and these tests don't assert anything era5-specific.

Also replace two `length(results) > 0` checks in asos_nudge_test with
`results != []` to silence credo warnings.
2026-04-12 15:02:48 -05:00
c98aabfc15 Add BackfillEnqueueWorker tests (0→100% coverage) 2026-04-06 10:15:38 -05:00