No description
Find a file
Graham McIntire 0ed47db8b6
Process one forecast hour per PropagationGridWorker perform
A full f00-f18 sweep takes ~170 min of wall time, longer than the
~90 min gap between deploys on this cluster. Over the last 7 days
every run was killed mid-sweep and zero runs completed. The
propagation_scores table only ever held the scraps from partial
runs that landed before the pod died, which is why the map looked
like it was showing "the current hour" (or nothing).

The worker now processes exactly one forecast hour per perform/1
(~8-10 min) and enqueues the next hour as a fresh Oban job. A cron
fire with empty args seeds the chain at f00; subsequent runs carry
run_time + forecast_hour args. At f18 the chain stops and the
pruner runs. Each step broadcasts propagation:updated immediately
so new hours appear on the map as they land.

Wall time per perform drops from ~170 min to ~10 min, so Lifeline's
2h rescue window is no longer a factor and a pod restart loses at
most one forecast hour instead of the whole sweep. Retries and
max_attempts now describe a single fh, not the whole chain.

Also: bump the :propagation queue from 1 to 2 slots so
PropagationPruneWorker can run alongside the chain job, drop the
worker timeout/1 from 90 min to 20 min to match single-fh runs,
and pull Lifeline rescue_after from 120 min to 45 min to keep the
safety net above the step timeout.

Adds a "Data from HH:MM UTC · Nh ago/now/+Nh" indicator above the
pipeline chip in both the mobile and desktop sidebars so the user
can tell which valid_time the map is showing when the bottom
timeline isn't visible.
2026-04-14 13:24:59 -05:00
.forgejo/workflows Install Docker CLI in CI runner container 2026-04-11 09:04:22 -05:00
.github Add contact edit approval system with admin review queue 2026-04-11 16:15:49 -05:00
assets Add grid squares and weather radar toggles to the weather map 2026-04-13 16:31:54 -05:00
bin Fix livebook path resolution in bin/notebook 2026-04-07 11:15:49 -05:00
config Process one forecast hour per PropagationGridWorker perform 2026-04-14 13:24:59 -05:00
docs Refresh 2026-04-13 recalibration report 2026-04-13 12:29:51 -05:00
k8s chore: update prop image to git.mcintire.me/graham/prop:main-1776183202-434fca2 [skip ci] 2026-04-14 16:15:09 +00:00
lib Process one forecast hour per PropagationGridWorker perform 2026-04-14 13:24:59 -05:00
lib_ml Retrain propagation ML on month-balanced HRRR → algorithm score 2026-04-14 08:16:08 -05:00
notebooks Auto-deploy analysis notebook as Livebook app 2026-04-07 11:15:49 -05:00
priv Retrain propagation ML on month-balanced HRRR → algorithm score 2026-04-14 08:16:08 -05:00
rel Add Erlang clustering via libcluster and expose LiveDashboard 2026-04-06 09:46:48 -05:00
scripts Extend recalibrate_algo.py with NEXRAD, native-duct, commercial sections 2026-04-13 12:14:05 -05:00
test Process one forecast hour per PropagationGridWorker perform 2026-04-14 13:24:59 -05:00
vendor Vendor oban_pro 1.6.13 2026-04-09 14:14:49 -05:00
.credo.exs credo 2026-04-12 10:26:53 -05:00
.dockerignore Optimize Dockerfile layer caching 2026-04-01 10:32:56 -05:00
.formatter.exs initial 2026-03-28 11:28:47 -05:00
.gitignore Fix /weather 500 on cold cache, untrack k8s secret 2026-04-12 13:56:59 -05:00
.tool-versions update deps 2026-04-11 12:03:43 -05:00
AGENTS.md initial 2026-03-28 11:28:47 -05:00
algo.md Wire NEXRAD, native ducts, and commercial links into scoring 2026-04-13 12:08:15 -05:00
app.json Migrate healthchecks from CHECKS to app.json 2026-04-04 11:24:56 -05:00
CLAUDE.md docs: update deployment notes for Kubernetes 2026-04-13 15:30:59 -05:00
Dockerfile Use NCEPLIBS-g2c v2.1.0 + enable wgrib2 PNG decoder 2026-04-14 11:06:04 -05:00
elevation.md add elevation findings 2026-03-29 16:23:03 -05:00
mix.exs Unblock CI: pull igniter into :prod for live_table compile 2026-04-12 17:58:13 -05:00
mix.lock Add live_table dep and convert /users to use it 2026-04-12 17:07:07 -05:00
prediction.md HRRR forecast hours (f00-f18) with timeline map UI 2026-03-31 16:44:47 -05:00
README.md initial 2026-03-28 11:28:47 -05:00
tail_logs.sh Add type filter to backfill enqueue 2026-04-06 11:42:26 -05:00
updates.md Update seasonal scores from sounding data, document commercial link patterns 2026-04-07 11:45:44 -05:00

Microwaveprop

To start your Phoenix server:

  • Run mix setup to install and setup dependencies
  • Start Phoenix endpoint with mix phx.server or inside IEx with iex -S mix phx.server

Now you can visit localhost:4000 from your browser.

Ready to run in production? Please check our deployment guides.

Learn more