graham/prop
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
No description
1953 commits 2 branches 0 tags 9.3 MiB
Find a file
Graham McIntire 95976b6b90
Some checks failed
Build prop-grid-rs / Test, build, push (push) Successful in 5m3s
Details
Build and Push / Build and Push Docker Image (push) Failing after 5m57s
Details
perf(grid-rs): drop legacy .mp.gz write, parallelize decodes, overlap fetch+decode, cap retries 
Four optimisations targeting the 11.4s chain-step critical path:

1. Drop legacy .mp.gz dual-write (~2.5-2.8s, 24% of step)
   The Elixir reader prefers .sgrid; the chunked gzip+msgpack write was
   pure overhead on the Rust write path. Removed scalar_future from both
   run_chain_step and run_analysis_step.

2. Parallelize wgrib2 decodes (~0.5-1s)
   sfc and prs decode ran serially (await sfc, then await prs). Both now
   spawn concurrently on the blocking pool.

3. Overlap sfc decode with prs fetch (~1-2s)
   sfc (9 messages) resolves before prs (39 messages). Start the sfc
   wgrib2 subprocess as soon as the sfc blob lands instead of waiting
   for both blobs. The sfc decode hides behind the remaining prs fetch.

4. Cap grib range retries from 5 to 3
   2^n exponential backoff meant 1+2+4+8+16 = 31s worst-case per failed
   range. Three retries caps at 1+2+4 = 7s. 404s already fail fast.

Also: instrument .sgrid write (observe_stage) so it appears in the
prop_grid_rs_stage_duration_seconds histogram alongside fetch/decode/derive.

Expected mean chain step: ~6-7s (down from 11.4s).
2026-08-01 12:48:01 -05:00
.forgejo/workflows ci: add postgres health-check options, remove fragile /dev/tcp wait 2026-07-28 08:31:28 -05:00
.github Add contact edit approval system with admin review queue 2026-04-11 16:15:49 -05:00
assets feat(weather): add duct cutoff band map layer 2026-05-15 19:51:24 -05:00
bin Fix livebook path resolution in bin/notebook 2026-04-07 11:15:49 -05:00
config feat: remove skippy HRRR proxy + disk cache, fetch directly from NOAA S3 2026-07-28 14:36:54 -05:00
docs feat: remove skippy HRRR proxy + disk cache, fetch directly from NOAA S3 2026-07-28 14:36:54 -05:00
k8s fix: revert prop-grid-rs memory limit to 3Gi after OOM at 1.5Gi 2026-08-01 12:24:05 -05:00
lib Fix LiveView crashes and stabilize test suite 2026-08-01 09:26:47 -05:00
lib_ml fix: resolve all compiler warnings except framework-generated phoenix_component_verify 2026-05-29 10:18:52 -05:00
nix fix: point cargo pre-commit hooks at rust/prop_grid_rs/Cargo.toml 2026-07-20 19:52:18 -05:00
notebooks refactor: normalize pos1/pos2 JSONB key to 'lon' everywhere 2026-04-17 09:10:32 -05:00
priv fix: resolve 27 security, architecture, test, and performance audit findings 2026-07-27 18:19:37 -05:00
rel Add Erlang clustering via libcluster and expose LiveDashboard 2026-04-06 09:46:48 -05:00
rust/prop_grid_rs perf(grid-rs): drop legacy .mp.gz write, parallelize decodes, overlap fetch+decode, cap retries 2026-08-01 12:48:01 -05:00
scripts update ferqs 2026-07-20 19:55:52 -05:00
test Fix LiveView crashes and stabilize test suite 2026-08-01 09:26:47 -05:00
vendor chore(deps): bump vendored oban_web from 2.12.3 to 2.12.5 2026-07-30 08:30:38 -05:00
.credo.exs fix: resolve all 281 credo issues across source and test files 2026-07-29 09:04:21 -05:00
.dockerignore Optimize Dockerfile layer caching 2026-04-01 10:32:56 -05:00
.envrc Track .envrc with use flake, move secrets to .envrc.local 2026-07-20 18:35:13 -05:00
.formatter.exs initial 2026-03-28 11:28:47 -05:00
.gitignore Track .envrc with use flake, move secrets to .envrc.local 2026-07-20 18:35:13 -05:00
.tool-versions feat: add Nix flake dev environment with Erlang 29.0.3 + Elixir 1.20.2 2026-07-15 12:57:10 -05:00
AGENTS.md initial 2026-03-28 11:28:47 -05:00
algo.md feat(weather): /weather-ca endpoint shows HRDPS-only Canadian data 2026-04-30 12:08:48 -05:00
app.json Migrate healthchecks from CHECKS to app.json 2026-04-04 11:24:56 -05:00
bugs.md fix: 6 bugs from bugs.md — ADIF parser, Es MUF, enrichment reset, profile privacy, HRRR OOM risks 2026-05-12 09:37:09 -05:00
CLAUDE.md prop-pipeline: .sgrid dense scalars, HRDPS rotated-pole decode-once, memory trim 2026-08-01 09:11:59 -05:00
Dockerfile fix: use apply/3 to avoid undefined-module warnings with Conditional ML module 2026-07-28 08:19:23 -05:00
Dockerfile.base fix: resolve 27 security, architecture, test, and performance audit findings 2026-07-27 18:19:37 -05:00
flake.lock feat: add Nix flake dev environment with Erlang 29.0.3 + Elixir 1.20.2 2026-07-15 12:57:10 -05:00
flake.nix feat: add Nix flake dev environment with Erlang 29.0.3 + Elixir 1.20.2 2026-07-15 12:57:10 -05:00
LICENSE Add GPLv3 license 2026-04-23 13:57:07 -05:00
Makefile Fix LiveView crashes and stabilize test suite 2026-08-01 09:26:47 -05:00
mix.exs fix: resolve all 281 credo issues across source and test files 2026-07-29 09:04:21 -05:00
mix.lock fix: resolve all 281 credo issues across source and test files 2026-07-29 09:04:21 -05:00
nix.md feat: add Nix flake dev environment with Erlang 29.0.3 + Elixir 1.20.2 2026-07-15 12:57:10 -05:00
prediction.md HRRR forecast hours (f00-f18) with timeline map UI 2026-03-31 16:44:47 -05:00
README.md initial 2026-03-28 11:28:47 -05:00
shell.nix feat: add Nix flake dev environment with Erlang 29.0.3 + Elixir 1.20.2 2026-07-15 12:57:10 -05:00
tail_logs.sh Add type filter to backfill enqueue 2026-04-06 11:42:26 -05:00
updates.md Update seasonal scores from sounding data, document commercial link patterns 2026-04-07 11:45:44 -05:00

README.md

Microwaveprop

To start your Phoenix server:

  • Run mix setup to install and setup dependencies
  • Start Phoenix endpoint with mix phx.server or inside IEx with iex -S mix phx.server

Now you can visit localhost:4000 from your browser.

Ready to run in production? Please check our deployment guides.

Learn more