graham/prop
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
No description
1473 commits 1 branch 0 tags 8.1 MiB
Find a file
Graham McIntire 03c91f4098
perf(db): tune autovacuum per-table + one-shot ANALYZE helper 
Two issues spotted in pg_stat_user_tables:

1. Most `hrrr_profiles` partitions have `last_autoanalyze IS NULL`.
   The partitions are append-only from the hourly grid worker, so
   autovacuum never crosses the analyze threshold — but without
   stats the planner falls back to estimates and picks bad joins
   on per-QSO lookups.

2. `contacts`, `grid_tasks`, `hrrr_fetch_tasks`, and
   `contact_common_volume_radar` see high UPDATE volumes
   (enrichment status flips, queue claim/complete cycles, mechanism
   classifier writes) but only accrue ~5% dead before the default
   20% / 10% autovacuum triggers fire. BackfillEnqueueWorker's
   status-priority ORDER BY + the Rust workers' FOR UPDATE SKIP
   LOCKED claim scan both want fresher stats than that.

Changes:
- Migration `20260424204656_tune_autovacuum_for_high_churn` sets
  per-table `autovacuum_*_scale_factor` + `_threshold = 50` on the
  four tables above. Aggressive enough to keep stats current;
  tame enough that vacuum doesn't thrash the Turing Pi 2 Postgres
  node. Reversible.
- `Weather.analyze_all/1` walks `pg_stat_user_tables`, runs
  `ANALYZE` on every table not auto-analyzed in the last 6 h,
  reports counts. Meant to be invoked once via
  `bin/microwaveprop rpc 'Microwaveprop.Weather.analyze_all()'`
  after deploy to seed statistics on the cold partitions;
  idempotent on re-run.
2026-04-24 15:48:46 -05:00
.forgejo/workflows ci(grid-rs): surface docker daemon diagnostics on wait-step failure 2026-04-20 08:41:53 -05:00
.github Add contact edit approval system with admin review queue 2026-04-11 16:15:49 -05:00
assets fix(weather): accept Rust profile shape in SoundingParams + WeatherLayers 2026-04-24 13:53:02 -05:00
bin Fix livebook path resolution in bin/notebook 2026-04-07 11:15:49 -05:00
config perf(weather): batch ASOS fetches into one multi-station request 2026-04-24 12:44:22 -05:00
docs infra(k8s): load-based HPAs, drop anti-affinity, retire talos5 2026-04-23 18:43:18 -05:00
k8s chore: update prop image to git.mcintire.me/graham/prop:main-1777063515-13fad93 [skip ci] 2026-04-24 20:47:29 +00:00
lib perf(db): tune autovacuum per-table + one-shot ANALYZE helper 2026-04-24 15:48:46 -05:00
lib_ml test: push coverage over 85% via PropagationAnalyze/Train smoke tests 2026-04-24 10:32:05 -05:00
notebooks refactor: normalize pos1/pos2 JSONB key to 'lon' everywhere 2026-04-17 09:10:32 -05:00
priv perf(db): tune autovacuum per-table + one-shot ANALYZE helper 2026-04-24 15:48:46 -05:00
rel Add Erlang clustering via libcluster and expose LiveDashboard 2026-04-06 09:46:48 -05:00
rust/prop_grid_rs fix: three more Rust/Elixir contract drift bugs on /map and /weather 2026-04-24 14:04:02 -05:00
scripts Add incremental prod-to-local DB sync script 2026-04-23 13:57:22 -05:00
test fix(weather): accept Rust profile shape in SoundingParams + WeatherLayers 2026-04-24 13:53:02 -05:00
vendor Vendor oban_pro 1.6.13 2026-04-09 14:14:49 -05:00
.credo.exs credo 2026-04-12 10:26:53 -05:00
.dockerignore Optimize Dockerfile layer caching 2026-04-01 10:32:56 -05:00
.formatter.exs initial 2026-03-28 11:28:47 -05:00
.gitignore Dual-write propagation scores to binary files on /data 2026-04-14 14:36:55 -05:00
.tool-versions update deps 2026-04-11 12:03:43 -05:00
AGENTS.md initial 2026-03-28 11:28:47 -05:00
algo.md docs(algo): align algo.md with Richardson-gated scoring implementation 2026-04-18 12:42:20 -05:00
app.json Migrate healthchecks from CHECKS to app.json 2026-04-04 11:24:56 -05:00
CLAUDE.md fix(maps): same reconnect/visibility fix for rover and weather hooks 2026-04-18 13:25:38 -05:00
Dockerfile fix(docker): use BuildKit cache mounts for apt so .debs skip the layer 2026-04-18 11:07:45 -05:00
elevation.md add elevation findings 2026-03-29 16:23:03 -05:00
LICENSE Add GPLv3 license 2026-04-23 13:57:07 -05:00
mix.exs chore: drop OpenTelemetry 2026-04-24 12:33:38 -05:00
mix.lock chore: drop OpenTelemetry 2026-04-24 12:33:38 -05:00
prediction.md HRRR forecast hours (f00-f18) with timeline map UI 2026-03-31 16:44:47 -05:00
README.md initial 2026-03-28 11:28:47 -05:00
tail_logs.sh Add type filter to backfill enqueue 2026-04-06 11:42:26 -05:00
updates.md Update seasonal scores from sounding data, document commercial link patterns 2026-04-07 11:45:44 -05:00

README.md

Microwaveprop

To start your Phoenix server:

  • Run mix setup to install and setup dependencies
  • Start Phoenix endpoint with mix phx.server or inside IEx with iex -S mix phx.server

Now you can visit localhost:4000 from your browser.

Ready to run in production? Please check our deployment guides.

Learn more