prop/lib/microwaveprop
Graham McIntire 03c91f4098
perf(db): tune autovacuum per-table + one-shot ANALYZE helper
Two issues spotted in pg_stat_user_tables:

1. Most `hrrr_profiles` partitions have `last_autoanalyze IS NULL`.
   The partitions are append-only from the hourly grid worker, so
   autovacuum never crosses the analyze threshold — but without
   stats the planner falls back to estimates and picks bad joins
   on per-QSO lookups.

2. `contacts`, `grid_tasks`, `hrrr_fetch_tasks`, and
   `contact_common_volume_radar` see high UPDATE volumes
   (enrichment status flips, queue claim/complete cycles, mechanism
   classifier writes) but only accrue ~5% dead before the default
   20% / 10% autovacuum triggers fire. BackfillEnqueueWorker's
   status-priority ORDER BY + the Rust workers' FOR UPDATE SKIP
   LOCKED claim scan both want fresher stats than that.

Changes:
- Migration `20260424204656_tune_autovacuum_for_high_churn` sets
  per-table `autovacuum_*_scale_factor` + `_threshold = 50` on the
  four tables above. Aggressive enough to keep stats current;
  tame enough that vacuum doesn't thrash the Turing Pi 2 Postgres
  node. Reversible.
- `Weather.analyze_all/1` walks `pg_stat_user_tables`, runs
  `ANALYZE` on every table not auto-analyzed in the last 6 h,
  reports counts. Meant to be invoked once via
  `bin/microwaveprop rpc 'Microwaveprop.Weather.analyze_all()'`
  after deploy to seed statistics on the cold partitions;
  idempotent on re-run.
2026-04-24 15:48:46 -05:00
..
accounts chore(specs): add @spec to public API in accounts/release/backtest 2026-04-21 13:32:42 -05:00
backtest fix(dialyzer): clear 125+ warnings under strict flags 2026-04-21 10:30:06 -05:00
beacon_monitors Add dialyzer specs and types across the codebase 2026-04-12 08:55:04 -05:00
beacons Short-circuit RangeEstimate below 5.76 GHz to stop browser lockup 2026-04-14 16:56:24 -05:00
callsign_location Drop gridmap.org dependency, resolve callsigns locally 2026-04-12 17:43:29 -05:00
commercial test: push line coverage from 71.7% → 78.67% 2026-04-23 18:43:18 -05:00
ionosphere fix(ionosphere): disable TLS verify for GIRO to unbreak ionosonde fetch 2026-04-22 09:49:03 -05:00
prom_ex feat(observability): instrument PropagationGridWorker with Instrument.span 2026-04-21 13:29:09 -05:00
propagation fix: three more Rust/Elixir contract drift bugs on /map and /weather 2026-04-24 14:04:02 -05:00
qrz feat(telemetry): broaden Instrument span coverage 2026-04-18 17:25:33 -05:00
radio feat(adif): carry NOTES / COMMENT into contact notes 2026-04-23 11:05:39 -05:00
space_weather fix(ingest): dedupe ASOS duplicates + shrink SWPC decode errors 2026-04-21 16:39:17 -05:00
terrain fix(dialyzer): clear 125+ warnings under strict flags 2026-04-21 10:30:06 -05:00
weather fix: three more Rust/Elixir contract drift bugs on /map and /weather 2026-04-24 14:04:02 -05:00
workers fix(weather): reconcile hrrr_status + iemre_status stuck in :queued 2026-04-24 13:38:52 -05:00
accounts.ex chore(specs): add @spec to public API in accounts/release/backtest 2026-04-21 13:32:42 -05:00
application.ex chore: drop OpenTelemetry 2026-04-24 12:33:38 -05:00
backtest.ex chore(specs): add @spec to public API in accounts/release/backtest 2026-04-21 13:32:42 -05:00
beacon_monitors.ex Add dialyzer specs and types across the codebase 2026-04-12 08:55:04 -05:00
beacons.ex fix(dialyzer): actually fix LiveTable warnings instead of hiding them 2026-04-21 12:58:10 -05:00
cache.ex fix(cache): add periodic TTL sweeper to bound ETS growth 2026-04-21 13:19:53 -05:00
callsign_location.ex Drop gridmap.org dependency, resolve callsigns locally 2026-04-12 17:43:29 -05:00
commercial.ex fix(dialyzer): actually fix LiveTable warnings instead of hiding them 2026-04-21 12:58:10 -05:00
format.ex refactor(format): consolidate number/bytes formatters + handle flaky IEM transports 2026-04-21 17:09:41 -05:00
geo.ex Fix all remaining credo --strict issues (0 issues) 2026-04-12 10:26:53 -05:00
geocoder.ex feat(telemetry): broaden Instrument span coverage 2026-04-18 17:25:33 -05:00
instrument.ex feat(telemetry): wide instrumentation + bump hrrr to 2 per pod 2026-04-18 16:33:34 -05:00
ionosphere.ex refactor(db): use explicit on_conflict set clauses over :replace_all_except 2026-04-21 14:22:15 -05:00
mailer.ex Send mail as prop@w5isp.com with reply-to graham@mcintire.me 2026-04-12 14:00:38 -05:00
markdown.ex Add dialyzer specs and types across the codebase 2026-04-12 08:55:04 -05:00
moon.ex feat(eme): add /eme Earth-Moon-Earth calculator 2026-04-23 16:18:20 -05:00
oban_error_reporter.ex feat(observability): attach Oban exception telemetry tap 2026-04-21 14:13:53 -05:00
prom_ex.ex fix: stabilize flaky tests + silence PromEx Oban poller in test 2026-04-21 13:49:07 -05:00
propagation.ex fix(propagation): bound ScoreCache to 18-hour forecast window 2026-04-22 08:34:45 -05:00
qrz.ex Drop gridmap.org dependency, resolve callsigns locally 2026-04-12 17:43:29 -05:00
radio.ex fix(dialyzer): clear 125+ warnings under strict flags 2026-04-21 10:30:06 -05:00
release.ex chore(specs): add @spec to public API in accounts/release/backtest 2026-04-21 13:32:42 -05:00
repo.ex initial 2026-03-28 11:28:47 -05:00
repo_listener.ex chore(specs): add @spec to public API in accounts/release/backtest 2026-04-21 13:32:42 -05:00
space_weather.ex refactor(db): use explicit on_conflict set clauses over :replace_all_except 2026-04-21 14:22:15 -05:00
terrain.ex Add dialyzer specs and types across the codebase 2026-04-12 08:55:04 -05:00
weather.ex perf(db): tune autovacuum per-table + one-shot ANALYZE helper 2026-04-24 15:48:46 -05:00