prop/docs
Graham McIntire 4f82bd691e
perf(prop): cap ScoreCache LRU + drop eager warmers to fix hot-pod OOMs
ScoreCache held 437 entries × ~1.94 MiB each (~850 MiB per pod) of
{band_mhz, valid_time} grids — data already on NFS as compact .prop
files. NotifyListener and ScoreCacheReconciler both eagerly materialised
every band into ETS on each Rust completion / 60s sweep, so 5 hot
replicas wasted ~4.2 GiB of redundant cache and OOMed at 6 GiB.

Bound the cache to 32 entries with eviction by oldest valid_time, drop
the eager warm loops, and let LiveView callers lazy-fill via the
existing ScoresFile read path. ScoreCacheReconciler had no remaining
purpose and is deleted; runbook + prom_ex counters updated to match.

Steady-state cache footprint ~60 MiB per pod instead of ~850 MiB.
2026-04-24 18:49:36 -05:00
..
algo-reports feat(propagation): per-band weight calibration from full-corpus correlation 2026-04-18 10:19:26 -05:00
plans chore(mrms): delete pipeline — consumer AsosAdjustmentWorker already disabled 2026-04-19 17:25:59 -05:00
research Phase 3 spike (parked) + Phase 6: temperature anomaly 2026-04-10 08:47:11 -05:00
algo_commercial.md Refactor HRRR fetch to batch points per hour 2026-03-30 17:21:47 -05:00
algo_original.md Refactor HRRR fetch to batch points per hour 2026-03-30 17:21:47 -05:00
algo_updated_with_data.md Refactor HRRR fetch to batch points per hour 2026-03-30 17:21:47 -05:00
analysis_output.txt Normalize ML features to prevent NaN gradient explosion 2026-04-01 09:08:33 -05:00
analysis_output_solar.txt Normalize ML features to prevent NaN gradient explosion 2026-04-01 09:08:33 -05:00
findings10.md Correct Finding 10: Great Lakes firing squad explains 24G PH anomaly 2026-04-01 11:07:58 -05:00
goes-r-research.md HRRR forecast hours (f00-f18) with timeline map UI 2026-03-31 16:44:47 -05:00
RRFS.md Add meteorologist feedback on model choice to RRFS doc 2026-03-31 09:27:23 -05:00
runbook_propagation_pipeline.md perf(prop): cap ScoreCache LRU + drop eager warmers to fix hot-pod OOMs 2026-04-24 18:49:36 -05:00