perf(weather): drop GridCache valid-time cap from 24 → 8
Found while auditing pod memory: :weather_grid_cache is by far the largest ETS table (~32 MiB compressed per entry, vs ~2 MiB/entry for ScoreCache) because each cell is a 22-field weather row instead of a scalar score. Cap of 24 meant the cache could hoard up to ~768 MiB on a single pod, and hot pods sit at 3-5 GiB against a 6 GiB limit — most of that headroom belongs to GridCache. The cap was sized for 'analysis + 18 forecasts + stragglers' but the real access pattern is current-hour + a short forward scrub. Beyond that, fall back to ScalarFile.read_bounds (sub-100 ms per file) instead of permanently parking ~500 MiB on data nobody looks at. Worst-case GridCache spend drops 768 MiB → 256 MiB. Steady-state win is smaller (most pods don't fill all 24 slots), but bounds the tail.
This commit is contained in:
parent
9bec5721d0
commit
803130ee64
1 changed files with 13 additions and 4 deletions
|
|
@ -988,10 +988,19 @@ defmodule Microwaveprop.Weather do
|
|||
end
|
||||
end
|
||||
|
||||
# Cap on cached valid_times. HRRR runs hourly with 18 forecast hours, so
|
||||
# 24 covers analysis + 18 forecasts plus a few late stragglers from the
|
||||
# previous run while a user is scrubbing.
|
||||
@grid_cache_valid_time_cap 24
|
||||
# Cap on cached valid_times. Each entry is a chunked map of the full
|
||||
# ~92k-cell CONUS grid with 22 fields per cell, ~32 MiB compressed in
|
||||
# ETS. The previous cap of 24 meant the cache could hoard up to
|
||||
# ~768 MiB on a single pod against a 6 GiB memory limit (the headroom
|
||||
# that disappeared in the 2026-05-03 OOM cascade).
|
||||
#
|
||||
# 8 covers the current hour plus the next ~7 forecast hours — the
|
||||
# typical user scrub window. Hours beyond 8 fall back to a disk read
|
||||
# (`ScalarFile.read_bounds`) which is sub-100 ms per file, so rare
|
||||
# deep-forecast scrubs pay a one-shot latency hit instead of every
|
||||
# pod permanently parking ~500 MiB on data the user almost never
|
||||
# looks at. Worst case ETS spend drops from 768 MiB → 256 MiB.
|
||||
@grid_cache_valid_time_cap 8
|
||||
|
||||
# Hydrate GridCache from the on-disk ScalarFile so concurrent viewport
|
||||
# reads for the same valid_time don't each gunzip+msgpack-decode the
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue