prop/lib/microwaveprop
Graham McIntire 32fdb2583d
Shrink PropagationGridWorker per-forecast-hour memory footprint
Two related optimizations in the propagation chain hot path. Both
land on the f01..f18 step that was previously OOM-killing prod
pods after the HRRR pressure-level footprint halved wasn't enough
to fit inside 4 Gi.

1. Skip the GridCache broadcast on forecast hours.

   /weather only ever renders the analysis hour (latest_grid_valid_time
   feeds the map). Building 92k rows, serializing them through PubSub,
   and rebuilding the {lat,lon}→row map on all three replicas was
   pure waste for f01..f18 — no consumer was reading that data. Only
   f00 now calls build_grid_cache_rows + broadcast_put. Point lookups
   for non-analysis hours still work through ProfilesFile on disk
   (weather_point_detail_from_profiles/3) exactly as before.

2. Fold replace_scores into a single streaming pass.

   The old path did `Enum.to_list/1` on the ~460k-entry score stream
   followed by `Enum.group_by/2`, holding two full copies of the grid
   before any file was written. A single `Enum.reduce/3` that folds
   each score into a per-band accumulator keeps only one copy and
   eliminates the group_by intermediate entirely. The public
   signature — an Enumerable in, {:ok, count} out — is unchanged.
2026-04-15 08:36:53 -05:00
..
accounts Send mail as prop@w5isp.com with reply-to graham@mcintire.me 2026-04-12 14:00:38 -05:00
backtest Fix all 12 dialyzer warnings 2026-04-11 18:08:18 -05:00
beacon_monitors Add dialyzer specs and types across the codebase 2026-04-12 08:55:04 -05:00
beacons Short-circuit RangeEstimate below 5.76 GHz to stop browser lockup 2026-04-14 16:56:24 -05:00
callsign_location Drop gridmap.org dependency, resolve callsigns locally 2026-04-12 17:43:29 -05:00
commercial Fix all remaining credo --strict issues (0 issues) 2026-04-12 10:26:53 -05:00
propagation Persist weather grid for every forecast hour via ProfilesFile 2026-04-14 16:30:40 -05:00
qrz Drop gridmap.org dependency, resolve callsigns locally 2026-04-12 17:43:29 -05:00
radio Drop gridmap.org dependency, resolve callsigns locally 2026-04-12 17:43:29 -05:00
terrain Fix all remaining credo --strict issues (0 issues) 2026-04-12 10:26:53 -05:00
weather Split HRRR pressure levels for grid hot path vs per-contact profiles 2026-04-15 08:19:33 -05:00
workers Shrink PropagationGridWorker per-forecast-hour memory footprint 2026-04-15 08:36:53 -05:00
accounts.ex User profiles at /u/:callsign, flexible band input, assorted UX cleanup 2026-04-12 16:13:08 -05:00
application.ex Persist weather grid for every forecast hour via ProfilesFile 2026-04-14 16:30:40 -05:00
backtest.ex Phase 9.3, 9.4, and Phase 3 NEXRAD pipeline 2026-04-10 12:48:36 -05:00
beacon_monitors.ex Add dialyzer specs and types across the codebase 2026-04-12 08:55:04 -05:00
beacons.ex Convert /beacons approved list to live_table 2026-04-12 17:15:05 -05:00
cache.ex Cache /contacts, fix map blink, make mode optional 2026-04-12 12:47:25 -05:00
callsign_location.ex Drop gridmap.org dependency, resolve callsigns locally 2026-04-12 17:43:29 -05:00
commercial.ex Hoist commercial link query + tie pipeline chip to ScoresFile 2026-04-14 15:01:12 -05:00
geo.ex Fix all remaining credo --strict issues (0 issues) 2026-04-12 10:26:53 -05:00
geocoder.ex Drop gridmap.org dependency, resolve callsigns locally 2026-04-12 17:43:29 -05:00
mailer.ex Send mail as prop@w5isp.com with reply-to graham@mcintire.me 2026-04-12 14:00:38 -05:00
markdown.ex Add dialyzer specs and types across the codebase 2026-04-12 08:55:04 -05:00
propagation.ex Shrink PropagationGridWorker per-forecast-hour memory footprint 2026-04-15 08:36:53 -05:00
qrz.ex Drop gridmap.org dependency, resolve callsigns locally 2026-04-12 17:43:29 -05:00
radio.ex Convert /admin/contact-edits list to live_table 2026-04-12 17:18:47 -05:00
release.ex Drop propagation_scores table, ScoresFile is the only store 2026-04-14 15:13:01 -05:00
repo.ex initial 2026-03-28 11:28:47 -05:00
repo_listener.ex Replace polling with Postgres LISTEN/NOTIFY for live updates 2026-04-02 16:26:22 -05:00
terrain.ex Add dialyzer specs and types across the codebase 2026-04-12 08:55:04 -05:00
weather.ex Extend skew-T plot to cover the full troposphere 2026-04-14 17:25:58 -05:00