graham/prop
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
prop/lib/microwaveprop
History
Graham McIntire bbf6481603
perf(nexrad): route fetch_decoded_frame through NexradCache 
Telemetry puts nexrad_decode_png at 1.76 s/call × 37,847 calls/h
— ~18.5 h of CPU across the cluster every real hour, the single
biggest CPU consumer in the system. The map-click path
(fetch_rain_cells) already cached decoded frames by 5-min
bucket, but the per-contact CommonVolumeRadarWorker path
(fetch_decoded_frame) went straight to network + decode on every
call. Backfill means many contacts share a 5-min window, so the
same 66 MB frame was being decoded dozens of times.

Wire fetch_decoded_frame through NexradCache keyed on the
rounded timestamp. Add a 20-entry size cap in NexradCache so
backfill processing contacts in random timestamp order can't
grow the ETS table to hundreds of GB. Each frame is ~66 MB, so
20 = ~1.3 GB worst case, well under typical pod memory.

Expected impact: cuts sustained decode load by an order of
magnitude depending on backfill temporal locality; map-click
path is unchanged.
2026-04-19 09:05:55 -05:00
..
accounts feat(auth): self-service password reset 2026-04-18 08:42:35 -05:00
backtest Fix all 12 dialyzer warnings 2026-04-11 18:08:18 -05:00
beacon_monitors Add dialyzer specs and types across the codebase 2026-04-12 08:55:04 -05:00
beacons Short-circuit RangeEstimate below 5.76 GHz to stop browser lockup 2026-04-14 16:56:24 -05:00
callsign_location Drop gridmap.org dependency, resolve callsigns locally 2026-04-12 17:43:29 -05:00
commercial Fix all remaining credo --strict issues (0 issues) 2026-04-12 10:26:53 -05:00
ionosphere feat(telemetry): broaden Instrument span coverage 2026-04-18 17:25:33 -05:00
prom_ex refactor(map): drop the 7-day outlook strip, fix missing Analysis, trim timeline 2026-04-19 08:17:47 -05:00
propagation feat(map): shareable URL params for view/band/time 2026-04-19 08:53:07 -05:00
qrz feat(telemetry): broaden Instrument span coverage 2026-04-18 17:25:33 -05:00
radio feat(contacts): private flag with scope-aware visibility 2026-04-18 17:49:01 -05:00
space_weather feat(telemetry): broaden Instrument span coverage 2026-04-18 17:25:33 -05:00
terrain feat(telemetry): broaden Instrument span coverage 2026-04-18 17:25:33 -05:00
weather perf(nexrad): route fetch_decoded_frame through NexradCache 2026-04-19 09:05:55 -05:00
workers fix(propagation): hourly grid chain preempts MRMS/prune backlog 2026-04-19 08:42:22 -05:00
accounts.ex feat(auth): self-service password reset 2026-04-18 08:42:35 -05:00
application.ex feat(telemetry): PromEx Prometheus exporter at /metrics 2026-04-18 16:39:39 -05:00
backtest.ex refactor: normalize pos1/pos2 JSONB key to 'lon' everywhere 2026-04-17 09:10:32 -05:00
beacon_monitors.ex Add dialyzer specs and types across the codebase 2026-04-12 08:55:04 -05:00
beacons.ex Convert /beacons approved list to live_table 2026-04-12 17:15:05 -05:00
cache.ex Cache /contacts, fix map blink, make mode optional 2026-04-12 12:47:25 -05:00
callsign_location.ex Drop gridmap.org dependency, resolve callsigns locally 2026-04-12 17:43:29 -05:00
commercial.ex Hoist commercial link query + tie pipeline chip to ScoresFile 2026-04-14 15:01:12 -05:00
format.ex feat(format): miles-primary distance formatting sitewide 2026-04-18 15:53:40 -05:00
geo.ex Fix all remaining credo --strict issues (0 issues) 2026-04-12 10:26:53 -05:00
geocoder.ex feat(telemetry): broaden Instrument span coverage 2026-04-18 17:25:33 -05:00
instrument.ex feat(telemetry): wide instrumentation + bump hrrr to 2 per pod 2026-04-18 16:33:34 -05:00
ionosphere.ex Sporadic-E physics module + Ionosphere.nearest_foes 2026-04-15 14:43:48 -05:00
mailer.ex Send mail as prop@w5isp.com with reply-to graham@mcintire.me 2026-04-12 14:00:38 -05:00
markdown.ex Add dialyzer specs and types across the codebase 2026-04-12 08:55:04 -05:00
prom_ex.ex feat(telemetry): PromEx Prometheus exporter at /metrics 2026-04-18 16:39:39 -05:00
propagation.ex refactor(map): drop the 7-day outlook strip, fix missing Analysis, trim timeline 2026-04-19 08:17:47 -05:00
qrz.ex Drop gridmap.org dependency, resolve callsigns locally 2026-04-12 17:43:29 -05:00
radio.ex feat(contacts): private flag with scope-aware visibility 2026-04-18 17:49:01 -05:00
release.ex Drop propagation_scores table, ScoresFile is the only store 2026-04-14 15:13:01 -05:00
repo.ex initial 2026-03-28 11:28:47 -05:00
repo_listener.ex Replace polling with Postgres LISTEN/NOTIFY for live updates 2026-04-02 16:26:22 -05:00
space_weather.ex SpaceWeather: SWPC JSON ingestion (Kp, F10.7, GOES X-ray) 2026-04-15 14:55:24 -05:00
terrain.ex Add dialyzer specs and types across the codebase 2026-04-12 08:55:04 -05:00
weather.ex feat(telemetry): wide instrumentation + bump hrrr to 2 per pod 2026-04-18 16:33:34 -05:00