graham/prop - Forgejo: Beyond coding. We forge.

graham/prop

Fork 0

Commit graph

Author	SHA1	Message	Date
Graham McIntire	bbf6481603	perf(nexrad): route fetch_decoded_frame through NexradCache Telemetry puts nexrad_decode_png at 1.76 s/call × 37,847 calls/h — ~18.5 h of CPU across the cluster every real hour, the single biggest CPU consumer in the system. The map-click path (fetch_rain_cells) already cached decoded frames by 5-min bucket, but the per-contact CommonVolumeRadarWorker path (fetch_decoded_frame) went straight to network + decode on every call. Backfill means many contacts share a 5-min window, so the same 66 MB frame was being decoded dozens of times. Wire fetch_decoded_frame through NexradCache keyed on the rounded timestamp. Add a 20-entry size cap in NexradCache so backfill processing contacts in random timestamp order can't grow the ETS table to hundreds of GB. Each frame is ~66 MB, so 20 = ~1.3 GB worst case, well under typical pod memory. Expected impact: cuts sustained decode load by an order of magnitude depending on backfill temporal locality; map-click path is unchanged.	2026-04-19 09:05:55 -05:00
Graham McIntire	e7a7ae073d	Phase 9.3, 9.4, and Phase 3 NEXRAD pipeline Task 9.3 - Weight recalibration via gradient descent: - Recalibrator module fits logistic regression weights using Nx - Trains on QSO positives vs random baseline negatives - Cross-validates by month, normalizes weights to sum to 1.0 - Mix task: mix recalibrate_scorer --sample 5000 --epochs 2000 Task 9.4 - Side-by-side scorer comparison: - ScorerDiff.compare/3 re-scores grid with old vs new weights - Reports mean diff, regressions, improvements, per-band breakdown - Mix task: mix scorer_diff --new-weights '{...}' Phase 3 - NEXRAD ingestion pipeline: - NexradClient fetches IEM n0q composite PNGs, extracts per-point box statistics (mean/max dBZ, texture variance) - NexradObservation schema with unique (lat, lon, observed_at) - NexradWorker on :nexrad queue for background processing - nexrad_texture backtest feature in Features module - mix nexrad_backfill --limit 200 All tasks added to AdminTaskWorker and Release for production use. 1116 tests, 0 failures.	2026-04-10 12:48:36 -05:00

Author

SHA1

Message

Date

Graham McIntire

bbf6481603

perf(nexrad): route fetch_decoded_frame through NexradCache

Telemetry puts nexrad_decode_png at 1.76 s/call × 37,847 calls/h
— ~18.5 h of CPU across the cluster every real hour, the single
biggest CPU consumer in the system. The map-click path
(fetch_rain_cells) already cached decoded frames by 5-min
bucket, but the per-contact CommonVolumeRadarWorker path
(fetch_decoded_frame) went straight to network + decode on every
call. Backfill means many contacts share a 5-min window, so the
same 66 MB frame was being decoded dozens of times.

Wire fetch_decoded_frame through NexradCache keyed on the
rounded timestamp. Add a 20-entry size cap in NexradCache so
backfill processing contacts in random timestamp order can't
grow the ETS table to hundreds of GB. Each frame is ~66 MB, so
20 = ~1.3 GB worst case, well under typical pod memory.

Expected impact: cuts sustained decode load by an order of
magnitude depending on backfill temporal locality; map-click
path is unchanged.

2026-04-19 09:05:55 -05:00

Graham McIntire

e7a7ae073d

Phase 9.3, 9.4, and Phase 3 NEXRAD pipeline

Task 9.3 - Weight recalibration via gradient descent:
- Recalibrator module fits logistic regression weights using Nx
- Trains on QSO positives vs random baseline negatives
- Cross-validates by month, normalizes weights to sum to 1.0
- Mix task: mix recalibrate_scorer --sample 5000 --epochs 2000

Task 9.4 - Side-by-side scorer comparison:
- ScorerDiff.compare/3 re-scores grid with old vs new weights
- Reports mean diff, regressions, improvements, per-band breakdown
- Mix task: mix scorer_diff --new-weights '{...}'

Phase 3 - NEXRAD ingestion pipeline:
- NexradClient fetches IEM n0q composite PNGs, extracts per-point
  box statistics (mean/max dBZ, texture variance)
- NexradObservation schema with unique (lat, lon, observed_at)
- NexradWorker on :nexrad queue for background processing
- nexrad_texture backtest feature in Features module
- mix nexrad_backfill --limit 200

All tasks added to AdminTaskWorker and Release for production use.
1116 tests, 0 failures.

2026-04-10 12:48:36 -05:00

2 commits