Commit graph

5 commits

Author SHA1 Message Date
Graham McInitre
b3ad3bdbe6 fix: sweep bugs across Python MQTT listener, Elixir PSKR/propagation, and Rust
Python pskr_mqtt_listen.py:
- Guard against malformed CONNACK/SUBACK/PUBLISH packets
- Fix _s() truthiness: is not None instead of if v (zero SNR was lost)
- EINTR-safe select loop, OOB data handling, VBInt overflow check
- Proper socket cleanup with try/finally + DISCONNECT on shutdown

Elixir:
- Log dropped spot errors in aggregator (was silently discarded)
- Wire retain_scores_window into NotifyListener chain completion
- Recurse sweep_tmp_dir into band/weather_scalars subdirectories
- Rescue recalibrator.run/0 to write failed status row on crash
- Handle nil in fmt_snr/fmt_callsigns (no more nil dB crashes)
- Replace Stream.run with Enum.reduce logging exits in poll_worker
- Handle File.stat race in ms_footprints prune, grid_center nil log
- Fix unused variables, stale comments, missing get_path!/1

Rust:
- Graceful JoinError handling in fetcher/hrdps_fetcher (no more panics)
- round_to_5min returns Option (leap-second safe)
- Acquire/Release ordering on shutdown flags (was Relaxed)
- Parameterized NOTIFY in db.rs, defensive scalar sweep, NaN guard
- OsString::push for tmp naming, clippy fixes
2026-07-20 19:48:45 -05:00
Graham McInitre
4ac606e682 fix: ensure all unused historical NFS data gets cleaned up
Close 6 cleanup gaps on the shared /data NFS mount:

- HRDPS .hrdps.prop score files: parse_valid_time regex now matches
  the .hrdps.prop extension, so these files are found and pruned
- HRDPS scalar dirs (weather_scalars/{iso}.hrdps/): strip .hrdps suffix
  before DateTime.from_iso8601 in list_valid_time_dirs
- ScalarFile: prune_older_than was defined but never called from
  Propagation.prune_old_scores - now called alongside Scores/Profiles
- Orphan .tmp.* files: sweep all three base dirs (scores/profiles/scalars)
  for .tmp.* files left by crashed atomic-write processes
- Building cache (/data/buildings/): add MsFootprints.prune_older_than
  with 30-day mtime cutoff
- Canopy cache (/data/canopy/ + staging/): add Canopy.prune_older_than
  with 30-day cutoff, cleanup empty staging dir

All cleanup is cutoff-based (older than X time) so if the prune worker
is missed for any period it catches up fully on the next run.

PropagationPruneWorker updated to call all new cleanup functions.
2026-07-20 09:27:36 -05:00
cea2507fac
test: HTTP injection for MsFootprints/HrdpsClient/NexradClient (79.68%)
- MsFootprints (51% → 93%): http_get injection for dataset_index and
  download_tile, with stubbed CSV parsing, empty CSV, caching, and
  disk-cache-skip tests
- HrdpsClient (47% → 71%): http_get/http_head injection for fetch_grid
  and cycle_available, with stubbed probe, success/failure/transport-
  error tests, plus fetch_grid error path
- NexradClient (57% → 58%): http_get injection, fetch_frame success
  path with valid PNG stub, process_frame coverage
- HrrrNativeClient: http_get injection for fetch_idx (no direct test
  since function is private)

Coverage: 79.43% → 79.68% (need 0.32% more)
2026-05-07 16:36:17 -05:00
a290c99184
fix(buildings): use explicit step in descending range in encode_quadkey
zoom..1 without a step raises a runtime warning in Elixir 1.19 and was
causing the BulkFetch Task to crash immediately. Changed to zoom..1//-1.
2026-04-26 10:54:44 -05:00
5a97776fd5
feat(buildings): MS Global ML Building Footprints quadkey + tile fetcher
Phase 1 of building-blockage support: implements quadkey math (Bing/MS
tile encoding), looks up the MS dataset-links index for UnitedStates,
and downloads csv.gz quadkey tiles to /data/buildings (overridable via
:buildings_cache_dir). Index is cached 24h; tiles are written once and
reused. No path-analysis wiring yet — that's the next slice.
2026-04-26 10:33:25 -05:00