- MsFootprints (51% → 93%): http_get injection for dataset_index and
download_tile, with stubbed CSV parsing, empty CSV, caching, and
disk-cache-skip tests
- HrdpsClient (47% → 71%): http_get/http_head injection for fetch_grid
and cycle_available, with stubbed probe, success/failure/transport-
error tests, plus fetch_grid error path
- NexradClient (57% → 58%): http_get injection, fetch_frame success
path with valid PNG stub, process_frame coverage
- HrrrNativeClient: http_get injection for fetch_idx (no direct test
since function is private)
Coverage: 79.43% → 79.68% (need 0.32% more)
- Replace %Struct{} with Struct.t() in all @spec annotations
- Replace length(x) > 0 with x != [] in test assertions
- Fix multi-line spec struct references in weather.ex
Registered users can suggest edits to any contact's core fields
(callsigns, grids, band, mode, timestamp). Edits enter an admin
approval queue with field-by-field diff view. On approve, changes
are applied and enrichment re-enqueued if grids/band changed.
Users receive email notification on approve or reject.
Also updates dependabot.yml for mix ecosystem.
When clicking a grid point with ducting, the panel now shows each
duct layer with base-top height in feet, thickness in meters, and
minimum trapped frequency. Data flows from Duct.analyze through
the scoring factors as a ducts array.
The PropagationGridWorker now fetches native hybrid-sigma levels
(TMP, SPFH, HGT, PRES × 50 levels) alongside the standard surface
and pressure products. Native data provides 10-50m vertical spacing
vs 250m from pressure levels, detecting thin surface ducts invisible
to the standard product.
Key design: cell-by-cell reducer in Wgrib2.extract_grid_from_file_mapped
processes each of the 95k CONUS cells through a duct analysis function
inline, keeping only scalar metrics per cell. Peak memory ~86 MB
instead of ~1.8 GB for the full grid map.
Per-cell output: native_min_gradient, best_duct_freq_ghz,
max_duct_thickness_m, duct_count. The scorer prefers the native
gradient over the pressure-level gradient when available.
Native fetch is optional — if it fails, scoring continues with
pressure-level data only.
Points spread coast-to-coast created a ~476k cell bounding grid
(350 messages × 476k cells × 4 bytes ≈ 665 MB), causing OOM.
Switch to -lon which extracts values at specific lat/lon points
with text output. One wgrib2 call, one file scan, negligible
BEAM memory regardless of point geographic spread.
Instead of holding ~530MB GRIB binary in BEAM memory, download
ranges directly to a temp file and run wgrib2 on it. Peak memory
drops from ~530MB to just HTTP chunk buffers.
wgrib2 -lola ... bin writes Fortran unformatted records (4-byte
length header + data + 4-byte length trailer per message).
parse_lola_binary was treating the binary as tightly packed,
causing every message after the first to read from the wrong
offset — values came out as garbage across all grid points.
Fix: account for the 8-byte record overhead per message when
computing the data offset for each message's grid values.
This bug affects both the existing propagation grid extraction
(which may have been producing subtly wrong scores) and the new
native-level extraction (which was producing obviously wrong
values). The fix is a one-line stride change.
Also adds Backtest.Features.native_surface_refractivity for the
Phase 1 sanity check, plus a tighter wgrib2 match pattern that
selects only hybrid-level messages from the native file.
The Elixir GRIB2 decoder didn't map level type 105 (hybrid) or
variable IDs for SPFH and TKE, so native-level messages decoded as
"unknown:105:N" keys that build_native_profile couldn't find. Add
the three missing mappings to Section.identify_level/identify_var.
Also add HrrrNativeClient.extract_native_profiles/2 which uses
wgrib2's -lola on a tight bounding-box subgrid for speed (the pure
Elixir decoder takes ~70s per point on a 395 MB file; wgrib2 handles
40 points in seconds). The worker now routes through this path.
- Spike docs at docs/research/hrrr_native_levels.md confirming files
are on AWS S3 for 5+ years, 50 hybrid levels, and include TKE and
SPFH needed for Phase 2 turbulence features. Architectural finding:
per-point on-demand fetching is impractical (~530 MB/file), so
the ingestion worker batches per (date, hour) instead.
- hrrr_native_profiles schema: arrays per level plus cached surface
scalars and placeholder columns for Phase 2/4 derived fields.
Strictly additive — the existing hrrr_profiles table is untouched.
- HrrrNativeClient: pure URL/message-list helpers, build_native_profile/1
that turns a parsed wgrib2 map into the schema shape (TDD'd).
- Exposed HrrrClient.download_grib_ranges/2 so the native client
reuses the existing parallel byte-range download + disk cache.
- HrrrNativeGridWorker: Oban worker keyed on {year, month, day, hour},
unique at :infinity, pulls distinct (lat, lon) points from contacts
in the ±30 min window, downloads the native grib2, extracts per
point, bulk-upserts.
- mix hrrr_native_backfill --limit N enqueues the top-N hours by
contact count.
Phase 1 gate still pending Task 1.6 (sanity-check backtest after
live data lands).
