extract_profile_from_file shells out to cdo:
cdo -outputtab,name,lev,value -remapnn,lon=X_lat=Y <merged.grb>
parses the text output (varNNN ↔ NCEP GRIB1 param number lookup),
maps surface records to era5_profiles fields, builds the pressure
profile from each (TMP, HGT, SPFH) triple, and merges in the derived
fields from SoundingParams.derive/1. Returns the same attrs shape as
Era5BatchClient.build_profile_attrs/5 so the existing
bulk_insert_profiles path can ingest NARR rows unchanged.
fetch_profile_at picks records from a fetched inventory, byte-range
fetches each one into a per-record temp file, runs cdo -merge to build
a composite, calls extract_profile_from_file, and cleans up via
try/after.
Drive-by fix: ThetaE.dewpoint_from_spfh/2 returns Kelvin (its existing
test asserts that explicitly). Era5BatchClient was passing that
Kelvin value straight into "dwpc" (Celsius), which would then crash
SoundingParams.compute_refractivity_profile when it called Buck's
saturation-vapor-pressure equation with t=294 instead of t=21. The
bug never surfaced because era5_profiles is empty in production —
no ERA5 backfill has ever succeeded. Fixed in both era5_batch_client
and the new narr_client by subtracting 273.15 in the wrapper. Both
wrappers now have a comment pointing at the K→C conversion.
Test fixture test/fixtures/narr/narr_dfw_2010-06-15_12z.grb is the
spike-captured composite (1.1 MB — Lambert conformal projection
isn't sellonlatbox-subsettable so we can't shrink it further).
Replaces the broken ERA5/CDS path with NCEP NARR fetched anonymously
from NCEI. NARR is GRIB1, anonymous HTTPS, byte-range-fetchable, with
1979-01-01 → 2014-10-02 coverage that lines up perfectly against the
HRRR archive start. Variables match the existing era5_profiles schema
1:1 (DPT is a direct surface dewpoint record, no SPFH derivation
needed at the surface).
Phase 1 spike done by hand: byte-range fetched 5 surface records and
3 pressure-level records for DFW 2010-06-15 12Z. wgrib2 turned out to
NOT read GRIB1 messages (rd_grib2_seq_msg, grib1 message ignored, use
wgrib), so the plan now uses cdo for both GRIB1 decoding and Lambert
conformal nearest-neighbor remap. cdo -merge is the right operator
for combining records of different variables — cdo -cat is for time
concat and silently drops mismatched levels, shell cat produces an
invalid composite. cdo emits param numbers as varNNN; mapping table
is in the plan's Architecture section.
Spike values for DFW 2010-06-15 12Z (all physically sane):
TMP_2m=296.94 K (23.8°C) DPT_2m=294.49 K (21.3°C)
HPBL=703 m PRES_sfc=99383 Pa
PWAT=45.1 mm TMP_850mb=291.52 K
HGT_850mb=1547.8 m SPFH_850mb=10.4 g/kg
Plan file kept at the original 2026-04-15-merra2-historical-backfill
filename for git history continuity even though the implementation
is NARR not MERRA-2 (history block at the top documents the pivot).
