prop/docs
Graham McIntire 13180d572e
NARR backfill plan + spike-captured inventory fixture
Replaces the broken ERA5/CDS path with NCEP NARR fetched anonymously
from NCEI. NARR is GRIB1, anonymous HTTPS, byte-range-fetchable, with
1979-01-01 → 2014-10-02 coverage that lines up perfectly against the
HRRR archive start. Variables match the existing era5_profiles schema
1:1 (DPT is a direct surface dewpoint record, no SPFH derivation
needed at the surface).

Phase 1 spike done by hand: byte-range fetched 5 surface records and
3 pressure-level records for DFW 2010-06-15 12Z. wgrib2 turned out to
NOT read GRIB1 messages (rd_grib2_seq_msg, grib1 message ignored, use
wgrib), so the plan now uses cdo for both GRIB1 decoding and Lambert
conformal nearest-neighbor remap. cdo -merge is the right operator
for combining records of different variables — cdo -cat is for time
concat and silently drops mismatched levels, shell cat produces an
invalid composite. cdo emits param numbers as varNNN; mapping table
is in the plan's Architecture section.

Spike values for DFW 2010-06-15 12Z (all physically sane):
  TMP_2m=296.94 K (23.8°C)   DPT_2m=294.49 K (21.3°C)
  HPBL=703 m                  PRES_sfc=99383 Pa
  PWAT=45.1 mm                TMP_850mb=291.52 K
  HGT_850mb=1547.8 m          SPFH_850mb=10.4 g/kg

Plan file kept at the original 2026-04-15-merra2-historical-backfill
filename for git history continuity even though the implementation
is NARR not MERRA-2 (history block at the top documents the pivot).
2026-04-15 17:10:46 -05:00
..
algo-reports Refresh 2026-04-13 recalibration report 2026-04-13 12:29:51 -05:00
plans NARR backfill plan + spike-captured inventory fixture 2026-04-15 17:10:46 -05:00
research Phase 3 spike (parked) + Phase 6: temperature anomaly 2026-04-10 08:47:11 -05:00
algo_commercial.md Refactor HRRR fetch to batch points per hour 2026-03-30 17:21:47 -05:00
algo_original.md Refactor HRRR fetch to batch points per hour 2026-03-30 17:21:47 -05:00
algo_updated_with_data.md Refactor HRRR fetch to batch points per hour 2026-03-30 17:21:47 -05:00
analysis_output.txt Normalize ML features to prevent NaN gradient explosion 2026-04-01 09:08:33 -05:00
analysis_output_solar.txt Normalize ML features to prevent NaN gradient explosion 2026-04-01 09:08:33 -05:00
findings10.md Correct Finding 10: Great Lakes firing squad explains 24G PH anomaly 2026-04-01 11:07:58 -05:00
goes-r-research.md HRRR forecast hours (f00-f18) with timeline map UI 2026-03-31 16:44:47 -05:00
RRFS.md Add meteorologist feedback on model choice to RRFS doc 2026-03-31 09:27:23 -05:00