Add RRFS data assessment — not yet suitable, continue with HRRR

docs/RRFS.md

@@ -0,0 +1,84 @@
+# RRFS (Rapid Refresh Forecast System) Data Assessment
+
+## Overview
+
+RRFS is NOAA's next-generation convection-allowing ensemble forecast system, intended to replace HRRR, RAP, NAM-NEST, and HREF. As of March 2026, RRFS is in operational evaluation — the ensemble post-processing products are publicly available on AWS S3, but the full deterministic model output with pressure-level fields is not yet publicly distributed.
+
+## Data Availability
+
+### AWS S3 Bucket: `noaa-rrfs-pds`
+
+Two public prefixes with identical content:
+- `rrfs_a/refs.YYYYMMDD/HH/enspost/` (primary)
+- `rrfs_public/refs.YYYYMMDD/HH/enspost/` (mirror)
+
+### Run Schedule
+- 4x daily: 00Z, 06Z, 12Z, 18Z
+- Data available ~2 hours after init time
+- Recent data goes back about 10 days before being pruned
+
+### Available Products (CONUS)
+
+| Product | File Pattern | Contents | Usefulness for Propagation |
+|---------|-------------|----------|---------------------------|
+| `avrg` | `refs.tHHz.avrg.fNN.conus.grib2` | Ensemble mean | Only APCP (precipitation accumulation). **Not useful** — missing temp, dewpoint, wind, pressure, refractivity profile |
+| `eas` | `refs.tHHz.eas.fNN.conus.grib2` | Exceedance probability | APCP/ASNOW exceedance probabilities. **Not useful** for propagation scoring |
+| `ffri` | `refs.tHHz.ffri.fNN.conus.grib2` | Flash flood risk index | PPFFG exceedance. **Not useful** |
+| `lpmm` | `refs.tHHz.lpmm.fNN.conus.grib2` | Ensemble mean (alternate) | APCP only. Same limitation as `avrg` |
+
+### Forecast Hours
+- f01 through f07 (1-7 hour forecasts)
+- ~1.5MB per file for ensemble mean CONUS
+
+### Grid
+- 3km CONUS domain (same as HRRR)
+- Lambert Conformal projection
+- GRIB2 format with standard .idx files
+
+## Key Finding: RRFS Public Data is NOT Sufficient for Propagation Scoring
+
+The publicly available RRFS data on S3 contains **only ensemble post-processing products** — primarily precipitation statistics. It does NOT include:
+
+- Surface temperature (TMP:2m)
+- Dewpoint temperature (DPT:2m)
+- Surface pressure (PRES:surface)
+- Boundary layer height (HPBL)
+- Precipitable water (PWAT)
+- Wind speed/direction (UGRD/VGRD:10m)
+- Cloud cover (TCDC)
+- Pressure-level profiles (TMP/DPT/HGT at 1000-700mb) for refractivity gradient computation
+
+These fields are essential for propagation scoring (all 9 factors). HRRR provides all of them.
+
+## RRFS vs HRRR Comparison
+
+| Feature | HRRR | RRFS (Public) |
+|---------|------|---------------|
+| Resolution | 3km | 3km |
+| Update frequency | Hourly | 4x daily (00/06/12/18Z) |
+| Forecast range | 18h (48h from 00/06/12/18Z) | 7h (from public data) |
+| Surface fields | Full (TMP, DPT, PRES, wind, cloud, precip) | APCP only |
+| Pressure levels | 8 standard levels (1000-700mb) | Not available publicly |
+| Refractivity profile | Derivable from pressure-level T/Td/P | Not possible |
+| S3 bucket | `noaa-hrrr-bdp-pds` | `noaa-rrfs-pds` |
+| Byte-range requests | Yes (.idx files) | Yes (.idx files) |
+| Historical depth | ~2 days on S3 | ~10 days on S3 |
+
+## Recommendation
+
+**Continue using HRRR as the primary data source.** RRFS public data is not yet suitable for propagation scoring due to missing atmospheric profile and surface fields.
+
+Monitor for:
+1. NOAA making full RRFS deterministic output public (likely when RRFS becomes the operational replacement for HRRR)
+2. Additional fields being added to the ensemble post-processing products
+3. NOMADS distribution of full RRFS output (currently returns 403)
+
+When RRFS full output becomes available, key advantages would be:
+- Ensemble spread information (confidence intervals on propagation scores)
+- Potentially improved boundary layer and moisture forecasts from the FV3 dynamic core
+- Longer forecast horizon for propagation prediction
+
+## References
+- AWS S3: `s3://noaa-rrfs-pds/`
+- NOAA RRFS: https://gsl.noaa.gov/focus-areas/unified_forecast_system/rrfs
+- NOAA EMC RRFS: https://github.com/NOAA-EMC/rrfs-workflow