graham/prop
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Fix ERA5 pressure-level request: use specific_humidity not dewpoint_temperature

Browse source 
CDS rejects `reanalysis-era5-pressure-levels` requests that include
dewpoint_temperature because it's only a single-level 2m variable; no
pressure-level dewpoint exists in ERA5. The failure mode is a 2–3 second
`{"status": "failed"}` response with no message field, which is why
prod was burning through Era5MonthBatchWorker attempts without anything
landing in era5_profiles.

Switch the pressure-level moisture variable to specific_humidity (SPFH),
update the wgrib2 extraction regex, and convert SPFH → dewpoint °C via
Weather.ThetaE.dewpoint_from_spfh/2 in the profile builder. Surface
dewpoint keeps using the single-level 2m dewpoint_temperature, which is
valid and already working.
This commit is contained in:
Graham McIntire 2026-04-13 12:17:03 -05:00
parent faf4a05793
commit caf466ba85
No known key found for this signature in database
GPG key ID: F4ABF488E6029E59
2 changed files with 18 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

16
lib/microwaveprop/weather/era5_batch_client.ex
View file

@ -24,6 +24,7 @@ defmodule Microwaveprop.Weather.Era5BatchClient do
alias Microwaveprop.Weather.Era5Profile
alias Microwaveprop.Weather.Grib2.Wgrib2
alias Microwaveprop.Weather.SoundingParams
alias Microwaveprop.Weather.ThetaE
require Logger
@ -173,7 +174,7 @@ defmodule Microwaveprop.Weather.Era5BatchClient do
with {:ok, single_msgs} <-
Wgrib2.extract_grid_messages(single_grib, ":(TMP|DPT|PRES|UGRD|VGRD|PWAT|HPBL):", grid_spec),
{:ok, pressure_msgs} <- Wgrib2.extract_grid_messages(pressure_grib, ":(TMP|DPT|HGT):", grid_spec) do
{:ok, pressure_msgs} <- Wgrib2.extract_grid_messages(pressure_grib, ":(TMP|SPFH|HGT):", grid_spec) do
# Index both message sets by (datetime, lat, lon) → var:level map.
points_single = index_messages(single_msgs)
points_pressure = index_messages(pressure_msgs)
@ -230,7 +231,11 @@ defmodule Microwaveprop.Weather.Era5BatchClient do
%{
"pres" => level * 1.0,
"tmpc" => kelvin_to_celsius(pressure_fields["TMP:#{pres_key}"]),
"dwpc" => kelvin_to_celsius(pressure_fields["DPT:#{pres_key}"]),
"dwpc" =>
dewpoint_from_spfh(
pressure_fields["SPFH:#{pres_key}"],
level * 100.0
),
"hght" => geopotential_to_height(pressure_fields["HGT:#{pres_key}"])
}
end)
@ -275,6 +280,13 @@ defmodule Microwaveprop.Weather.Era5BatchClient do
defp geopotential_to_height(nil), do: nil
defp geopotential_to_height(gp), do: gp / 9.80665
# ERA5 pressure-level moisture is specific humidity (kg/kg). Convert to
# dewpoint in °C via the Magnus-Tetens inverse already used for HRRR native
# profiles. `level_pa` is the pressure level itself in Pa.
defp dewpoint_from_spfh(nil, _level_pa), do: nil
defp dewpoint_from_spfh(spfh, _level_pa) when spfh <= 0, do: nil
defp dewpoint_from_spfh(spfh, level_pa), do: ThetaE.dewpoint_from_spfh(spfh, level_pa)
# -- Bulk insert -----------------------------------------------------------
# Insert in chunks to keep each query under Postgres' parameter limit

5
lib/microwaveprop/weather/era5_client.ex
View file

@ -33,7 +33,10 @@ defmodule Microwaveprop.Weather.Era5Client do
boundary_layer_height
)
@pressure_level_vars ~w(temperature dewpoint_temperature geopotential)
# `dewpoint_temperature` is only a single-level 2m variable in ERA5 — CDS
# rejects pressure-level requests that include it. Use specific_humidity on
# pressure levels and derive Td in the profile builder via ThetaE.
@pressure_level_vars ~w(temperature specific_humidity geopotential)
@doc false
@spec pressure_levels() :: [String.t()]