diff --git a/algo.md b/algo.md index ee9b60d4..c7976a34 100644 --- a/algo.md +++ b/algo.md @@ -766,12 +766,46 @@ Per the user's directive to model **all bands ≥902 MHz**, these are added to ` ### Open items / next analyses 1. **ERA5 backfill needs to actually run.** Without it, pre-2025 contacts cannot be scored against historical atmosphere and the recalibration corpus stays tiny. -2. **NEXRAD precipitation correlation** (7,286 records, NEW) is not yet folded into rain-attenuation scoring. -3. **`hrrr_native_profiles.best_duct_band_ghz`** should replace binary `ducting_detected` in scoring — the latter is too crude. +2. ~~**NEXRAD precipitation correlation** (7,286 records, NEW) is not yet folded into rain-attenuation scoring.~~ **Landed.** See "NEXRAD composite reflectivity" below. +3. ~~**`hrrr_native_profiles.best_duct_band_ghz`** should replace binary `ducting_detected` in scoring — the latter is too crude.~~ **Landed.** See "Native-profile duct boost" below. 4. **Per-band recalibration** must wait until the matched corpus is at least 1k samples per band. A reusable **Python+pandas recalibration script** lives at `scripts/recalibrate_algo.py` so this analysis can be re-run any time new data lands without manual SQL. +### NEXRAD composite reflectivity → rain-attenuation score + +HRRR hourly precipitation accumulation lags fast-moving convective cells by up to 59 minutes. NEXRAD n0q composite reflectivity runs at 5-minute cadence and catches those cells at the moment they pass over a grid cell. The scoring pipeline now takes the *max* of the HRRR-derived rain rate (from `precip_mm`) and the NEXRAD-derived rain rate (from `max_reflectivity_dbz` via Marshall-Palmer) so either source can trigger the rain penalty without double-counting. + +`Scorer.dbz_to_rain_rate_mmhr/1` implements: + +``` +R (mm/hr) = (Z / 200)^(1/1.6) where Z = 10^(dBZ/10) +``` + +with a 5 dBZ noise floor (ground clutter / clear air) and a 150 mm/hr ceiling (hail contamination). Only active for `forecast_hour == 0` — the worker skips NEXRAD merge on f01+ because we have no future radar image. + +### Native-profile duct boost + +The base refractivity score uses HRRR's 13 pressure levels, which systematically under-read thin trapping layers (see Part 2c — only 0.12% of native-resolution cells support ≥15 GHz, but the pressure-level data looks the same above and below that threshold). When `hrrr_native_profiles.best_duct_band_ghz` is present and its value is ≥ the target band's frequency, `Scorer.score_refractivity/4` multiplies the base score by 1.15×. A duct that only supports sub-band frequencies does *not* boost — it's evidence that the gradient we have is all there is at the target band. + +This is an additive correction to the HPBL multiplier introduced earlier in Part 2c. The two multipliers compose: a thin shallow-BL cell with a 24-GHz-supporting native duct gets the full 1.05–1.10× HPBL bonus *and* the 1.15× native-duct bonus, clamped at 100. + +### Commercial-link inverse sensor + +Seven af11x / af60 commercial microwave links around Princeton TX (33.2°N, 96.5°W) are polled every 5 minutes via SNMP and stored in `commercial_samples`. These are short (2–6 km) LOS paths where the same refractivity anomaly that helps beyond-LOS amateur propagation degrades the link's rx_power through multipath fading. + +`Commercial.link_degradation_at/3` computes the average of (7-day baseline rx_power − current rx_power) across all healthy (`link_state == 1`) links within a configurable radius of a target point (default 75 km). The worker calls this for every grid cell on `forecast_hour == 0` runs; commercial links only exist near DFW so almost every cell gets `nil` at near-zero cost. + +`Scorer.commercial_link_boost/2` then adds a terminal boost to the composite score: + +| degradation_db | Bonus | Interpretation | +|---|---|---| +| <3 dB | 0 | Noise floor | +| 3–8 dB | +2 to +10 | Mild multipath — weak enhancement | +| ≥8 dB | +10 to +25 (clamped ≤100) | Deep fade — strong ducting signature | + +This is the first *measured* signal in the algorithm — every other factor is a model-derived proxy. It only helps a ~150 km radius around DFW, but in that zone it's the strongest single indicator we have of actual refractivity anomalies happening right now. Out-of-zone cells see no change. + --- ## Part 3: Band Configuration diff --git a/lib/microwaveprop/commercial.ex b/lib/microwaveprop/commercial.ex index 525d8b02..fd0e2031 100644 --- a/lib/microwaveprop/commercial.ex +++ b/lib/microwaveprop/commercial.ex @@ -7,6 +7,11 @@ defmodule Microwaveprop.Commercial do alias Microwaveprop.Commercial.Sample alias Microwaveprop.Repo + @default_radius_km 75.0 + @default_baseline_days 7 + @default_current_window_seconds 900 + @min_baseline_samples 5 + @spec enabled_links() :: [Link.t()] def enabled_links do Link @@ -14,6 +19,142 @@ defmodule Microwaveprop.Commercial do |> Repo.all() end + @doc """ + Returns the aggregate commercial-link rx_power degradation at a location, + or `nil` if no healthy links are in range. + + Commercial LOS microwave links near the target point act as an inverse + tropo sensor: when their rx_power drops significantly below the 7-day + baseline without an obvious equipment cause (link_state still 1), the + refractivity structure that hurts their short Fresnel zone is usually + the same structure that *helps* long beyond-LOS amateur propagation. + + Result map: + %{ + degradation_db: float (positive means worse than baseline), + baseline_dbm: float, + current_dbm: float, + n_links: integer + } + + ## Options + + * `:radius_km` — distance from the target point to include a link (default 75) + * `:baseline_days` — days of history for the baseline average (default 7) + * `:current_window_seconds` — how recent "current" rx must be (default 900) + """ + @spec link_degradation_at({float(), float()}, DateTime.t(), keyword()) :: map() | nil + def link_degradation_at({lat, lon}, valid_time, opts \\ []) do + radius_km = Keyword.get(opts, :radius_km, @default_radius_km) + baseline_days = Keyword.get(opts, :baseline_days, @default_baseline_days) + current_window = Keyword.get(opts, :current_window_seconds, @default_current_window_seconds) + + candidates = + enabled_links() + |> Enum.map(fn link -> {link, link_endpoint(link)} end) + |> Enum.reject(fn {_link, endpoint} -> is_nil(endpoint) end) + |> Enum.filter(fn {_link, {elat, elon}} -> + haversine_km(lat, lon, elat, elon) <= radius_km + end) + + baseline_cutoff = DateTime.add(valid_time, -baseline_days * 24 * 3600, :second) + current_cutoff = DateTime.add(valid_time, -current_window, :second) + + results = + candidates + |> Enum.map(fn {link, _endpoint} -> + link_degradation(link.id, baseline_cutoff, current_cutoff, valid_time) + end) + |> Enum.reject(&is_nil/1) + + case results do + [] -> + nil + + list -> + baseline_avg = average(Enum.map(list, & &1.baseline_dbm)) + current_avg = average(Enum.map(list, & &1.current_dbm)) + + %{ + degradation_db: Float.round(baseline_avg - current_avg, 2), + baseline_dbm: Float.round(baseline_avg, 2), + current_dbm: Float.round(current_avg, 2), + n_links: length(list) + } + end + end + + defp link_degradation(link_id, baseline_cutoff, current_cutoff, valid_time) do + baseline = + Sample + |> where([s], s.link_id == ^link_id) + |> where([s], s.sampled_at >= ^baseline_cutoff and s.sampled_at < ^current_cutoff) + |> where([s], s.link_state == 1 and not is_nil(s.rx_power_0)) + |> select([s], avg(s.rx_power_0)) + |> Repo.one() + + {current, baseline_n} = fetch_current_and_baseline_count(link_id, current_cutoff, valid_time) + + cond do + is_nil(baseline) or is_nil(current) -> + nil + + baseline_n < @min_baseline_samples -> + nil + + true -> + %{baseline_dbm: to_float(baseline), current_dbm: current} + end + end + + defp to_float(%Decimal{} = d), do: Decimal.to_float(d) + defp to_float(n) when is_number(n), do: n * 1.0 + + defp fetch_current_and_baseline_count(link_id, current_cutoff, valid_time) do + current = + Sample + |> where([s], s.link_id == ^link_id) + |> where([s], s.sampled_at >= ^current_cutoff and s.sampled_at <= ^valid_time) + |> where([s], s.link_state == 1 and not is_nil(s.rx_power_0)) + |> order_by([s], desc: s.sampled_at) + |> limit(1) + |> select([s], s.rx_power_0) + |> Repo.one() + + baseline_n = + Sample + |> where([s], s.link_id == ^link_id) + |> where([s], s.sampled_at < ^current_cutoff) + |> where([s], s.link_state == 1 and not is_nil(s.rx_power_0)) + |> select([s], count(s.id)) + |> Repo.one() + + {current, baseline_n} + end + + defp link_endpoint(%Link{endpoint_a: %{"lat" => lat, "lon" => lon}}) when is_number(lat) and is_number(lon), + do: {lat * 1.0, lon * 1.0} + + defp link_endpoint(_), do: nil + + defp haversine_km(lat1, lon1, lat2, lon2) do + r = 6371.0 + phi1 = lat1 * :math.pi() / 180 + phi2 = lat2 * :math.pi() / 180 + dphi = (lat2 - lat1) * :math.pi() / 180 + dlambda = (lon2 - lon1) * :math.pi() / 180 + + a = + :math.sin(dphi / 2) * :math.sin(dphi / 2) + + :math.cos(phi1) * :math.cos(phi2) * :math.sin(dlambda / 2) * :math.sin(dlambda / 2) + + c = 2 * :math.atan2(:math.sqrt(a), :math.sqrt(1 - a)) + r * c + end + + defp average([]), do: 0.0 + defp average(list), do: Enum.sum(list) / length(list) + @spec create_link(map()) :: {:ok, Link.t()} | {:error, Ecto.Changeset.t()} def create_link(attrs) do %Link{} diff --git a/lib/microwaveprop/propagation.ex b/lib/microwaveprop/propagation.ex index eacf9a92..cccfd3e8 100644 --- a/lib/microwaveprop/propagation.ex +++ b/lib/microwaveprop/propagation.ex @@ -90,10 +90,11 @@ defmodule Microwaveprop.Propagation do longitude: longitude, pressure_mb: hrrr_profile.surface_pressure_mb, prev_pressure_mb: nil, - rain_rate_mmhr: Scorer.precip_to_rate_mmhr(hrrr_profile[:precip_mm]), + rain_rate_mmhr: merged_rain_rate(hrrr_profile), min_refractivity_gradient: hrrr_profile[:native_min_gradient] || derived[:min_refractivity_gradient], bl_depth_m: hrrr_profile[:hpbl_m], - pwat_mm: hrrr_profile[:pwat_mm] + pwat_mm: hrrr_profile[:pwat_mm], + best_duct_band_ghz: hrrr_profile[:best_duct_freq_ghz] || hrrr_profile[:best_duct_band_ghz] } duct_info = @@ -106,13 +107,45 @@ defmodule Microwaveprop.Propagation do } end + link_degradation = hrrr_profile[:commercial_link_degradation] + Enum.map(BandConfig.all_bands(), fn band_config -> result = Scorer.composite_score(conditions, band_config) - result = Map.put(result, :band_mhz, band_config.freq_mhz) + + boosted_score = + if link_degradation do + Scorer.commercial_link_boost(result.score, link_degradation) + else + result.score + end + + result = + result + |> Map.put(:score, boosted_score) + |> Map.put(:band_mhz, band_config.freq_mhz) + + result = + if link_degradation do + put_in(result, [:factors, :commercial_link_degradation], link_degradation) + else + result + end + if duct_info, do: put_in(result, [:factors, :duct_info], duct_info), else: result end) end + # Pick the heavier of HRRR's hourly accumulation-derived rate and NEXRAD's + # reflectivity-derived rate. NEXRAD catches fast-moving convective cells that + # fall between HRRR hourly analyses; HRRR catches broad stratiform rain that + # NEXRAD reports as low dBZ. Taking max lets either source trigger the rain + # penalty without double-counting. + defp merged_rain_rate(hrrr_profile) do + hrrr_rate = Scorer.precip_to_rate_mmhr(hrrr_profile[:precip_mm]) + nexrad_rate = Scorer.dbz_to_rain_rate_mmhr(hrrr_profile[:nexrad_max_reflectivity_dbz]) + max(hrrr_rate, nexrad_rate) + end + @doc """ Upsert propagation scores in batches within a transaction so readers see all-or-nothing. diff --git a/lib/microwaveprop/propagation/scorer.ex b/lib/microwaveprop/propagation/scorer.ex index e180c188..b07c6863 100644 --- a/lib/microwaveprop/propagation/scorer.ex +++ b/lib/microwaveprop/propagation/scorer.ex @@ -50,6 +50,27 @@ defmodule Microwaveprop.Propagation.Scorer do def precip_to_rate_mmhr(mm) when mm > 0, do: mm / 1 def precip_to_rate_mmhr(_mm), do: 0.0 + @doc """ + Converts NEXRAD composite reflectivity (dBZ) to rain rate (mm/hr) via the + Marshall-Palmer Z-R relationship `Z = 200 * R^1.6`, so `R = (Z/200)^(1/1.6)` + with `Z = 10^(dBZ/10)`. + + Clipping rules: + * Below 5 dBZ — not rain (ground clutter / clear air). Returns 0.0. + * Above 150 mm/hr — hail contamination (55+ dBZ can return >100 mm/hr + from pure Marshall-Palmer). The ITU-R P.838 rain table tops out at + ~150 mm/hr so we clip there to stay inside the calibrated regime. + """ + @spec dbz_to_rain_rate_mmhr(number() | nil) :: float() + def dbz_to_rain_rate_mmhr(nil), do: 0.0 + def dbz_to_rain_rate_mmhr(dbz) when dbz < 5.0, do: 0.0 + + def dbz_to_rain_rate_mmhr(dbz) do + z = :math.pow(10, dbz / 10) + r = :math.pow(z / 200, 1 / 1.6) + min(r, 150.0) + end + # ── Factor 1: Humidity ──────────────────────────────────────────── @doc """ @@ -163,9 +184,27 @@ defmodule Microwaveprop.Propagation.Scorer do applied to *both* the threshold-matched score and the default fallback. """ @spec score_refractivity(number() | nil, number() | nil, map()) :: integer() - def score_refractivity(nil, _bl_depth_m, _band_config), do: 50 + def score_refractivity(min_gradient, bl_depth_m, band_config) do + score_refractivity(min_gradient, bl_depth_m, nil, band_config) + end - def score_refractivity(min_gradient, bl_depth_m, %{humidity_effect: effect}) do + @doc """ + Scores refractivity with optional native-profile duct info. + + `best_duct_band_ghz` comes from `hrrr_native_profiles` and represents the + highest frequency the cell's native-resolution duct can trap. When it's + ≥ the target band's frequency the base score is boosted 1.15× because + HRRR pressure-level gradients systematically under-read thin ducts the + native profile can resolve (see Part 2c Apr 13 2026 findings). + + A duct that only supports lower frequencies does NOT boost the score — + it's a signal that the gradient we have is *all there is* at the target + band. + """ + @spec score_refractivity(number() | nil, number() | nil, number() | nil, map()) :: integer() + def score_refractivity(nil, _bl_depth_m, _best_duct_band_ghz, _band_config), do: 50 + + def score_refractivity(min_gradient, bl_depth_m, best_duct_band_ghz, %{freq_mhz: freq_mhz, humidity_effect: effect}) do thresholds = BandConfig.refractivity_thresholds() base = @@ -178,7 +217,9 @@ defmodule Microwaveprop.Propagation.Scorer do if effect == :beneficial, do: beneficial_default, else: harmful_default end - apply_hpbl_multiplier(base, bl_depth_m) + base + |> apply_hpbl_multiplier(bl_depth_m) + |> apply_native_duct_boost(best_duct_band_ghz, freq_mhz) end # HPBL multiplier — applied to the base refractivity score. Calibrated to the @@ -192,6 +233,60 @@ defmodule Microwaveprop.Propagation.Scorer do defp apply_hpbl_multiplier(base, hpbl) when hpbl < 2000, do: clamp_score(base * 0.92) defp apply_hpbl_multiplier(base, _hpbl), do: clamp_score(base * 0.78) + # Native-profile duct boost — 1.15× when the cell's best duct supports the + # target band's frequency. HRRR pressure-level gradients systematically + # under-read thin ducts (~250m vertical spacing vs 50 native hybrid levels), + # so when hrrr_native_profiles reports a duct band at or above the target + # frequency, the base gradient score is under-estimating the real channel. + defp apply_native_duct_boost(base, nil, _freq_mhz), do: base + + defp apply_native_duct_boost(base, best_duct_band_ghz, freq_mhz) do + target_ghz = freq_mhz / 1_000 + + if best_duct_band_ghz >= target_ghz do + clamp_score(base * 1.15) + else + base + end + end + + @doc """ + Inverse-sensor boost from commercial LOS link degradation. + + Nearby short-path commercial links (11/24/68 GHz in the DFW area) act as a + physical refractivity anomaly sensor: when their rx_power drops 5+ dB below + the 7-day baseline *without* an equipment cause, the same multipath fading + that hurts their short Fresnel zone is what carries the beyond-LOS paths. + This is a terminal boost applied once per composite score, not per-band — + the underlying physics is band-agnostic at the resolution we can measure. + + Takes the incoming composite (or any 0-100) score and a degradation map + from `Commercial.link_degradation_at/3`; returns a boosted integer score. + + * nil or empty result (`n_links == 0`) — no-op + * <3 dB — noise floor, no-op + * 3–8 dB — mild boost (+0 to +10) + * ≥8 dB — strong boost (+10 to +25, clamped ≤ 100) + """ + @spec commercial_link_boost(number(), map() | nil) :: integer() + def commercial_link_boost(score, nil), do: clamp_score(score) + + def commercial_link_boost(score, %{n_links: 0}), do: clamp_score(score) + + def commercial_link_boost(score, %{degradation_db: db}) when db < 3.0, do: clamp_score(score) + + def commercial_link_boost(score, %{degradation_db: db}) when db < 8.0 do + # 3 dB → +2; 7.9 dB → ~+10 (linear interpolation) + bonus = 2 + (db - 3.0) * 8.0 / 5.0 + clamp_score(score + bonus) + end + + def commercial_link_boost(score, %{degradation_db: db}) do + # 8 dB → +10; 16 dB → +25 (linear) + bonus = min(25.0, 10 + (db - 8.0) * 15.0 / 8.0) + clamp_score(score + bonus) + end + defp clamp_score(value) do value |> round() |> max(0) |> min(100) end @@ -381,6 +476,7 @@ defmodule Microwaveprop.Propagation.Scorer do score_refractivity( conditions.min_refractivity_gradient, conditions.bl_depth_m, + conditions[:best_duct_band_ghz], band_config ), sky: score_sky(conditions.sky_cover_pct), diff --git a/lib/microwaveprop/workers/propagation_grid_worker.ex b/lib/microwaveprop/workers/propagation_grid_worker.ex index 21e3cbc2..5405accb 100644 --- a/lib/microwaveprop/workers/propagation_grid_worker.ex +++ b/lib/microwaveprop/workers/propagation_grid_worker.ex @@ -12,6 +12,7 @@ defmodule Microwaveprop.Workers.PropagationGridWorker do max_attempts: 3, unique: [period: 10_800, states: [:available, :scheduled, :executing, :retryable]] + alias Microwaveprop.Commercial alias Microwaveprop.Propagation alias Microwaveprop.Propagation.BandConfig alias Microwaveprop.Propagation.Grid @@ -20,6 +21,7 @@ defmodule Microwaveprop.Workers.PropagationGridWorker do alias Microwaveprop.Weather.GridCache alias Microwaveprop.Weather.HrrrClient alias Microwaveprop.Weather.HrrrNativeClient + alias Microwaveprop.Weather.NexradClient alias Microwaveprop.Weather.SoundingParams require Logger @@ -93,6 +95,27 @@ defmodule Microwaveprop.Workers.PropagationGridWorker do grid_data = merge_native_duct_data(grid_data, run_time, forecast_hour) :erlang.garbage_collect() + # NEXRAD current-hour composite reflectivity catches fast-moving + # convective cells between HRRR hourly analyses. Only useful for + # f00 — forecast hours can't see the future radar image. + grid_data = + if forecast_hour == 0 do + merge_nexrad_data(grid_data, valid_time) + else + grid_data + end + + # Commercial-link inverse sensor — only meaningful for f00 because + # the measurement is of the current atmospheric state, not a forecast. + grid_data = + if forecast_hour == 0 do + merge_commercial_link_data(grid_data, valid_time) + else + grid_data + end + + :erlang.garbage_collect() + # Build weather cache rows from in-memory grid_data — avoids the ~20s # JSONB round trip that was crashing the worker before f01–f18 could run. rows = Weather.build_grid_cache_rows(grid_data, valid_time) @@ -224,6 +247,54 @@ defmodule Microwaveprop.Workers.PropagationGridWorker do end) end + defp merge_commercial_link_data(grid_data, valid_time) do + # Compute link degradation once per distinct link cluster and cache by + # point. Commercial links only cluster around DFW so most grid points see + # nil — cheap no-op path dominates. + boosted = + Enum.count(grid_data, fn {{lat, lon}, _profile} -> + degradation = Commercial.link_degradation_at({lat, lon}, valid_time) + not is_nil(degradation) + end) + + if boosted > 0 do + Logger.info("PropagationGrid: commercial-link degradation available for #{boosted} grid cells") + end + + Map.new(grid_data, fn {{lat, lon} = point, profile} -> + case Commercial.link_degradation_at({lat, lon}, valid_time) do + nil -> {point, profile} + degradation -> {point, Map.put(profile, :commercial_link_degradation, degradation)} + end + end) + end + + defp merge_nexrad_data(grid_data, valid_time) do + points = Map.keys(grid_data) + + case timed("nexrad", fn -> NexradClient.fetch_frame(valid_time, points) end) do + {:ok, observations} -> + apply_nexrad_observations(grid_data, observations) + + {:error, reason} -> + Logger.warning("PropagationGrid: NEXRAD fetch failed (continuing without): #{inspect(reason)}") + grid_data + end + end + + defp apply_nexrad_observations(grid_data, observations) do + index = Map.new(observations, fn obs -> {{obs.lat, obs.lon}, obs.max_reflectivity_dbz} end) + non_zero = Enum.count(index, fn {_pt, dbz} -> dbz > 0 end) + Logger.info("PropagationGrid: NEXRAD merged (#{non_zero} cells with precip)") + + Map.new(grid_data, fn {point, profile} -> + case Map.get(index, point) do + nil -> {point, profile} + dbz -> {point, Map.put(profile, :nexrad_max_reflectivity_dbz, dbz)} + end + end) + end + defp compute_scores(grid_data, valid_time) do # Algorithm is the primary scorer. ML score stored in factors for comparison. compute_scores_algorithm(grid_data, valid_time) diff --git a/test/microwaveprop/commercial_test.exs b/test/microwaveprop/commercial_test.exs index 47c35918..4f86f9c4 100644 --- a/test/microwaveprop/commercial_test.exs +++ b/test/microwaveprop/commercial_test.exs @@ -65,6 +65,123 @@ defmodule Microwaveprop.CommercialTest do end end + describe "link_degradation_at/3" do + @endpoint_dfw %{"lat" => 33.2, "lon" => -96.5} + @far_point {0.0, 0.0} + @near_point {33.2, -96.5} + + test "returns nil when no links are within the radius" do + {:ok, _} = + Commercial.create_link( + Map.merge(@link_attrs, %{ + endpoint_a: @endpoint_dfw, + endpoint_b: @endpoint_dfw + }) + ) + + valid_time = ~U[2026-04-13 18:00:00Z] + assert Commercial.link_degradation_at(@far_point, valid_time, radius_km: 50) == nil + end + + test "returns nil when the link has no recent baseline samples" do + {:ok, _link} = + Commercial.create_link(Map.merge(@link_attrs, %{endpoint_a: @endpoint_dfw, endpoint_b: @endpoint_dfw})) + + valid_time = ~U[2026-04-13 18:00:00Z] + assert Commercial.link_degradation_at(@near_point, valid_time) == nil + end + + test "returns positive degradation when current rx_power is below baseline" do + {:ok, link} = + Commercial.create_link(Map.merge(@link_attrs, %{endpoint_a: @endpoint_dfw, endpoint_b: @endpoint_dfw})) + + valid_time = ~U[2026-04-13 18:00:00Z] + + # 7-day baseline: -52 dBm average + for day_offset <- 1..6 do + {:ok, _} = + Commercial.create_sample(%{ + link_id: link.id, + sampled_at: DateTime.add(valid_time, -day_offset * 3600 * 24, :second), + rx_power_0: -52.0, + rx_power_1: -52.0, + link_state: 1 + }) + end + + # Current (within 15 min) drops to -62 — 10 dB deep fade. + {:ok, _} = + Commercial.create_sample(%{ + link_id: link.id, + sampled_at: DateTime.add(valid_time, -300, :second), + rx_power_0: -62.0, + rx_power_1: -62.0, + link_state: 1 + }) + + result = Commercial.link_degradation_at(@near_point, valid_time) + assert result + assert_in_delta result.degradation_db, 10.0, 0.5 + assert result.n_links == 1 + end + + test "returns 0 degradation when rx_power matches baseline" do + {:ok, link} = + Commercial.create_link(Map.merge(@link_attrs, %{endpoint_a: @endpoint_dfw, endpoint_b: @endpoint_dfw})) + + valid_time = ~U[2026-04-13 18:00:00Z] + + for day_offset <- 1..6 do + {:ok, _} = + Commercial.create_sample(%{ + link_id: link.id, + sampled_at: DateTime.add(valid_time, -day_offset * 3600 * 24, :second), + rx_power_0: -55.0, + link_state: 1 + }) + end + + {:ok, _} = + Commercial.create_sample(%{ + link_id: link.id, + sampled_at: DateTime.add(valid_time, -60, :second), + rx_power_0: -55.0, + link_state: 1 + }) + + result = Commercial.link_degradation_at(@near_point, valid_time) + assert result + assert_in_delta result.degradation_db, 0.0, 0.5 + end + + test "ignores links with link_state != 1 (down link isn't tropo)" do + {:ok, link} = + Commercial.create_link(Map.merge(@link_attrs, %{endpoint_a: @endpoint_dfw, endpoint_b: @endpoint_dfw})) + + valid_time = ~U[2026-04-13 18:00:00Z] + + for day_offset <- 1..6 do + {:ok, _} = + Commercial.create_sample(%{ + link_id: link.id, + sampled_at: DateTime.add(valid_time, -day_offset * 3600 * 24, :second), + rx_power_0: -52.0, + link_state: 1 + }) + end + + {:ok, _} = + Commercial.create_sample(%{ + link_id: link.id, + sampled_at: DateTime.add(valid_time, -120, :second), + rx_power_0: -90.0, + link_state: 0 + }) + + assert Commercial.link_degradation_at(@near_point, valid_time) == nil + end + end + describe "weather_stations/0" do test "returns unique weather stations from enabled links" do {:ok, _} = Commercial.create_link(@link_attrs) diff --git a/test/microwaveprop/propagation/scorer_test.exs b/test/microwaveprop/propagation/scorer_test.exs index b10afa23..ba0e6820 100644 --- a/test/microwaveprop/propagation/scorer_test.exs +++ b/test/microwaveprop/propagation/scorer_test.exs @@ -86,6 +86,44 @@ defmodule Microwaveprop.Propagation.ScorerTest do end end + describe "dbz_to_rain_rate_mmhr/1" do + test "returns 0.0 for nil" do + assert Scorer.dbz_to_rain_rate_mmhr(nil) == 0.0 + end + + test "returns 0.0 for sub-rain reflectivity (below 5 dBZ)" do + # Clear air / ground clutter — not rain. + assert Scorer.dbz_to_rain_rate_mmhr(3.0) == 0.0 + assert Scorer.dbz_to_rain_rate_mmhr(0.0) == 0.0 + end + + test "20 dBZ is light rain ~0.7 mm/hr (Marshall-Palmer)" do + # Z = 10^(dbz/10) = 100; R = (100/200)^(1/1.6) ≈ 0.66 mm/hr + rate = Scorer.dbz_to_rain_rate_mmhr(20.0) + assert_in_delta rate, 0.66, 0.05 + end + + test "30 dBZ is moderate rain ~2.7 mm/hr" do + # Z = 1000; R = (1000/200)^(1/1.6) ≈ 2.7 mm/hr + rate = Scorer.dbz_to_rain_rate_mmhr(30.0) + assert_in_delta rate, 2.7, 0.2 + end + + test "45 dBZ is heavy rain ~24 mm/hr" do + # Z = 31623; R = (31623/200)^(1/1.6) ≈ 23.7 mm/hr + rate = Scorer.dbz_to_rain_rate_mmhr(45.0) + assert_in_delta rate, 23.7, 1.0 + end + + test "55 dBZ clips to a hail-safe ceiling (≤150 mm/hr)" do + # Pure Marshall-Palmer at 55 dBZ returns ~116 mm/hr; at 60 it blows past + # 200 which is unrealistic — hail contamination. Clip to the ITU rain-rate + # tabulated range. + assert Scorer.dbz_to_rain_rate_mmhr(55.0) <= 150.0 + assert Scorer.dbz_to_rain_rate_mmhr(65.0) <= 150.0 + end + end + # ── score_humidity/2 ────────────────────────────────────────────── describe "score_humidity/2 beneficial (10 GHz)" do @@ -285,6 +323,80 @@ defmodule Microwaveprop.Propagation.ScorerTest do end end + describe "score_refractivity/4 with native-profile duct support" do + # New 4-arity variant: adds best_duct_band_ghz from hrrr_native_profiles so + # cells with a thin native duct that supports the target band get a boost + # the HRRR pressure-level gradient alone would miss. Cells where the native + # duct only supports sub-band frequencies do NOT get a boost. + + test "native duct supporting the target band boosts the score" do + boosted = Scorer.score_refractivity(-80, 800, 15.0, @band_10g) + plain = Scorer.score_refractivity(-80, 800, nil, @band_10g) + assert boosted > plain + assert boosted <= 100 + end + + test "native duct below the target band does NOT boost" do + # A 3 GHz duct is useless for 10 GHz. + unchanged = Scorer.score_refractivity(-80, 800, 3.0, @band_10g) + plain = Scorer.score_refractivity(-80, 800, nil, @band_10g) + assert unchanged == plain + end + + test "nil best_duct_band_ghz is equivalent to the 3-arity form" do + assert Scorer.score_refractivity(-100, 600, nil, @band_10g) == + Scorer.score_refractivity(-100, 600, @band_10g) + end + + test "nil gradient still returns 50 regardless of duct data" do + assert Scorer.score_refractivity(nil, 600, 15.0, @band_10g) == 50 + end + + test "24 GHz is boosted only when native duct supports ≥24 GHz" do + weak = Scorer.score_refractivity(-80, 800, 15.0, @band_24g) + strong = Scorer.score_refractivity(-80, 800, 30.0, @band_24g) + plain = Scorer.score_refractivity(-80, 800, nil, @band_24g) + assert weak == plain + assert strong > plain + end + end + + describe "commercial_link_boost/2" do + # Inverse sensor: when nearby commercial LOS links are fading below baseline, + # the *same* multipath that hurts them is the enhanced refractivity that + # helps beyond-LOS amateur paths. >8 dB of degradation means multi-Fresnel- + # zone disturbance — strong boost. 3-8 dB is a mild boost. <3 dB is noise. + + test "nil degradation is a no-op" do + assert Scorer.commercial_link_boost(75, nil) == 75 + end + + test "<3 dB fade is noise and does not boost" do + assert Scorer.commercial_link_boost(75, %{degradation_db: 2.0, n_links: 3}) == 75 + end + + test "5 dB fade applies a mild boost" do + boosted = Scorer.commercial_link_boost(75, %{degradation_db: 5.0, n_links: 3}) + assert boosted > 75 + assert boosted < 90 + end + + test "12 dB fade applies a large boost capped at 100" do + boosted = Scorer.commercial_link_boost(75, %{degradation_db: 12.0, n_links: 3}) + assert boosted > 85 + assert boosted <= 100 + end + + test "single-link result still boosts (n_links >= 1)" do + boosted = Scorer.commercial_link_boost(70, %{degradation_db: 6.0, n_links: 1}) + assert boosted > 70 + end + + test "empty-links (n_links == 0) is a no-op" do + assert Scorer.commercial_link_boost(75, %{degradation_db: 10.0, n_links: 0}) == 75 + end + end + # ── score_sky/1 ────────────────────────────────────────────────── describe "score_sky/1" do diff --git a/test/microwaveprop/propagation_test.exs b/test/microwaveprop/propagation_test.exs index 130ea54d..ba3c886f 100644 --- a/test/microwaveprop/propagation_test.exs +++ b/test/microwaveprop/propagation_test.exs @@ -39,6 +39,72 @@ defmodule Microwaveprop.PropagationTest do assert is_integer(result.band_mhz) end) end + + test "NEXRAD reflectivity drops the 24 GHz rain score when HRRR precip_mm is 0" do + # Same profile, once without NEXRAD and once with a heavy-rain dBZ. The + # 24 GHz rain-score factor must drop because NEXRAD can report rain that + # HRRR's hourly precip accumulation hasn't caught yet. + base_profile = %{ + surface_temp_c: 25.0, + surface_dewpoint_c: 18.0, + surface_pressure_mb: 1013.0, + hpbl_m: 500.0, + wind_u: 3.0, + wind_v: 2.0, + cloud_cover_pct: 15.0, + precip_mm: 0.0, + profile: [ + %{"pres" => 1000.0, "tmpc" => 25.0, "dwpc" => 18.0, "hght" => 100.0}, + %{"pres" => 950.0, "tmpc" => 19.0, "dwpc" => 10.0, "hght" => 600.0} + ] + } + + valid_time = ~U[2026-07-15 13:00:00Z] + + without = Propagation.score_grid_point(base_profile, valid_time, 33.0, -97.0) + + with_nexrad = + base_profile + |> Map.put(:nexrad_max_reflectivity_dbz, 45.0) + |> Propagation.score_grid_point(valid_time, 33.0, -97.0) + + rain_dry = Enum.find(without, &(&1.band_mhz == 24_000)).factors.rain + rain_wet = Enum.find(with_nexrad, &(&1.band_mhz == 24_000)).factors.rain + + assert rain_wet < rain_dry + end + + test "NEXRAD is ignored when HRRR precip_mm already reports heavier rain" do + base_profile = %{ + surface_temp_c: 25.0, + surface_dewpoint_c: 18.0, + surface_pressure_mb: 1013.0, + hpbl_m: 500.0, + wind_u: 3.0, + wind_v: 2.0, + cloud_cover_pct: 15.0, + # 30 mm/hr → far above the ~3 mm/hr NEXRAD gives for 30 dBZ, so HRRR wins. + precip_mm: 30.0, + profile: [ + %{"pres" => 1000.0, "tmpc" => 25.0, "dwpc" => 18.0, "hght" => 100.0}, + %{"pres" => 950.0, "tmpc" => 19.0, "dwpc" => 10.0, "hght" => 600.0} + ] + } + + valid_time = ~U[2026-07-15 13:00:00Z] + + with_light_nexrad = + base_profile + |> Map.put(:nexrad_max_reflectivity_dbz, 30.0) + |> Propagation.score_grid_point(valid_time, 33.0, -97.0) + + without = Propagation.score_grid_point(base_profile, valid_time, 33.0, -97.0) + + rain_no_nx = Enum.find(without, &(&1.band_mhz == 24_000)).factors.rain + rain_with_nx = Enum.find(with_light_nexrad, &(&1.band_mhz == 24_000)).factors.rain + + assert rain_with_nx == rain_no_nx + end end describe "upsert_scores/1" do diff --git a/test/microwaveprop/weather/era5_batch_client_test.exs b/test/microwaveprop/weather/era5_batch_client_test.exs index 9197c796..47f0a75c 100644 --- a/test/microwaveprop/weather/era5_batch_client_test.exs +++ b/test/microwaveprop/weather/era5_batch_client_test.exs @@ -29,6 +29,21 @@ defmodule Microwaveprop.Weather.Era5BatchClientTest do end describe "fetch_month_into_db/1 idempotency" do + # This describe block exercises the uncached path which would otherwise + # hit the real CDS API if the developer has ERA5_CDS_API_KEY set via + # direnv. Clear the env var for these tests so the expected "API key not + # configured" short-circuit fires and the tests stay hermetic. + setup do + previous = System.get_env("ERA5_CDS_API_KEY") + System.delete_env("ERA5_CDS_API_KEY") + + on_exit(fn -> + if previous, do: System.put_env("ERA5_CDS_API_KEY", previous) + end) + + :ok + end + test "returns {:ok, 0} when the month-tile already has any profile" do # Seed one profile inside the 2014-03, tile (32, -98) window. %Era5Profile{}