prop/lib/microwaveprop_web/live/about_live.ex

defmodule MicrowavepropWeb.AboutLive do
  @moduledoc false
  use MicrowavepropWeb, :live_view

  alias Microwaveprop.Repo

  @impl true
  def mount(_params, _session, socket) do
    {:ok,
     socket
     |> assign(:page_title, "About")
     |> assign(:stats, fetch_stats())}
  end

  @impl true
  def render(assigns) do
    ~H"""
    <Layouts.app flash={@flash} current_scope={@current_scope} max_width="max-w-4xl">
      <.header>
        About NTMS Propagation Prediction
        <:subtitle>What we're building and why.</:subtitle>
      </.header>

      <section class="space-y-8 text-sm leading-relaxed">
        <div class="space-y-3">
          <p>
            Line-of-sight is wrong for the microwave bands. Every operator on
            902 MHz and up has made contacts over hills, through trees, and 200 km
            past the horizon — contacts that shouldn't have worked if LOS were the
            whole story. What's actually going on is the lower atmosphere:
            temperature inversions, humidity gradients, ducting layers, and rain
            cells that bend, trap, and absorb signals in ways that change by the
            hour. The window for a good contact is often minutes, not days.
          </p>
          <p>
            Jim KM5PO started this project to try to predict those windows from
            weather data. Graham W5ISP picked it up and is iterating on it from
            there. The goal: a map that tells you when to get on the air, per
            band, based on real atmospheric conditions and what we've learned
            from the contacts you and everyone else have already made.
          </p>
          <div role="alert" class="alert alert-warning">
            <p>
              <strong>Heads up:</strong>
              the scoring you see is hand-calibrated from the data we have so far.
              It isn't updated in real time — I (Graham) re-fit the weights
              manually when there's enough new data to move the needle. The more
              contacts and beacon reports we collect, the better it gets. See the
              <.link navigate={~p"/algo"} class="link link-hover font-semibold">algorithm page</.link>
              for what's actually under the hood, and please <.link
                navigate={~p"/submit"}
                class="link link-hover font-semibold"
              >submit your contacts</.link>.
            </p>
          </div>
          <div role="alert" class="alert alert-info">
            <p>
              <strong>Coverage:</strong>
              right now this only covers the continental US. Every upstream source
              we're using — NOAA HRRR, ASOS, IEMRE, the ITU terrain data — is
              CONUS-only. Hawaii, Alaska, Canada, and Mexico will need different
              data sources and aren't wired up yet.
            </p>
          </div>
        </div>

        <div class="space-y-3">
          <h2 class="text-xl font-bold">How it works, roughly</h2>
          <p>
            Two things, paired up. Most propagation tools keep them separate.
          </p>
          <ol class="list-decimal list-outside pl-5 space-y-2">
            <li>
              <strong>The weather.</strong>
              Hourly 3 km HRRR forecasts give surface fields and pressure-level
              profiles. We derive refractivity profiles, minimum refractivity
              gradient, ducting detection, boundary-layer depth, and precipitable
              water for every grid cell. ASOS surface obs, 12-hourly upper-air
              soundings, and gridded IEMRE reanalysis fill in the gaps. A 9-factor
              composite score is written out for every 0.125° cell across CONUS,
              for each hour of an 18-hour forecast.
            </li>
            <li>
              <strong>The contacts.</strong>
              58k+ amateur microwave QSOs, each tagged with the atmosphere at
              both ends of the path (and along it) at the time the contact
              happened. That's a ground-truth dataset we can actually fit against
              — "when you had this score, how far did the contact go, and did it
              happen at all?"
            </li>
          </ol>
          <p>
            The scoring is a weighted sum of ten factors: rain, humidity,
            precipitable water (PWAT), season, refractivity gradient, pressure,
            T–Td depression, sky cover, wind, and time of day. Weights were
            calibrated via gradient descent against 5,000 QSOs. The physics
            changes by frequency: humidity helps at 10 GHz (more refractivity)
            and hurts at 24+ GHz (absorption). Refractivity now uses native
            HRRR hybrid-sigma levels (10–50 m resolution) for duct detection.
          </p>
        </div>

        <div class="space-y-3">
          <h2 class="text-xl font-bold">What we've collected so far</h2>
          <p>Straight from the production database:</p>
          <div class="grid grid-cols-2 md:grid-cols-3 gap-3">
            <div class="stat bg-base-200 rounded-box">
              <div class="stat-title">Contacts</div>
              <div class="stat-value text-xl">{format_count(@stats.contacts)}</div>
            </div>
            <div class="stat bg-base-200 rounded-box">
              <div class="stat-title">Weather stations</div>
              <div class="stat-value text-xl">{format_count(@stats.weather_stations)}</div>
            </div>
            <div class="stat bg-base-200 rounded-box">
              <div class="stat-title">Surface observations</div>
              <div class="stat-value text-xl">{format_count(@stats.surface_observations)}</div>
            </div>
            <div class="stat bg-base-200 rounded-box">
              <div class="stat-title">Upper-air soundings</div>
              <div class="stat-value text-xl">{format_count(@stats.soundings)}</div>
            </div>
            <div class="stat bg-base-200 rounded-box">
              <div class="stat-title">HRRR profiles</div>
              <div class="stat-value text-xl">{format_count(@stats.hrrr_profiles)}</div>
            </div>
            <div class="stat bg-base-200 rounded-box">
              <div class="stat-title">IEMRE gridded obs</div>
              <div class="stat-value text-xl">{format_count(@stats.iemre_observations)}</div>
            </div>
            <div class="stat bg-base-200 rounded-box">
              <div class="stat-title">Terrain profiles</div>
              <div class="stat-value text-xl">{format_count(@stats.terrain_profiles)}</div>
            </div>
            <div class="stat bg-base-200 rounded-box">
              <div class="stat-title">Propagation scores</div>
              <div class="stat-value text-xl">{format_count(@stats.propagation_scores)}</div>
            </div>
            <div class="stat bg-base-200 rounded-box">
              <div class="stat-title">Beacons</div>
              <div class="stat-value text-xl">{format_count(@stats.beacons)}</div>
            </div>
            <div class="stat bg-base-200 rounded-box">
              <div class="stat-title">Commercial link samples</div>
              <div class="stat-value text-xl">{format_count(@stats.commercial_samples)}</div>
            </div>
          </div>
        </div>

        <div class="space-y-3">
          <h2 class="text-xl font-bold">How it's built</h2>
          <ul class="list-disc list-outside pl-5 space-y-2">
            <li>
              Elixir / Phoenix 1.8 + LiveView for everything web-facing. Ecto
              on Postgres for storage. Oban runs the background pipelines
              (HRRR pulls, terrain, ASOS, soundings, IEMRE, solar indices,
              commercial links).
            </li>
            <li>
              Leaflet + Canvas tile layers for the propagation heatmap — we
              render 0.125° cells at interactive frame rates. Tailwind v4 +
              daisyUI for layout, esbuild for JS bundling.
            </li>
            <li>
              Physics: ITU-R P.526-16 knife-edge + Deygout 3-edge terrain
              diffraction, ITU-R P.838-3 rain attenuation, dynamic k-factor
              from live HRRR refractivity gradients, great-circle geometry,
              FSPL with frequency-dependent O₂ and H₂O absorption.
            </li>
            <li>
              Data sources: NOAA HRRR (AWS S3, hourly analysis + 18 h
              forecasts), Iowa Environmental Mesonet (ASOS and upper-air
              soundings), IEMRE gridded reanalysis, SRTM 90 m terrain, SNMP
              polling on seven commercial microwave links near DFW at 5-minute
              intervals, and Copernicus ERA5 for pre-2014 contact enrichment.
            </li>
            <li>
              There's Nx/Axon/EXLA scaffolding for a small feed-forward model
              (13 features → 64 → 32 → 1 sigmoid). Not trained yet — waiting
              on enough calibrated data to not overfit.
            </li>
          </ul>
        </div>

        <div class="space-y-3">
          <h2 class="text-xl font-bold">What's next</h2>
          <ul class="list-disc list-outside pl-5 space-y-2">
            <li>
              <strong>Beacon calibration.</strong>
              A distributed network of amateur receivers reporting CW beacon
              signal levels on a schedule. Beacon submissions are already open
              to anyone via the Beacons page.
            </li>
            <li>
              <strong>Weight calibration.</strong>
              Fit the 9 factor weights against recorded distances / QSO counts
              per band so the score reflects real propagation, not my
              intuition.
            </li>
            <li>
              <strong>Training the model.</strong>
              Once there are enough clean (contact, conditions) pairs, train
              the Axon model to augment or replace the hand-tuned scoring.
            </li>
            <li>
              <strong>Better inputs.</strong> MRMS for precipitation at 24+ GHz where rain attenuation
              dominates, RTMA/URMA surface analysis blending, GOES total
              precipitable water.
            </li>
          </ul>
        </div>
        <div role="alert" class="alert alert-info text-center">
          <p>
            If this tool is useful to you, consider
            <a
              href="https://www.paypal.com/ncp/payment/53VLBD2E67JAE"
              target="_blank"
              class="link font-semibold underline"
            >
              donating to NTMS
            </a>
            to help keep the project running.
          </p>
        </div>
      </section>
    </Layouts.app>
    """
  end

  # For tables that get huge (tens of millions of rows), using count(*) on
  # every About page load is wasteful. We use PostgreSQL's pg_class.reltuples
  # estimate — it's maintained by ANALYZE and is accurate to within a few
  # percent, which is plenty for "big number on a marketing page."
  @estimate_tables ~w(hrrr_profiles propagation_scores surface_observations)

  @stat_keys ~w(
    contacts weather_stations surface_observations soundings hrrr_profiles
    iemre_observations terrain_profiles propagation_scores beacons
    commercial_samples
  )a

  defp fetch_stats do
    %{
      contacts: count_exact("contacts"),
      weather_stations: count_exact("weather_stations"),
      surface_observations: count_estimate("surface_observations"),
      soundings: count_exact("soundings"),
      hrrr_profiles: count_estimate("hrrr_profiles"),
      iemre_observations: count_exact("iemre_observations"),
      terrain_profiles: count_exact("terrain_profiles"),
      propagation_scores: count_estimate("propagation_scores"),
      beacons: count_exact("beacons"),
      commercial_samples: count_exact("commercial_samples")
    }
  rescue
    _ -> Map.new(@stat_keys, fn k -> {k, 0} end)
  end

  defp count_exact(table) do
    %Postgrex.Result{rows: [[count]]} =
      Repo.query!("SELECT count(*) FROM #{table}")

    count
  end

  defp count_estimate(table) when table in @estimate_tables do
    %Postgrex.Result{rows: [[estimate]]} =
      Repo.query!("SELECT reltuples::bigint FROM pg_class WHERE relname = $1", [table])

    max(estimate || 0, 0)
  end

  defp format_count(n) when is_integer(n) and n >= 1_000_000 do
    "#{Float.round(n / 1_000_000, 1)}M"
  end

  defp format_count(n) when is_integer(n) and n >= 1_000 do
    "#{Float.round(n / 1_000, 1)}k"
  end

  defp format_count(n) when is_integer(n), do: Integer.to_string(n)
  defp format_count(_), do: "—"
end