fix(about): per-table count fallback so missing tables don't blank everything
The whole-block try/rescue around fetch_stats meant any single failing query (a missing dev table, a transient pg blip) zeroed all 10 counters. Each count_exact/1 + count_estimate/1 now uses Repo.query/2 and falls back to 0 only for its own column. Bumps the inline 9-factor prose to 10-factor and pulls the contacts headline from the live count instead of a stale '58k+' string. Adds a CLAUDE.md note prompting future updates to refresh /about's prose when factor counts, data sources, or schema scale change.
This commit is contained in:
parent
7ab2aa8a11
commit
a0dfdd26af
2 changed files with 32 additions and 29 deletions
|
|
@ -221,6 +221,10 @@ User submits QSO → enqueue_for_qso() → weather/hrrr/terrain/iemre workers
|
|||
- **Always run `cargo clippy --all-targets -- -D warnings` before committing Rust changes.** The Forgejo CI step runs clippy with warnings-as-errors and a passing local `cargo build` does NOT catch lints (e.g. `manual_range_contains`, `iter_kv_map`, `useless_format`). A clippy failure blocks the image build and the deploy. The Rust precommit recipe is in the `Commands` section.
|
||||
- The CI build pipeline runs `cargo clippy --all-targets -- -D warnings && cargo test --release && cargo build --release`. Match this locally — `cargo build` alone is not enough.
|
||||
|
||||
### `/about` page maintenance
|
||||
- Refresh `lib/microwaveprop_web/live/about_live.ex` whenever **algorithm structure**, **factor count or weights**, **major data sources**, **schema scale claims (e.g. "58k+ contacts")**, or **roadmap items** change. The dashboard counts auto-refresh from the DB; the prose blocks (overview, "How it works", "How it's built", "What's next") do not.
|
||||
- The page renders a per-table count via `count_exact/1` and `count_estimate/1`. Each query is wrapped individually so a missing dev table doesn't blank the whole dashboard — keep that pattern when adding new stats.
|
||||
|
||||
### Deployment
|
||||
- Production runs on Kubernetes (`home-cluster` context, `prop` namespace) — use `kubectl -n prop ...`
|
||||
- Deployment name `prop` (3 replicas). Inspect with `kubectl -n prop get pods`, `kubectl -n prop logs deploy/prop`, `kubectl -n prop exec deploy/prop -- ...`
|
||||
|
|
|
|||
|
|
@ -76,27 +76,29 @@ defmodule MicrowavepropWeb.AboutLive do
|
|||
profiles. We derive refractivity profiles, minimum refractivity
|
||||
gradient, ducting detection, boundary-layer depth, and precipitable
|
||||
water for every grid cell. ASOS surface obs, 12-hourly upper-air
|
||||
soundings, and gridded IEMRE reanalysis fill in the gaps. A 9-factor
|
||||
composite score is written out for every 0.125° cell across CONUS,
|
||||
for each hour of an 18-hour forecast.
|
||||
soundings, and gridded IEMRE reanalysis fill in the gaps. A
|
||||
10-factor composite score is written out for every 0.125° cell
|
||||
across CONUS, for each hour of an 18-hour forecast.
|
||||
</li>
|
||||
<li>
|
||||
<strong>The contacts.</strong>
|
||||
58k+ amateur microwave QSOs, each tagged with the atmosphere at
|
||||
both ends of the path (and along it) at the time the contact
|
||||
happened. That's a ground-truth dataset we can actually fit against
|
||||
— "when you had this score, how far did the contact go, and did it
|
||||
happen at all?"
|
||||
{format_count(@stats.contacts)}+ amateur microwave QSOs, each
|
||||
tagged with the atmosphere at both ends of the path (and along it)
|
||||
at the time the contact happened. That's a ground-truth dataset we
|
||||
can actually fit against — "when you had this score, how far did
|
||||
the contact go, and did it happen at all?"
|
||||
</li>
|
||||
</ol>
|
||||
<p>
|
||||
The scoring is a weighted sum of ten factors: rain, humidity,
|
||||
precipitable water (PWAT), season, refractivity gradient, pressure,
|
||||
T–Td depression, sky cover, wind, and time of day. Weights were
|
||||
calibrated via gradient descent against 5,000 QSOs. The physics
|
||||
changes by frequency: humidity helps at 10 GHz (more refractivity)
|
||||
and hurts at 24+ GHz (absorption). Refractivity now uses native
|
||||
HRRR hybrid-sigma levels (10–50 m resolution) for duct detection.
|
||||
T–Td depression, sky cover, wind, and time of day — weights
|
||||
recalibrated via gradient descent against the contact dataset. The
|
||||
physics changes by frequency: humidity helps at 10 GHz (more
|
||||
refractivity) and hurts at 24+ GHz (absorption). Refractivity uses
|
||||
native HRRR hybrid-sigma levels (10–50 m resolution) for duct
|
||||
detection, with a pressure-level fallback (~250 m) when the native
|
||||
data isn't available.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
|
|
@ -233,12 +235,6 @@ defmodule MicrowavepropWeb.AboutLive do
|
|||
# percent, which is plenty for "big number on a marketing page."
|
||||
@estimate_tables ~w(hrrr_profiles propagation_scores surface_observations)
|
||||
|
||||
@stat_keys ~w(
|
||||
contacts weather_stations surface_observations soundings hrrr_profiles
|
||||
iemre_observations terrain_profiles propagation_scores beacons
|
||||
commercial_samples
|
||||
)a
|
||||
|
||||
defp fetch_stats do
|
||||
%{
|
||||
contacts: count_exact("contacts"),
|
||||
|
|
@ -252,22 +248,25 @@ defmodule MicrowavepropWeb.AboutLive do
|
|||
beacons: count_exact("beacons"),
|
||||
commercial_samples: count_exact("commercial_samples")
|
||||
}
|
||||
rescue
|
||||
_ -> Map.new(@stat_keys, fn k -> {k, 0} end)
|
||||
end
|
||||
|
||||
# Each count is wrapped individually so a missing table in dev (or a
|
||||
# transient query failure) never blanks the entire dashboard. The
|
||||
# previous outer try/rescue zeroed all 10 numbers if any single one
|
||||
# raised — every stat read 0 on environments lacking hrrr_profiles
|
||||
# or propagation_scores.
|
||||
defp count_exact(table) do
|
||||
%Postgrex.Result{rows: [[count]]} =
|
||||
Repo.query!("SELECT count(*) FROM #{table}")
|
||||
|
||||
count
|
||||
case Repo.query("SELECT count(*) FROM #{table}") do
|
||||
{:ok, %Postgrex.Result{rows: [[count]]}} -> count
|
||||
_ -> 0
|
||||
end
|
||||
end
|
||||
|
||||
defp count_estimate(table) when table in @estimate_tables do
|
||||
%Postgrex.Result{rows: [[estimate]]} =
|
||||
Repo.query!("SELECT reltuples::bigint FROM pg_class WHERE relname = $1", [table])
|
||||
|
||||
max(estimate || 0, 0)
|
||||
case Repo.query("SELECT reltuples::bigint FROM pg_class WHERE relname = $1", [table]) do
|
||||
{:ok, %Postgrex.Result{rows: [[estimate]]}} -> max(estimate || 0, 0)
|
||||
_ -> 0
|
||||
end
|
||||
end
|
||||
|
||||
defp format_count(n) when is_integer(n) and n >= 1_000_000 do
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue