fix(about): per-table count fallback so missing tables don't blank everything
The whole-block try/rescue around fetch_stats meant any single failing query (a missing dev table, a transient pg blip) zeroed all 10 counters. Each count_exact/1 + count_estimate/1 now uses Repo.query/2 and falls back to 0 only for its own column. Bumps the inline 9-factor prose to 10-factor and pulls the contacts headline from the live count instead of a stale '58k+' string. Adds a CLAUDE.md note prompting future updates to refresh /about's prose when factor counts, data sources, or schema scale change.
This commit is contained in:
parent
7ab2aa8a11
commit
a0dfdd26af
2 changed files with 32 additions and 29 deletions
|
|
@ -221,6 +221,10 @@ User submits QSO → enqueue_for_qso() → weather/hrrr/terrain/iemre workers
|
||||||
- **Always run `cargo clippy --all-targets -- -D warnings` before committing Rust changes.** The Forgejo CI step runs clippy with warnings-as-errors and a passing local `cargo build` does NOT catch lints (e.g. `manual_range_contains`, `iter_kv_map`, `useless_format`). A clippy failure blocks the image build and the deploy. The Rust precommit recipe is in the `Commands` section.
|
- **Always run `cargo clippy --all-targets -- -D warnings` before committing Rust changes.** The Forgejo CI step runs clippy with warnings-as-errors and a passing local `cargo build` does NOT catch lints (e.g. `manual_range_contains`, `iter_kv_map`, `useless_format`). A clippy failure blocks the image build and the deploy. The Rust precommit recipe is in the `Commands` section.
|
||||||
- The CI build pipeline runs `cargo clippy --all-targets -- -D warnings && cargo test --release && cargo build --release`. Match this locally — `cargo build` alone is not enough.
|
- The CI build pipeline runs `cargo clippy --all-targets -- -D warnings && cargo test --release && cargo build --release`. Match this locally — `cargo build` alone is not enough.
|
||||||
|
|
||||||
|
### `/about` page maintenance
|
||||||
|
- Refresh `lib/microwaveprop_web/live/about_live.ex` whenever **algorithm structure**, **factor count or weights**, **major data sources**, **schema scale claims (e.g. "58k+ contacts")**, or **roadmap items** change. The dashboard counts auto-refresh from the DB; the prose blocks (overview, "How it works", "How it's built", "What's next") do not.
|
||||||
|
- The page renders a per-table count via `count_exact/1` and `count_estimate/1`. Each query is wrapped individually so a missing dev table doesn't blank the whole dashboard — keep that pattern when adding new stats.
|
||||||
|
|
||||||
### Deployment
|
### Deployment
|
||||||
- Production runs on Kubernetes (`home-cluster` context, `prop` namespace) — use `kubectl -n prop ...`
|
- Production runs on Kubernetes (`home-cluster` context, `prop` namespace) — use `kubectl -n prop ...`
|
||||||
- Deployment name `prop` (3 replicas). Inspect with `kubectl -n prop get pods`, `kubectl -n prop logs deploy/prop`, `kubectl -n prop exec deploy/prop -- ...`
|
- Deployment name `prop` (3 replicas). Inspect with `kubectl -n prop get pods`, `kubectl -n prop logs deploy/prop`, `kubectl -n prop exec deploy/prop -- ...`
|
||||||
|
|
|
||||||
|
|
@ -76,27 +76,29 @@ defmodule MicrowavepropWeb.AboutLive do
|
||||||
profiles. We derive refractivity profiles, minimum refractivity
|
profiles. We derive refractivity profiles, minimum refractivity
|
||||||
gradient, ducting detection, boundary-layer depth, and precipitable
|
gradient, ducting detection, boundary-layer depth, and precipitable
|
||||||
water for every grid cell. ASOS surface obs, 12-hourly upper-air
|
water for every grid cell. ASOS surface obs, 12-hourly upper-air
|
||||||
soundings, and gridded IEMRE reanalysis fill in the gaps. A 9-factor
|
soundings, and gridded IEMRE reanalysis fill in the gaps. A
|
||||||
composite score is written out for every 0.125° cell across CONUS,
|
10-factor composite score is written out for every 0.125° cell
|
||||||
for each hour of an 18-hour forecast.
|
across CONUS, for each hour of an 18-hour forecast.
|
||||||
</li>
|
</li>
|
||||||
<li>
|
<li>
|
||||||
<strong>The contacts.</strong>
|
<strong>The contacts.</strong>
|
||||||
58k+ amateur microwave QSOs, each tagged with the atmosphere at
|
{format_count(@stats.contacts)}+ amateur microwave QSOs, each
|
||||||
both ends of the path (and along it) at the time the contact
|
tagged with the atmosphere at both ends of the path (and along it)
|
||||||
happened. That's a ground-truth dataset we can actually fit against
|
at the time the contact happened. That's a ground-truth dataset we
|
||||||
— "when you had this score, how far did the contact go, and did it
|
can actually fit against — "when you had this score, how far did
|
||||||
happen at all?"
|
the contact go, and did it happen at all?"
|
||||||
</li>
|
</li>
|
||||||
</ol>
|
</ol>
|
||||||
<p>
|
<p>
|
||||||
The scoring is a weighted sum of ten factors: rain, humidity,
|
The scoring is a weighted sum of ten factors: rain, humidity,
|
||||||
precipitable water (PWAT), season, refractivity gradient, pressure,
|
precipitable water (PWAT), season, refractivity gradient, pressure,
|
||||||
T–Td depression, sky cover, wind, and time of day. Weights were
|
T–Td depression, sky cover, wind, and time of day — weights
|
||||||
calibrated via gradient descent against 5,000 QSOs. The physics
|
recalibrated via gradient descent against the contact dataset. The
|
||||||
changes by frequency: humidity helps at 10 GHz (more refractivity)
|
physics changes by frequency: humidity helps at 10 GHz (more
|
||||||
and hurts at 24+ GHz (absorption). Refractivity now uses native
|
refractivity) and hurts at 24+ GHz (absorption). Refractivity uses
|
||||||
HRRR hybrid-sigma levels (10–50 m resolution) for duct detection.
|
native HRRR hybrid-sigma levels (10–50 m resolution) for duct
|
||||||
|
detection, with a pressure-level fallback (~250 m) when the native
|
||||||
|
data isn't available.
|
||||||
</p>
|
</p>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
|
@ -233,12 +235,6 @@ defmodule MicrowavepropWeb.AboutLive do
|
||||||
# percent, which is plenty for "big number on a marketing page."
|
# percent, which is plenty for "big number on a marketing page."
|
||||||
@estimate_tables ~w(hrrr_profiles propagation_scores surface_observations)
|
@estimate_tables ~w(hrrr_profiles propagation_scores surface_observations)
|
||||||
|
|
||||||
@stat_keys ~w(
|
|
||||||
contacts weather_stations surface_observations soundings hrrr_profiles
|
|
||||||
iemre_observations terrain_profiles propagation_scores beacons
|
|
||||||
commercial_samples
|
|
||||||
)a
|
|
||||||
|
|
||||||
defp fetch_stats do
|
defp fetch_stats do
|
||||||
%{
|
%{
|
||||||
contacts: count_exact("contacts"),
|
contacts: count_exact("contacts"),
|
||||||
|
|
@ -252,22 +248,25 @@ defmodule MicrowavepropWeb.AboutLive do
|
||||||
beacons: count_exact("beacons"),
|
beacons: count_exact("beacons"),
|
||||||
commercial_samples: count_exact("commercial_samples")
|
commercial_samples: count_exact("commercial_samples")
|
||||||
}
|
}
|
||||||
rescue
|
|
||||||
_ -> Map.new(@stat_keys, fn k -> {k, 0} end)
|
|
||||||
end
|
end
|
||||||
|
|
||||||
|
# Each count is wrapped individually so a missing table in dev (or a
|
||||||
|
# transient query failure) never blanks the entire dashboard. The
|
||||||
|
# previous outer try/rescue zeroed all 10 numbers if any single one
|
||||||
|
# raised — every stat read 0 on environments lacking hrrr_profiles
|
||||||
|
# or propagation_scores.
|
||||||
defp count_exact(table) do
|
defp count_exact(table) do
|
||||||
%Postgrex.Result{rows: [[count]]} =
|
case Repo.query("SELECT count(*) FROM #{table}") do
|
||||||
Repo.query!("SELECT count(*) FROM #{table}")
|
{:ok, %Postgrex.Result{rows: [[count]]}} -> count
|
||||||
|
_ -> 0
|
||||||
count
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
defp count_estimate(table) when table in @estimate_tables do
|
defp count_estimate(table) when table in @estimate_tables do
|
||||||
%Postgrex.Result{rows: [[estimate]]} =
|
case Repo.query("SELECT reltuples::bigint FROM pg_class WHERE relname = $1", [table]) do
|
||||||
Repo.query!("SELECT reltuples::bigint FROM pg_class WHERE relname = $1", [table])
|
{:ok, %Postgrex.Result{rows: [[estimate]]}} -> max(estimate || 0, 0)
|
||||||
|
_ -> 0
|
||||||
max(estimate || 0, 0)
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
defp format_count(n) when is_integer(n) and n >= 1_000_000 do
|
defp format_count(n) when is_integer(n) and n >= 1_000_000 do
|
||||||
|
|
|
||||||
Loading…
Add table
Reference in a new issue