Commit graph

1519 commits

Author SHA1 Message Date
FluxCD
c3f1875df8 chore: update prop image to git.mcintire.me/graham/prop:main-1777145135-77bc894 [skip ci] 2026-04-25 19:35:19 +00:00
77bc894f56
feat(skewt): add wet-bulb / virtual-T traces and right-edge level rail
Three additions that bring the diagram closer to the SPC sounding
viewer's full-information chart:

  * Wet-bulb temperature trace, plotted as a thinner blue line under
    the dewpoint trace. Comes from a Stipanuk-style bracket between
    the level's dewpoint and dry-bulb temperatures (already used by
    `SkewtParams` for the wet-bulb-zero level — now public so the
    renderer can call it per level).

  * Environment virtual-temperature trace and parcel virtual-T
    trajectory, both as thin pinkish dashed lines. The parcel
    virtual-T is the actual buoyancy curve CAPE integrates against,
    so showing it next to the dry-bulb parcel ascent makes the
    moisture correction visible. New `SkewtParams.virtual_temp_c/2`
    and `/3` helpers, plus `SkewtParams.mixing_ratio/2`.

  * Critical-level rail on the right edge. Each of LCL / LFC / EL /
    0 °C / WBZ that resolves on the current profile gets a short
    horizontal tick at the right of the plot and a `<label> <p> mb`
    tag just outside it — matches the SPC chart's `143 mb (mix)` /
    `158 mb (mix)` annotation column. SkewtLive computes the
    pressures (and converts the height-keyed 0 °C and WBZ from
    SkewtParams back to pressure via `pressure_at_height/2`) before
    handing them to `SkewtSvg.render/2`.

Wind-derived overlays (hodograph, wind barbs) remain off — HRRR
profile data is wind-free above 10 m AGL.

mix test: 27 / 27 across the four /skewt-related test files,
2,902 / 2,902 + 221 properties on the full suite. mix credo --strict
clean (one cyclomatic-complexity refactor extracted along the way).
2026-04-25 14:24:28 -05:00
FluxCD
2ae9befd7f chore: update prop image to git.mcintire.me/graham/prop:main-1777144814-fb0c4f7 [skip ci] 2026-04-25 19:23:15 +00:00
fb0c4f7c77
fix(skewt): only fall back to profiles with all 13 pressure levels
When the on-disk ProfilesFile is empty SkewtLive falls back to the
hrrr_profiles Postgres table, but per-QSO HrrrFetchWorker rows
sometimes land with a partial profile (4–12 levels) when individual
byte-range fetches fail. The fallback was happily picking those rows,
which rendered a stubby Skew-T missing the upper troposphere.

HrrrProfileLookup.{list_valid_times_near, read_point_near}/3 now
filter on `coalesce(cardinality(profile), 0) >= min_profile_levels`,
defaulting to 13 (HRRR's standard pressure-level set: 1000/925/850/
700/500/400/300/250/200/150/100 mb plus the two boundary surfaces).
Pass `min_profile_levels: 0` to keep the partial rows for diagnostic
queries.

Confirmed against the dev mirror: the recent table has ~3.1 M rows
at 13 levels and ~250 k at 4–12 levels; the new default skips the
partial slice cleanly.

Tests: 8/8 HrrrProfileLookupTest cases (added two new ones for the
threshold). mix credo --strict clean.
2026-04-25 14:19:56 -05:00
FluxCD
5a0bd3bbf5 chore: update prop image to git.mcintire.me/graham/prop:main-1777141855-b34b186 [skip ci] 2026-04-25 18:41:06 +00:00
b34b186e61
fix(skewt): clip grid + traces to the plot box
Skewed isotherms, dry adiabats, and mixing-ratio lines were drifting
outside the frame at the corners — most visibly the warm-side
isotherms exiting the right edge near the top of the diagram. The
existing `clip_polyline/1` only filtered with a ±200 px slack, so
segments crossing the boundary still drew past the frame.

Wrap every grid layer + the T/Td/parcel polylines in a
`<g clip-path="url(#skewt-plot-clip)">`. The clipPath defines a
rectangle from (left, top) to (right, bottom). Frame outline and
axis labels stay outside the clip group so they keep rendering at
the box edge unchanged. The hover-cursor crosshair also lives
outside the clip so its readout box can sit flush against the plot
corner without being trimmed.

mix test: 2,902 / 2,902 + 221 properties green. mix credo --strict
clean.
2026-04-25 13:30:25 -05:00
FluxCD
e255fcf3b6 chore: update prop image to git.mcintire.me/graham/prop:main-1777141619-be11749 [skip ci] 2026-04-25 18:29:01 +00:00
be1174914f
feat(skewt): NOAA SPC line colors + sounding-parameter parity
Two changes that bring /skewt visually and analytically into parity
with the NOAA SPC sounding viewer
(https://www.spc.noaa.gov/exper/soundings/):

NOAA-style line set
-------------------

Updated the SkewtSvg style block to match the public SPC chart:

  * Temperature trace ........ pure red (#ff0000) at 2.5 px stroke
  * Dewpoint trace ........... pure green (#00aa00) at 2.5 px stroke
  * Isotherms ................ dashed teal (#14b8a6)
  * Dry adiabats ............. solid orange (#f59e0b)
  * Moist (saturated) adiabats dashed sky-blue (#38bdf8)
  * Mixing-ratio lines ....... dotted darker green (#16a34a), with
                               labels at 700 mb showing the g/kg
                               value, drawn only below 600 mb (matches
                               the SPC chart's lower-band number row)
  * Parcel ascent ............ new dashed grey (#475569) line, drawn
                               whenever SkewtParams.derive returns a
                               non-empty parcel_trace

The mixing-ratio set is the SPC standard (0.4 / 0.7 / 1 / 2 / 3 / 5 /
8 / 12 / 16 / 20 g/kg) instead of the prior arbitrary set.

SkewtParams: SPC parcel-theory + indices module
-----------------------------------------------

New `Microwaveprop.Weather.SkewtParams` derives the parameter set that
ships in the SPC `*.txt` companion file:

  * LCL pressure / temperature / height (Bolton 1980 eq 22)
  * LFC pressure / height
  * EL pressure / height
  * SBCAPE, SBCIN (J/kg, virtual-temperature integration with
    surface parcel)
  * 3 km CAPE
  * Lifted Index (parcel ↔ environment T at 500 mb)
  * Freezing level (T = 0 °C height) and Wet-Bulb Zero
  * DCAPE (downdraft CAPE — picks the min-θₑ source level in
    400–700 mb, descends moist-adiabatically to surface)
  * Layer lapse rates: sfc–3 km, 700–500 mb, 850–500 mb
  * `parcel_trace` — pressure/temperature trajectory used by
    SkewtSvg to draw the dashed parcel-ascent overlay

Wind-derived indices (SRH, BWD, Bunkers motion, SCP, STP, SHIP) are
deliberately *not* produced and the LiveView surfaces a one-line note
explaining why: HRRR persists wind only at 10 m AGL, so per-pressure-
level shear can't be computed from this data source.

LiveView panel restructured into grouped sections (Surface, Convective
parcel, Levels, Lapse rates, Moisture/stability, Refractivity) so the
layout matches the SPC viewer's order. The Levels rows show pressure
*and* height ("872 mb · 1,140 m") for quick cross-reference between
the diagram axis and the readout.

Tests: 8/8 new SkewtParamsTest cases (LCL Bolton check, saturated-air
edge case, full field-set presence, summer profile produces SBCAPE > 0
and LI < 0, capped profile produces SBCAPE = 0 and LI > 0, freezing
level interpolated correctly, sfc–3 km lapse rate, empty profile no
crash). 25/25 across all skewt-related test files. mix test green
(2,902 / 2,902 + 221 properties), mix credo --strict clean, mix
assets.build clean.
2026-04-25 13:26:41 -05:00
a4cd7632eb
feat(skewt): hover crosshair + DB fallback for missing on-disk profiles
Two follow-ups to the earlier /skewt work:

1. Interactive crosshair (matches the SPC-viewer feel from the
   reference screenshot). Mousing over the diagram now drops a
   horizontal pressure cursor with a top-left readout box showing the
   pressure at the cursor's altitude (mb), the height in m and ft,
   and the linearly interpolated temperature and dewpoint at that
   level. Two coloured dots track the T and Td traces along the
   cursor so the user can read the spread visually.

   Mechanics: SkewtSvg.geometry/0 exposes the plot geometry (left,
   right, top, bottom, p_bottom, p_top, t_min, t_max, skew) as JSON;
   SkewtSvg emits an initially-hidden <g class="skewt-cursor"> layer.
   A new Skewt hook in assets/js/app.ts inverts pressure_y to recover
   pressure from the cursor's viewBox-Y, walks the (descending-pres-
   sorted) profile to interp T/Td/height, and updates the elements.
   The hook also handles mouseleave to hide the cursor and uses the
   SVG's screen-CTM so the math stays correct under any container
   aspect-ratio resize.

2. DB fallback for `available_valid_times`/`load_profile`. When the
   on-disk `ProfilesFile` has nothing for the location-and-window
   (true on dev, transient on prod between chain runs), SkewtLive
   now falls back to `HrrrProfileLookup.{list_valid_times_near,
   read_point_near}/3` against the `hrrr_profiles` Postgres table.
   The lookup uses the same ±0.07° spatial tolerance and the
   `(lat, lon, valid_time)` unique index that `Weather.find_nearest_
   hrrr/3` already relies on, so the queries are index-only.

   Returned shape matches `ProfilesFile.read_point/3` so SkewtLive
   reads from either source transparently. NaiveDateTime → UTC
   normalisation is centralised in `ensure_utc/1` so the LiveView
   sees `%DateTime{time_zone: "Etc/UTC"}` regardless of which side
   produced the value.

Tests: 6/6 new HrrrProfileLookupTest cases (list/limit/tolerance,
read nearest cell, nil for far cells, picks closest among matches),
3/3 SkewtLiveTest still green, 2,894 total Elixir tests + 221
properties green via mix test (one pre-existing flaky test in
admin/contact_edit_live re-runs clean). mix credo --strict clean.
mix assets.build clean.
2026-04-25 13:17:58 -05:00
FluxCD
7ca9381ba6 chore: update prop image to git.mcintire.me/graham/prop:main-1777139917-9e776a0 [skip ci] 2026-04-25 18:02:52 +00:00
9e776a07d5
fix(skewt): write profile files for f01..f18 forecast hours
The /skewt page was empty in prod because Phase 2 of the Rust cutover
moved f01..f18 grid scoring to Rust but left profile-file writing only
on the analysis (f00) path. Result: /data/scores/profiles/ accumulated
one file per chain run instead of nineteen, so SkewtLive only ever saw
the analysis hour — and that hour falls outside the 1 h past cutoff
within an hour of the chain finishing, leaving the page empty most of
the time.

Two surgical fixes:

  * `pipeline::run_chain_step` now builds `profile_entries` alongside
    `prepared` from the merged grid in a single iter pass, then spawns
    `profiles_file::write_atomic` on the blocking pool to overlap the
    NFS write with the score-band scoring/write fan-out — same pattern
    `run_analysis_step` already uses. `ChainStepStats` gains
    `profile_cells_written` so the worker log line is symmetric with
    the analysis step's existing field and a future failure mode that
    drops the profile would show up as a divergence.

  * `SkewtLive.available_valid_times/0` widens the past cutoff from
    1 h to 3 h. The chain's analysis hour can be ~70 min old by the
    time the next chain finishes (and ~2 h on a missed cycle), so the
    1 h cutoff was leaving the page blank during normal transients.
    18 h forward window unchanged.

Cosmetic: cargo fmt picked up trivial rewrites in decoder/fetcher/
native_duct/hrrr_point_worker that had drifted; rolled in.

Verification: cargo build --release, cargo clippy --lib --bins -D
warnings, cargo test --lib (134/134), mix test
test/microwaveprop_web/live/skewt_live_test.exs (3/3) all green.
2026-04-25 12:58:10 -05:00
3a1e79fb67
feat(scorer): retire HPBL multiplier and native-duct 1.15× boost
Both retired in light of the 2026-04-25 revision report
(`docs/algo-reports/2026-04-25-algo-revisions.md`):

  * HPBL boundary-layer multiplier in `Scorer.score_refractivity/{3,4,5}`
    is gone. On the n=47,418 10 GHz HRRR-matched corpus rho_hpbl is
    +0.004; binned distance is flat to within ±5 % across 200–2,000 m.
    The previously reported 2.3× ratio between shallow and deep HPBL
    bins was a small-corpus artefact that disappeared once the matched
    corpus passed ~5,000 contacts.

  * Native-profile 1.15× duct boost is gone. At 10 GHz on n=52,341
    matched contacts, cells where `best_duct_band_ghz` ≥ band ran
    *shorter* than no-duct cells (198 km vs 211 km, n=173 vs 44,658).
    The Bulk Richardson gate that existed only to suppress that boost
    in turbulent conditions is also retired; arity preserved on every
    `score_refractivity` overload so callers compile unchanged.

Same retirements applied in the Rust port (`rust/prop_grid_rs/src/scorer.rs`)
to keep the Elixir/Rust parity tests green.

algo.md updated end-to-end:

  * Finding 4 (HPBL) rewritten to "no usable signal" with the
    n=47,418 bin table.
  * Finding 5 (gradient) tightened: load-bearing only at 24 GHz,
    set band-specific gradient weight to 0 outside [10, 47] GHz on
    the next derive_band_weights run.
  * Finding 8 (time-of-day) augmented with the robust hour-bucketed
    amplitude index (40 / 82 / 101 % at 10 / 24 / 47 GHz) and a
    selection-bias caveat for VHF/UHF (where 129–148 % amplitude
    reflects evening contest scheduling, not propagation).
  * Finding 9 (sounding) refreshed against the n=27,058 corpus.
  * Part 2c "Native-profile duct boost" rewritten as
    "retired 2026-04-25" with the falsifying join.
  * Part 2c "Commercial-link inverse sensor" promoted from deferred
    with the 22-day, 38k-sample SNMP corpus, the +0.61 / −0.61 PWAT
    and pressure correlations against the 37-hour HRRR overlap, and
    the 04–05 / 10–12 local two-peak morning fade window.
  * Part 2b "Pressure" annotated with the U-shape at 10 GHz and the
    next-iteration target.
  * Part 2d "What stayed, what left" + "Open items" updated to
    reflect the four code/doc moves.

119 Elixir scorer tests green, 134 Rust scorer tests green, 2,888
total Elixir tests + 221 properties green via `mix precommit`.
2026-04-25 12:42:56 -05:00
2b071c3b56
feat(recalibrate): classify data gaps + add NARR/HRRR coverage breakdowns
The "Data gaps" section was a flat key/value dump that buried what was
actually broken (hrrr_climatology=0, rtma=0, metar5=0) under metrics
that are merely informational (pre2014_contacts=207). Operators reading
the report had to know each metric's healthy range by heart.

This commit:

  * Classifies every gap metric BROKEN / WARN / OK via DATA_GAP_RULES,
    with a one-line remediation hint per non-OK row (e.g. "run
    `mix hrrr_climatology`").
  * Prints BROKEN items to stderr at runtime so a CI/cron run surfaces
    them without anyone opening the markdown file.
  * Adds NARR-coverage-by-year and HRRR-status-by-year tables. With
    only 358 NARR rows, the year breakdown shows whether the backfill
    ever started; with 36k pending HRRR contacts, the year breakdown
    shows whether the backlog is a stalled historical backfill or a
    stuck live queue — different remediations.
  * Reports HRRR enrichment completeness as a percentage instead of
    forcing the reader to do the division.
2026-04-25 12:42:56 -05:00
FluxCD
0f21821ca5 chore: update prop image to git.mcintire.me/graham/prop:main-1777137248-e40ade3 [skip ci] 2026-04-25 17:16:34 +00:00
e40ade3b19
feat(skewt): add /skewt LiveView with HRRR-backed Skew-T-Log-P plot
Single search bar accepts an address, Maidenhead grid square, or
callsign; resolves it to lat/lon via Geocoder / Maidenhead /
CallsignLocation (cheapest classification first), snaps to the HRRR
grid via the existing ProfilesFile reader, and renders an SVG
Skew-T-Log-P with isobars, skewed isotherms, dry/moist adiabats, and
mixing-ratio lines. A button row picks between every available
forecast hour (now → f18); default is the most recent analysis.
Right pane lists derived stability and refractivity stats from
SoundingParams.derive (PWAT, BL depth, K-index, lifted index, min
dN/dh, ducting status + per-duct base/top/ΔM).

Renderer is server-side SVG so the page works without JS and serializes
into the LiveView payload as a single tag. Wind barbs are deliberately
omitted — HRRR's persisted profile carries wind only at 10 m AGL.
2026-04-25 12:13:46 -05:00
FluxCD
c1f04df169 chore: update prop image to git.mcintire.me/graham/prop-grid-rs:main-1777132414-6aa91e7 [skip ci] 2026-04-25 16:11:22 +00:00
FluxCD
6fa65657ee chore: update prop image to git.mcintire.me/graham/prop:main-1777132397-6aa91e7 [skip ci] 2026-04-25 15:56:16 +00:00
6aa91e7656
fix: April 2026 codebase review — address 13 bugs across propagation chain
Each fix is covered by a regression test that fails on `main` and
passes on this commit.

Round 1 (initial review):

* propagation: thread `latitude` into the conditions map so
  `score_season/4` actually picks up regional multipliers
* hrrr_client / fetcher.rs: `nearest_hrrr_hour` rounds DOWN, never at
  a future cycle that NOAA hasn't published yet
* radio: spherical-vector great-circle midpoint replaces the
  arithmetic mean — anti-meridian paths no longer fold to Greenwich
* weather: `reconcile_weather_statuses` scales the longitude band by
  `1 / cos(lat)` so the bbox stays ~150 km wide at every latitude
* radio/maidenhead: clamp 90°/180° below the field-bucket overflow so
  `from_latlon` never emits invalid characters like 'S'
* prop_grid_rs/pipeline: merge HRRR + NEXRAD-derived rain rates and
  read `best_duct_freq_ghz` into `best_duct_band_ghz` so the Native
  Duct Boost actually fires
* propagation/region (Elixir + Rust): inclusive upper bounds so points
  exactly at lat_max get the regional multiplier
* weather/sounding_params (Elixir + Rust): drop the 10 m gradient
  floor so HRRR's thin near-surface layers stop hiding sharp ducts
* weather/sounding_params: when the profile ends inside a duct,
  finalize it with the highest sample as the top instead of throwing
  it away (Rust port already correct)

Round 2 (post-fix sweep):

* radio + commercial: single canonical haversine in Radio (atan2
  form); Commercial delegates instead of carrying a second copy that
  could disagree at threshold distances
* prop_grid_rs/profiles_file: `snap_coords` matches Elixir's
  step-aware snap (`round(coord/0.125) * 0.125`, then 3-dp round) so
  Rust-keyed and Elixir-keyed profile maps land on the same cell
* weather/grib2/wgrib2: `parse_lon_val_segment` uses `Float.parse`
  uniformly — wgrib2 dropping the trailing `.0` from a longitude no
  longer crashes the whole chain step
2026-04-25 10:52:51 -05:00
FluxCD
15050bd2d8 chore: update prop image to git.mcintire.me/graham/prop:main-1777129865-90bf44c [skip ci] 2026-04-25 15:15:58 +00:00
FluxCD
0ebe0e2111 chore: update prop image to git.mcintire.me/graham/prop:main-1777129708-c17f912 [skip ci] 2026-04-25 15:10:50 +00:00
90bf44ce90
fix(submit): allowlist switch_tab values instead of trusting client input
`handle_event("switch_tab", ...)` ran `String.to_existing_atom/1` on
the raw `tab` value from the browser, so a stale, forged, or
mistyped client event with any unknown string crashed the LiveView
with `ArgumentError: not an already existing atom`.

Compares against an explicit `@valid_tabs` allowlist (`:single`,
`:csv`, `:adif`) — known values switch the active tab, anything
else is ignored.
2026-04-25 10:10:42 -05:00
c17f912622
fix(radio): invalidate contact-map cache on edits, not just inserts
`apply_edit_to_contact/2` is the path for both owner direct-edits and
admin reviewed-edits. The public contact-map + total-count + gzipped
controller payload caches are built from `private == false` rows, but
only the insert path was clearing them — every other update left
/api/contacts/map and /contacts/map serving the previous snapshot for
up to 10 minutes. A `private: false -> true` flip on a contact already
present in the cached payload was a temporary privacy leak; grid
corrections, callsign changes, and flagged_invalid flips silently
diverged from the live row.

Extracts the three `Cache.invalidate` calls into
`invalidate_contact_map_caches/0` and runs it from both
`apply_edit_to_contact/2` and the existing insert path so the public
view stays in sync.
2026-04-25 10:08:14 -05:00
FluxCD
ff5b0acf98 chore: update prop image to git.mcintire.me/graham/prop:main-1777129494-1399658 [skip ci] 2026-04-25 15:07:35 +00:00
13996583bd
fix(radio): add private clause to current_value/2
The /contacts/:id edit form always submits the `private` checkbox, but
`current_value/2` had no `"private"` clause and fell through to the
catch-all `nil`. The Contact schema defaults `private: false`, so an
unchanged box compared `false != nil` and was always tagged as a
change — for non-owners that meant a spurious pending review queued
on every save, and for owners/admins it triggered a no-op direct
update.

Adds the missing clause so the diff sees boolean-vs-boolean and
correctly returns `:no_changes` when the box is untouched.
2026-04-25 10:04:39 -05:00
2365c19321
fix(radio): coerce qso_timestamp string in normalize_proposed
The /contacts/:id edit form pre-fills qso_timestamp via
`Calendar.strftime(ts, "%Y-%m-%d %H:%M")` and submits whatever the
user typed back unchanged. Two failure modes resulted:

1. `diff_against_contact/2` compared that string against the stored
   `%DateTime{}`, never matched, and treated every untouched submit
   as a modification.
2. `build_contact_changes/2` then ran `DateTime.from_iso8601/1` on
   the space-separated, no-seconds, no-Z form and crashed with
   `MatchError` on `{:error, :invalid_format}` — direct owner/admin
   submits raised instead of saving.

Adds `normalize_timestamp_field/2` to `normalize_proposed/1` that
parses both the form's `"YYYY-MM-DD HH:MM[:SS]"` and the strict
`"YYYY-MM-DDTHH:MM:SSZ"` shapes into a UTC `%DateTime{}` (with
`NaiveDateTime.from_iso8601` doing the heavy lifting after a small
seconds-padding pass). Empty strings drop the field; unparseable
input drops it too so a typo becomes "no changes" rather than a
500.
2026-04-25 10:03:26 -05:00
FluxCD
8fae4d30bb chore: update prop image to git.mcintire.me/graham/prop:main-1777128883-938af4d [skip ci] 2026-04-25 14:56:57 +00:00
938af4d6f2
fix(plugs): normalize IPv4-mapped IPv6 peers so cf-connecting-ip wins
Bandit's dual-stack listener delivers IPv4 connections to the app as
IPv4-mapped IPv6 tuples (`::ffff:a.b.c.d`). The trust check compared
that 128-bit form against our IPv4 trusted_proxies CIDRs and never
matched, so cf-connecting-ip was ignored on every cloudflared-relayed
request — every visitor logged as the cloudflared sidecar pod IP
(rendered as `0:0:0:0:0:65535:2804:101` etc.).

Collapses the mapped tuple to plain IPv4 before the trust check and
records it that way on the conn, so logs and session storage see the
real client IP that Cloudflare set in cf-connecting-ip.
2026-04-25 09:54:20 -05:00
FluxCD
b525a4c34e chore: update prop image to git.mcintire.me/graham/prop:main-1777128564-3e458fb [skip ci] 2026-04-25 14:51:55 +00:00
3e458fb76a
feat(nav): add EME menu item to map + weather sidebars
The top navbar in the root layout already linked to /eme, but the
in-page side sidebars on /map and /weather (mobile + desktop) skipped
straight from Path Calculator to Beacons. Adds the EME entry between
them in all four panels so the Earth-Moon-Earth calculator is reachable
without bouncing through the navbar.

Also demotes the LiveStashGuard "stash skipped" warning to debug. It's
the known OTP 28 / LiveStash 0.2.0 ArgumentError that fires on every
event triggering a stash; the guard already handles it, the noise was
adding nothing.
2026-04-25 09:49:06 -05:00
FluxCD
82a83977ca chore: update prop image to git.mcintire.me/graham/prop:main-1777125492-0e0529c [skip ci] 2026-04-25 14:00:28 +00:00
0e0529cd10
feat(weather): ?layer= URL param to deep-link a specific overlay
Adds a `layer` query parameter to `/weather`. Loading
`/weather?layer=temp_700mb` boots straight into that overlay; clicking
a different layer pill push_patches the URL so the active selection is
shareable and survives a refresh. Unknown layer ids fall back to the
default (temperature) instead of crashing.
2026-04-25 08:57:59 -05:00
FluxCD
64d4297a88 chore: update prop image to git.mcintire.me/graham/prop:main-1777125078-f0fa9e0 [skip ci] 2026-04-25 13:53:27 +00:00
f0fa9e0393
fix(propagation): fire grid cron every 15 min to catch fresh HRRR
NOAA publish latency for HRRR pressure-level f18 straddles ~50-65 min
after cycle hour. The HH:05 cron frequently probes a (now-1h) cycle
that's not yet on S3 and falls back to (now-2h). Until today the next
re-attempt was a full hour later, leaving weather visibly stale.

Quadrupling the schedule lets pick_run_time/1 upgrade to the freshest
cycle within 15 min of it publishing. seed_with_analysis is idempotent
(on_conflict: :nothing), so re-fires against an already-seeded run_time
are a single cheap INSERT batch.
2026-04-25 08:51:06 -05:00
FluxCD
5ac8340ba4 chore: update prop image to git.mcintire.me/graham/prop:main-1777079868-2e5b61f [skip ci] 2026-04-25 01:21:33 +00:00
2e5b61f2f9
fix(weather): drop expensive grid fetch from mount to avoid LV join timeout
/weather mount called Weather.latest_weather_grid/1 directly, which
runs WeatherLayers.derive over 92k cells and Jason-encodes the
result. Production HTTP logs showed dead renders consistently
landing at 9.8–10.2 s — right at LiveView's socket join deadline.
The browser would render half the page, the WS would fail to mount,
and the user saw "attempting to reconnect".

The JS hook already fires map_bounds via requestAnimationFrame
immediately after mount, and the existing handle_event("map_bounds")
path returns viewport-scoped weather data via the update_weather
push event. So the dead-render data fetch is redundant — drop it.
Mount now returns initial_data_json: "[]"; the layer paints
~50–200 ms after WS connect via the bounds-driven flow.
2026-04-24 20:17:31 -05:00
FluxCD
46000e7d4d chore: update prop image to git.mcintire.me/graham/prop:main-1777077860-3b25add [skip ci] 2026-04-25 00:46:24 +00:00
3b25add3f2
feat(prop): adaptive HRRR cycle selection — prefer fresh, fall back if late
PropagationGridWorker hard-coded `now-2h` for run_time selection on
the conservative theory that NOAA always finishes publishing within
two hours. In practice f18 is on the mirror by HH:55–HH:00 of the
next hour, so the cron at HH:05 was always picking a cycle that was
already an hour stale.

`HrrrClient.cycle_available?/1` HEADs the wrfprsf18 idx (the slowest
file in the cycle) — a 200 means every earlier forecast hour is also
on disk. `pick_run_time/1` probes `now-1h` first; if the probe
returns true we use that cycle, otherwise we fall back to the
existing `now-2h` slot. Test hook via :hrrr_cycle_available_fn env
keeps the behaviour fully exercisable without hitting NOAA.

End-to-end: weather/score data lag drops from ~2 h to ~1 h on every
hour where NOAA delivers on time. The fallback path keeps the chain
running on slow-publish hours.
2026-04-24 19:44:07 -05:00
FluxCD
41510840fb chore: update prop image to git.mcintire.me/graham/prop:main-1777076889-3ac6963 [skip ci] 2026-04-25 00:34:20 +00:00
3ac6963c74
feat(weather): add 700 mb T/Td and mid-layer lapse rate for cap diagnostics
850 mb T alone shows that warm air is present aloft but doesn't reveal
whether it's the textbook cap structure — a warm 700 mb above an EML
plume. The three new layers expose the canonical capping diagnostics
visually:

  * T @ 700 mb — ≥10 °C is the southern Plains "moderate cap" threshold
  * Td @ 700 mb — wide T-Td depression at 700 mb signals the EML plume
  * Lapse 850→700 — steep mid-layer lapse rate (≥7 °C/km) over a moist
    boundary layer is the cap mechanism itself

All three derive purely from the existing pressure-level profile (Rust
already fetches up through 700 mb), so no Rust pipeline changes needed.
2026-04-24 19:27:23 -05:00
FluxCD
f6ddf38071 chore: update prop image to git.mcintire.me/graham/prop:main-1777074592-4f82bd6 [skip ci] 2026-04-24 23:52:47 +00:00
4f82bd691e
perf(prop): cap ScoreCache LRU + drop eager warmers to fix hot-pod OOMs
ScoreCache held 437 entries × ~1.94 MiB each (~850 MiB per pod) of
{band_mhz, valid_time} grids — data already on NFS as compact .prop
files. NotifyListener and ScoreCacheReconciler both eagerly materialised
every band into ETS on each Rust completion / 60s sweep, so 5 hot
replicas wasted ~4.2 GiB of redundant cache and OOMed at 6 GiB.

Bound the cache to 32 entries with eviction by oldest valid_time, drop
the eager warm loops, and let LiveView callers lazy-fill via the
existing ScoresFile read path. ScoreCacheReconciler had no remaining
purpose and is deleted; runbook + prom_ex counters updated to match.

Steady-state cache footprint ~60 MiB per pod instead of ~850 MiB.
2026-04-24 18:49:36 -05:00
FluxCD
2db8717d37 chore: update prop image to git.mcintire.me/graham/prop:main-1777066814-e192ca3 [skip ci] 2026-04-24 21:43:10 +00:00
e192ca3300
fix(weather): analyze_all skip_recent check reads both analyze timestamps
pg_stat_user_tables tracks two timestamps — `last_autoanalyze`
is only bumped by the autovacuum daemon, while `last_analyze` is
bumped by manual ANALYZE statements (including the ones our Task
issues). The previous query only filtered on `last_autoanalyze`,
so on a re-run within the skip_recent window every manually-
analyzed table came back for another ANALYZE pass.

Uses `GREATEST(last_analyze, last_autoanalyze)` so fresh manual
analyses count as recent.
2026-04-24 16:40:02 -05:00
FluxCD
36049808a4 chore: update prop image to git.mcintire.me/graham/prop:main-1777065178-560be6c [skip ci] 2026-04-24 21:14:57 +00:00
560be6ce32
fix(weather): analyze_all is now fire-and-forget
Running ANALYZE serially against the 13M-row HRRR monthly partitions
took longer than the kubectl exec idle timeout (~300s per partition
on the Turing Pi 2), so the one-shot RPC kept getting killed before
the big partitions ran. Moves the loop into a supervised Task on
the caller's node so the RPC returns immediately; the BEAM keeps
running ANALYZE regardless of kubectl connection state.

Also sorts tables ascending by n_live_tup so small tables finish
first and their fresh stats are visible to the planner right away.
Progress is logged per table via `Logger.info`.
2026-04-24 16:12:42 -05:00
FluxCD
d6c6d1548d chore: update prop image to git.mcintire.me/graham/prop:main-1777063737-03c91f4 [skip ci] 2026-04-24 20:51:32 +00:00
03c91f4098
perf(db): tune autovacuum per-table + one-shot ANALYZE helper
Two issues spotted in pg_stat_user_tables:

1. Most `hrrr_profiles` partitions have `last_autoanalyze IS NULL`.
   The partitions are append-only from the hourly grid worker, so
   autovacuum never crosses the analyze threshold — but without
   stats the planner falls back to estimates and picks bad joins
   on per-QSO lookups.

2. `contacts`, `grid_tasks`, `hrrr_fetch_tasks`, and
   `contact_common_volume_radar` see high UPDATE volumes
   (enrichment status flips, queue claim/complete cycles, mechanism
   classifier writes) but only accrue ~5% dead before the default
   20% / 10% autovacuum triggers fire. BackfillEnqueueWorker's
   status-priority ORDER BY + the Rust workers' FOR UPDATE SKIP
   LOCKED claim scan both want fresher stats than that.

Changes:
- Migration `20260424204656_tune_autovacuum_for_high_churn` sets
  per-table `autovacuum_*_scale_factor` + `_threshold = 50` on the
  four tables above. Aggressive enough to keep stats current;
  tame enough that vacuum doesn't thrash the Turing Pi 2 Postgres
  node. Reversible.
- `Weather.analyze_all/1` walks `pg_stat_user_tables`, runs
  `ANALYZE` on every table not auto-analyzed in the last 6 h,
  reports counts. Meant to be invoked once via
  `bin/microwaveprop rpc 'Microwaveprop.Weather.analyze_all()'`
  after deploy to seed statistics on the cold partitions;
  idempotent on re-run.
2026-04-24 15:48:46 -05:00
FluxCD
609f04f649 chore: update prop image to git.mcintire.me/graham/prop:main-1777063515-13fad93 [skip ci] 2026-04-24 20:47:29 +00:00
13fad936a3
fix(status): mobile layout — tables no longer truncate column headers
The three tables on /status (Job Queue, Rust Workers, Table Sizes)
were collapsing to fit the viewport on narrow phones, wrapping
header text ("Siz…") and splitting values like "160 GB" into two
lines ("160." / "GB"). The overflow-x-auto wrappers were already
present but column auto-sizing still squeezed width below content.

Adds `whitespace-nowrap` to each `<table>` so cells never wrap, and
`-mx-2 px-2` on the overflow wrapper so horizontal scroll extends
edge-to-edge of the mobile viewport instead of being inset by the
layout's outer padding. Tables now scroll horizontally as intended
on mobile; desktop layout unchanged.
2026-04-24 15:45:01 -05:00
FluxCD
b81b482119 chore: update prop image to git.mcintire.me/graham/prop:main-1777063066-ea69712 [skip ci] 2026-04-24 20:40:25 +00:00