Commit graph

567 commits

Author SHA1 Message Date
6f63f3bb5e
feat(admin): surface flagged contacts on the contact-edits page
Flagged contacts were only reachable from the contact detail page;
once the Flag column came off the /contacts index there was no
at-a-glance view of what's been flagged. Add a second table on the
admin contact-edits page (below the pending-edits table) that lists
every flagged contact newest-first with a direct View link. Hidden
entirely when nothing is flagged.
2026-04-19 12:40:20 -05:00
a6a3d3ced3
chore(contacts): drop the Flag column from the table
The `!` flag column was a tiny sliver that rarely fired in practice
and was pushing more useful columns offscreen on narrow layouts. The
underlying `flagged_invalid` field stays on the schema for the detail
page; only the index column goes.
2026-04-19 12:36:52 -05:00
4b155c5419
feat(contacts): hide Private column when the viewer has nothing private
Row-level privacy was already correct: private rows only render for
the owner and admins. The header column was still showing for every
viewer, leaving an always-empty slot for anyone without private
contacts of their own. Drop `:private` from the fields list unless
the scope can actually see at least one private row.
2026-04-19 12:36:12 -05:00
ffb14cb64f
feat(map): auto-advance cursor, full forecast timeline, panel spacing
Three UI fixes:

1. MapLive now ticks every 60s and, when the selected_time is still
   the "Now" slot, advances to the newer latest-past-or-now hour as
   the clock crosses into it. The URL is patched in place so shared
   links stay accurate without polluting history. Manually picking a
   specific past/future hour turns tracking off.

2. point_forecast/3 now reads its valid_times list straight from the
   on-disk .ntms files (same source the main-map timeline uses), then
   consults the cache per-hour for a fast score lookup. The old cache-
   or-store branch could leave the click-to-detail sparkline at 13
   hours while the bottom timeline showed 14+; both now line up.

3. Point-detail panel moves from bottom-14 → bottom-24 on desktop so
   its lower edge clears the forecast-timeline bar (+ max-height tuned
   to match), fixing the overlap visible in screenshots.
2026-04-19 12:10:36 -05:00
01b181b1e8
perf(propagation): shrink hourly chain wall time
Four changes sized by measured prod telemetry (83m of spans):

1. propagation queue: 2 → 1 slot per pod. Two concurrent forecast-hour
   steps per pod stacked HRRR grid + native duct grid + scored band
   map into ~5-6 GiB RSS, OOM-killing every ~15 min. 3-way parallelism
   cluster-wide still finishes the chain inside the hourly interval.

2. weather queue: 3 → 1 slot per pod. ASOS backfill was 429-thrashing
   IEM (1,296 retryable jobs; logs were nothing but 429 backoffs).

3. PropagationGridWorker: skip native-level duct fetch on f01..f18.
   At ~7-11 min/fh and 18 forecast hours, this was the largest single
   cost per chain. Forecast hours fall back to
   derived[:min_refractivity_gradient] from the pressure-level
   profile. f00 still gets full native-level duct analysis.

4. HrrrClient.download_grib_ranges_to_file: parallelize with
   Task.async_stream (max_concurrency 8). The file-backed variant was
   sequential, dominating native-duct fetch time on the remaining f00
   path. ~20s → ~3s per call.
2026-04-19 12:01:41 -05:00
0d022392fe
fix(propagation): use map_size on grid_data telemetry count
The :score_band span was reading point_count via length(grid_data),
but grid_data is a %{{lat, lon} => profile} map. Every forecast-hour
job was crashing with ArgumentError before any scores landed, leaving
the map without current-hour forecasts. Swap to map_size/1.
2026-04-19 10:26:46 -05:00
f122eedfa8
perf(propagation): parallel chain fan-out + hoist shared factors
Two wins on the hourly propagation pipeline:

1. Parallelize the chain. seed_chain was enqueuing only f00 and
   each step self-enqueued f00+1, serializing ~2.5 min × 19
   forecast hours into a ~48 min chain. Fan out all 19 jobs at
   once — they're genuinely independent (different HRRR URLs,
   different output files) — and let queue concurrency (2 slots
   × 3 pods = 6 parallel workers) drop wall time to ~10 min.

   Side effect: one permanently-failing step no longer takes out
   the rest of the chain, so the rescue-chain logic I added last
   commit becomes unnecessary. Removed.

   The :final cleanup (retain_window + purge) used to run on the
   fh=18 transition; with parallel execution we don't know which
   step finishes last. Cleanup now relies on the existing
   PropagationPruneWorker 15-min cron for time-based pruning.
   Stale chain leftovers are already a minor concern with the
   15-min prune; if file bloat becomes real, PruneWorker can
   gain retain_window later.

2. Hoist band-invariant factors. score_time_of_day, score_sky,
   score_wind, score_pressure depend on conditions only, not the
   band — but composite_score was recomputing them 17 times per
   point. Added Scorer.precompute_band_invariants/1, called once
   per point before the bands loop; composite_score uses the
   cached values when present and falls back to computing for
   any caller that doesn't pre-warm (test harness, path
   integrator). Saves ~30% of the scoring inner loop.
2026-04-19 09:39:06 -05:00
5cfb9e6c8e
fix(propagation): chain survives permanent step failures
Telemetry showed ~66 PropagationGridWorker exceptions per 6h with
55 ArgumentErrors and 11 TimeoutErrors, producing ~13 discarded
chain steps. Each discard broke the chain: subsequent forecast
hours were never enqueued, leaving the score store with huge gaps
(e.g. at 14:11 UTC the earliest available forecast was 18:00,
because f00-f05 all failed somewhere upstream and nothing ran
after them).

Three changes:

1. PropagationGridWorker: on the final attempt, still enqueue
   fh+1 even when this step failed. Oban discards the current
   job normally — but the rest of the chain keeps running, so
   one bad hour doesn't take out the remaining 12-18. The
   rescue is factored into a tested public helper.

2. HrrrClient.parse_idx: skip malformed idx lines instead of
   raising. NOAA S3 occasionally serves an HTML error page as
   the idx body, and the old strict String.to_integer path
   raised ArgumentError on the first non-numeric line and took
   down the chain step. This is the root cause of the 55
   ArgumentErrors.

3. JS renderTimeline: when no forecast hour is at-or-before
   wall-clock (all times are future — the gap scenario the
   fixes above are designed to prevent), stop labeling the
   earliest future slot "Now". Lets the user see honest
   "+Nh" offsets instead of a lie on the pill.
2026-04-19 09:26:20 -05:00
bbf6481603
perf(nexrad): route fetch_decoded_frame through NexradCache
Telemetry puts nexrad_decode_png at 1.76 s/call × 37,847 calls/h
— ~18.5 h of CPU across the cluster every real hour, the single
biggest CPU consumer in the system. The map-click path
(fetch_rain_cells) already cached decoded frames by 5-min
bucket, but the per-contact CommonVolumeRadarWorker path
(fetch_decoded_frame) went straight to network + decode on every
call. Backfill means many contacts share a 5-min window, so the
same 66 MB frame was being decoded dozens of times.

Wire fetch_decoded_frame through NexradCache keyed on the
rounded timestamp. Add a 20-entry size cap in NexradCache so
backfill processing contacts in random timestamp order can't
grow the ETS table to hundreds of GB. Each frame is ~66 MB, so
20 = ~1.3 GB worst case, well under typical pod memory.

Expected impact: cuts sustained decode load by an order of
magnitude depending on backfill temporal locality; map-click
path is unchanged.
2026-04-19 09:05:55 -05:00
bb6333bd27
fix(map): recover with sane defaults on garbage URL params
Every parser already returned nil on malformed input, so the ||
chains fell through to defaults — except parse_time_param, which
happily accepted any well-formed ISO8601 even if it pointed
outside the current HRRR forecast window (e.g. a stale shared
link from last week). Gate it behind the valid-times window and
fall back to the normal 'now' cursor when it's out of range.

Adds explicit coverage: non-numeric band, unknown band MHz,
non-numeric lat/lon/zoom, out-of-range values, malformed time,
ancient time, and all-params-garbage-at-once. Each case mounts
cleanly and lands on the defaults.
2026-04-19 08:55:12 -05:00
160439db31
feat(map): shareable URL params for view/band/time
The main map page now reads band, selected time, and map
center/zoom from URL params and updates the URL as the user pans,
zooms, picks a band, or scrubs the timeline. Pan/zoom use
push_patch replace-mode so browser history isn't spammed, while
band and time changes push fresh entries.

Also fix the 'Up to date · just now ago' pipeline chip — 'just
now' already reads as an instant so the trailing 'ago' is
redundant. Any older duration keeps it.
2026-04-19 08:53:07 -05:00
504f5147f6
fix(propagation): hourly grid chain preempts MRMS/prune backlog
The :propagation queue (2 slots) is shared by PropagationGridWorker,
MrmsFetchWorker (every 2 min), and PropagationPruneWorker (every
15 min). Oban dispatches priority-then-FIFO, so a post-deploy
MRMS/prune backlog could delay the hourly chain step until the
siblings drained.

Set PropagationGridWorker priority: 0 explicitly and drop the two
siblings to priority: 5 so new chain steps always jump ahead.
Backfill workers live on separate queues (:hrrr, :weather, :terrain,
:iemre, :narr, :radar, :mechanism) and already don't interact with
the predictor.
2026-04-19 08:42:22 -05:00
70e65ba034
refactor(map): drop the 7-day outlook strip, fix missing Analysis, trim timeline
- Remove the 7-day outlook strip and daily_outlook_at/3. It was
  derived from point_forecast, which is capped at HRRR's 18-hour
  horizon, so it never had 7 days of data to show.
- factors_for: fall back to the nearest persisted analysis profile
  at or before the requested time. Only f00 hours persist profiles,
  so clicking any other hour used to yield an empty Analysis and
  factor table. After the recent cursor fix lands 'Now' on a
  forecast hour more often, this regressed further.
- available_valid_times/1: cap the forward horizon to 18h from now.
  Leftover valid_times from prior cycles used to pile on the
  timeline without adding information.
2026-04-19 08:17:47 -05:00
28a9eb3a92
fix(map-timeline): never label a future forecast hour as 'Now'
Both the Elixir initial-cursor pick and the JS timeline renderer
used nearest-by-absolute-distance, so any wall-clock time past
:30 snapped the 'Now' label onto the next top-of-hour forecast —
up to ~30 min in the future. Switch to the latest valid_time at
or before wall-clock, with a fallback to the earliest time when
every slot is future (shouldn't happen for HRRR, safe default).
2026-04-19 08:09:14 -05:00
6c652ef2d4
feat(contacts): private flag with scope-aware visibility
Contacts can be marked private at submit time (single form, CSV
import, ADIF import) and edit time. Private contacts are hidden
from the public map, the public /u/callsign profile, and the
browse table for anonymous and non-owning viewers. The original
submitter and admins see them inline on the browse table with a
"Yes" badge, and the detail page shows a lock icon. Non-authorized
viewers get a 404 on the detail page.
2026-04-18 17:49:01 -05:00
4e6c87eca2
feat(telemetry): broaden Instrument span coverage
Adds spans to 15 previously-unmeasured hot paths so every question we
might ask while tuning has a histogram to answer it:

External I/O:
- iem.fetch_iemre (gridded weather reanalysis)
- mrms.list_latest / mrms.download (precip radar)
- rtma.fetch_observation
- ncei.fetch_metar (historical 5-min METAR backfill)
- solar.fetch_indices (GFZ solar indices)
- swpc.fetch (SWPC Kp/F10.7/X-ray)
- giro.fetch (ionosonde)
- qrz.request, geocoder.geocode (callsign enrichment)
- srtm.download_tile (terrain tile download + gunzip)
- hrrr.download_grib_ranges (parallel byte-range fetch phase)

Subprocess:
- wgrib2.extract_grid / extract_grid_from_file / extract_grid_from_file_mapped

LiveView hot paths:
- propagation.scores_at (map score fetch + cache hit/miss counter)
- propagation.point_forecast (sparkline)
- propagation.point_detail (click-to-inspect)
- propagation.daily_outlook_at (/map outlook strip)

Worker-level end-to-end:
- worker.terrain_profile
- worker.mechanism_classify
- worker.mrms_fetch

Each event is registered in Microwaveprop.PromEx.InstrumentPlugin as
a Prometheus histogram (default / long buckets as appropriate) plus
a counter for the scores_at cache hit/miss ratio. Prometheus at
10.0.15.25 will start seeing the new series on the next scrape after
deploy.
2026-04-18 17:25:33 -05:00
9af50a9e34
fix(metrics): serve scrape body via PromEx.get_metrics
PromEx.Plug's internal path check doesn't match when forwarded under
/metrics (conn.path_info is [] after the forward strips the prefix),
so it silently returned without sending a response. Call
PromEx.get_metrics/1 directly and send the text body ourselves —
tested, returns 200 with all 23 registered event handlers producing
proper Prometheus histogram output.
2026-04-18 16:50:37 -05:00
c7bc3ed5d0
feat(telemetry): PromEx Prometheus exporter at /metrics
Wires PromEx with its built-in Application / BEAM / Phoenix / Ecto /
Oban plugins plus a custom `Microwaveprop.PromEx.InstrumentPlugin`
that registers Prometheus histograms for every
`Microwaveprop.Instrument` span (HRRR/NEXRAD/IEM/GEFS/NARR/UWYO
/Elevation fetches, DB batch upserts, terrain analysis, radar
aggregation, propagation score band). Queue-depth gauges from the
10-second poller land at `microwaveprop_oban_queue_count{queue,state}`.

Router forwards `/metrics` to MicrowavepropWeb.MetricsPlug which
optionally requires a bearer token (`PROMETHEUS_AUTH_TOKEN` env var in
runtime.exs); leave it unset to expose metrics publicly and restrict
at the ingress / Cloudflare Access layer instead.

Example external Prometheus scrape config:

    scrape_configs:
      - job_name: microwaveprop
        scrape_interval: 30s
        metrics_path: /metrics
        scheme: https
        authorization:
          type: Bearer
          credentials: <token>
        static_configs:
          - targets: ['prop.w5isp.com:443']
2026-04-18 16:39:39 -05:00
2da74c5cd8
feat(telemetry): wide instrumentation + bump hrrr to 2 per pod
Config:
- runtime.exs hrrr queue 1 → 2 (6 concurrent HRRR jobs across 3 pods)

New helper Microwaveprop.Instrument.span/3 wraps :telemetry.span with
a [:microwaveprop | event_suffix] prefix so metrics can key off it.

Spans added (each emits duration + result tag):
- HrrrClient.fetch_grid / fetch_profile / fetch_idx
- NexradClient.fetch_frame + decode_png
- IemClient.fetch_asos / fetch_raob
- GefsClient.fetch_grid_profiles
- NarrClient.fetch_profile_at
- UwyoSoundingClient.fetch_sounding
- ElevationClient.fetch_elevation_profile
- Weather.upsert_hrrr_profiles_batch / upsert_gefs_profiles_batch
- Propagation.replace_scores
- PropagationGridWorker.compute_scores_algorithm
- TerrainAnalysis.analyse
- CommonVolumeRadarWorker.aggregate_stats

Telemetry catalog (MicrowavepropWeb.Telemetry):
- Oban job duration / queue_time / count / exception by (worker, queue, state)
- Per-span summary metrics for every instrumented phase above
- Periodic (10s) poller emits oban queue depth by (queue, state) — drops
  into the /admin/dashboard Metrics tab immediately

Also drops the now-redundant "fetching n0q frame" and "fetching <url>"
info lines from CommonVolumeRadarWorker / NexradClient; the span events
cover that and the worker's "ingested" line stays for per-job signal.
2026-04-18 16:33:34 -05:00
1a62e51f82
chore(radar): drop per-fetch ok-bytes log line 2026-04-18 16:22:04 -05:00
ced5e3d1a6
feat(radar): log NEXRAD fetches + worker progress
Previously the CommonVolumeRadarWorker ran silently — no URL logged,
no sign in the logs that it was doing any work. Added info-level
fetch/ok/error logs in NexradClient (same shape as WeatherFetchWorker
RAOB logs) plus start-of-work + ingestion lines in the worker itself.
Unchanged: the existing "no frame" warning and all control flow.
2026-04-18 16:20:59 -05:00
af6c676df5
chore(branding): rename site to "Microwave Propagation"
Nav bar and browser tab now show "Microwave Propagation" instead of
"NTMS Propagation Prediction". The About and Privacy pages still
reference NTMS since those describe the organization running the
service, which is unchanged.
2026-04-18 16:18:21 -05:00
be62e272e9
feat(contacts): widen sounding search, backfill on demand
Contact detail now searches for soundings in progressively wider radii
(150 → 300 → 600 → 1000 km) before declaring "No soundings found".

When even 1000 km turns up nothing, enqueue WeatherFetchWorker RAOB
jobs for every station at the widest non-empty radius. The worker
passes the contact_id through so it can PubSub back to
contact_enrichment:<id> as each sounding lands, and the LiveView
rehydrates without a refresh.

Also adds Weather.soundings_with_widening_radius/1 and
ContactWeatherEnqueueWorker.enqueue_raob_fetch_with_widening/1 so the
same logic can be reused from other callers.
2026-04-18 16:04:50 -05:00
a9a355be2a
feat(format): miles-primary distance formatting sitewide
Add Microwaveprop.Format.distance_km/1 and a matching formatDistanceKm
helper for the JS side. Both emit "X mi (Y km)" with one decimal under
10 mi and whole numbers above.

Replace the ad-hoc format_dist/format_km_mi helpers with the shared
formatter and update every user-facing distance render: contact
detail, contacts index column, user profile contact/involving tables,
path finder summary + sounding list, contacts map line popups, beacon
coverage tooltip, and propagation map range estimate + rain scatter
cell popup.
2026-04-18 15:53:40 -05:00
ca07c0288d
feat(contacts): antenna height fields feed terrain analysis
Add optional height1_ft/height2_ft to contact schema, submit form, and
edit form (direct + suggest-edit paths). Heights flow through to the
elevation-profile terrain analysis so clearance and diffraction are
computed from the actual antenna height instead of a 10-ft default.
Editing heights resets terrain_status and re-enqueues
TerrainProfileWorker so the stored path analysis picks up the new
geometry.

Also fix mark_likely_duct/3: the pattern was grabbing the element
instead of the index, so no duct ever got flagged as the likely
propagation path even when extract_ducts returned a valid layer.
2026-04-18 15:48:14 -05:00
a6af6b115a
feat(map): 7-day outlook strip
Adds a compact seven-card horizontal strip to both mobile and desktop
map sidebars showing the per-day peak score at the viewport center
for the selected band. Cards are color-coded (emerald for 80+,
rose for bad) so the weekend verdict reads at a glance.

- Propagation.daily_outlook_at/3: groups point_forecast entries by
  UTC date, picks each day's peak, returns ascending order
- MapLive mounts with today's outlook for the initial center; the
  select_band handler refreshes it from the viewport midpoint so the
  strip tracks the user's current band and rough location
- Empty-state message covers the case where no extended-horizon
  scores have landed yet (fresh deploy, or before the first GEFS
  run completes)

This is the consumer of the GEFS pipeline — once the cron starts
running and f024-f168 scores accumulate, days 2-7 of the strip
populate automatically.
2026-04-18 14:46:44 -05:00
4d0c15e3b8
feat(gefs): score extended-horizon grid and seed cron
Wires the GEFS fetch pipeline into the existing scoring machinery:

- GefsFetchWorker runs Propagation.score_grid_point over every
  fetched grid cell and persists the result via replace_scores/2, so
  Day 2-7 valid_times show up on the map through the same
  Propagation.scores_at/3 path HRRR uses
- Empty-args perform/1 seeds the chain by enqueueing f024-f168 (25
  jobs at 6-hour cadence) for the most recent GEFS run that should
  be published given NOMADS' ~3-4h publication lag
- Cron: seeds at 05:30, 11:30, 17:30, 23:30 UTC — 5 hours after each
  00/06/12/18Z run
- GefsClient.build_profile now emits :wind_u / :wind_v atom keys to
  match the scorer's input shape; DB columns keep the *_mps suffix
  for unit clarity, remapped at write time

GEFS pgrb2a lacks HPBL, native-level duct data, NEXRAD, and
commercial-link degradation — the extended-horizon score is a
rougher signal than the HRRR-driven one but covers the window HRRR
can't reach.
2026-04-18 14:41:41 -05:00
0537a1831b
feat(gefs): ingest worker and grid-profile extraction
- GefsClient.build_profile/1: converts raw wgrib2 output into the
  scorer-shaped profile, deriving Td from RH via Magnus at both the
  surface and each pressure level
- GefsClient.fetch_grid_profiles/3: downloads the idx sidecar from
  NOMADS, byte-ranges only the surface messages, hands the slim
  GRIB2 binary to wgrib2 -lola for bilinear regridding from 0.5°
  onto the CONUS 0.125° grid
- GefsFetchWorker: one Oban job per (run_time, forecast_hour),
  upserts into gefs_profiles and runs SoundingParams.derive so the
  same refractivity/ducting metrics flow through

Oban gets a new :gefs queue (2 slots dev/config, 1 slot per prod
pod). Test env gets a gefs_req_options Req.Test stub slot so future
integration tests can drive the full fetch path. No cron wiring or
scorer integration yet — that's Step 3.
2026-04-18 14:36:03 -05:00
a5c7dce147
feat(gefs): scaffold extended-horizon forecast ingestion
Lays the groundwork for a Day 2-7 propagation outlook driven by NCEP
GEFS ensemble-mean output, picking up where HRRR's 18-hour horizon
leaves off.

- GefsClient: NOMADS URL builder, forecast-hour list (3h to +240,
  6h to +384), ensemble-mean message inventory, and a Magnus
  dewpoint derivation (pgrb2a publishes RH rather than Td)
- gefs_profiles table + GefsProfile schema mirroring hrrr_profiles
  so the scorer can run over rows from either source
- Weather.upsert_gefs_profile/1 and upsert_gefs_profiles_batch/1

No worker or UI yet — those land in follow-up commits.
2026-04-18 14:28:44 -05:00
cf6812f154
fix(contact): stop loading-flash on already-enriched contacts
Two related fixes:

- Drop the global contact_enrichment:{hrrr,weather,solar} subscriptions.
  These topics fire for every QSO's enrichment completion, and the
  handlers blindly re-queried data for the currently-displayed contact,
  causing spurious re-renders whenever any enrichment worker finished
  anywhere in the system. The per-contact terrain subscription stays —
  that one is correctly scoped.

- Only add a slot to `hydration_pending` when the contact's status
  column says the work is genuinely in flight (:queued or :processing).
  Previously every connected mount flashed spinners over all eight
  enrichment sections, even for contacts whose data was already cached
  and about to return instantly from the async task.

Also drop the Donate-to-NTMS footer link.
2026-04-18 13:56:01 -05:00
e3248e335b
revert(map): remove antenna-height control
The antenna-height input only scaled the displayed range labels and
the viewshed max-range — it never fed into the propagation score
itself. Showing the control implied the colored overlay responded to
it, which was misleading. Viewshed now uses a fixed 10 m baseline,
matching the previous 33 ft default.
2026-04-18 13:50:56 -05:00
5488744f44
feat(contact): loading spinners, callsign-aware summaries, axis callsigns
Three improvements to the contact detail page:

* Track async hydration per source in `hydration_pending` so the
  Terrain / Soundings / Atmospheric Profile sections render a loading
  spinner while their task is in-flight instead of briefly flashing
  "no data available" before the payload arrives.
* Propagation analysis summary now names the origin station by
  callsign ("Path is terrain-obstructed at N mi from W5XD") instead
  of the generic "station 1" placeholder.
* Elevation profile chart labels the 0-mi and max-mi ticks with the
  matching callsign so it's obvious which end of the profile is
  which station.
2026-04-18 13:43:35 -05:00
7330d64c5a
feat(contacts): surface tropo duct count and suggest-edit hint
Propagation analysis panel now adds a "Tropo duct detected at N/M HRRR
samples along the path" line whenever any HRRR sample flags ducting,
even if the overall mechanism classification doesn't end up pointing at
the duct (e.g. short paths, clear-LoS contacts). Also rewords the map
caveat to invite users with better coordinates to file a suggested
edit — clickable when logged in, otherwise a login link.
2026-04-18 13:37:54 -05:00
e827cacc48
feat(map): scale propagation ranges with antenna height
Antenna height now drives typical/extended/exceptional range estimates
and viewshed max range via a sqrt(h_m/10) factor clamped to [0.6, 2.0].
Baseline 33 ft matches the existing defaults. Changing the height input
now pushes update_band_info and refreshes the clicked point's detail
panel plus viewshed so the range-estimate line and coverage shape track
the user's actual station height.
2026-04-18 13:32:24 -05:00
32ce43f04a
feat(map): show last-deployed stamp in map sidebar
Deploy stamp was in Layouts.app's nav but the map page uses a custom
full-screen layout that skips the standard nav, so the info wasn't
visible on the page most users land on. Add a compact
"Deployed 2h ago" under the sidebar's data-timestamp/pipeline-status
block with the full UTC time as a tooltip, matching the shape of
Layouts.deploy_stamp.
2026-04-18 12:53:37 -05:00
d3b0420457
feat(scoring): Richardson-gated native-duct boost
Added 5-arity Scorer.score_refractivity/5 that takes bulk_richardson
alongside best_duct_band_ghz. The existing 1.15× boost now only
applies when Richardson is in the stable regime (< 25) — native-
profile duct cells average 8.7-18.4 for ducting vs 38.3 for
non-ducting, so a duct-band reading under turbulent conditions is
more likely to be torn up by mechanical mixing than to carry a
beyond-LOS signal.

Wiring:
- Weather.nearest_native_duct_info/3 returns {ghz, richardson}; the
  bare nearest_native_duct_ghz/3 now delegates.
- PathLive and Propagation.score_with_algorithm/7 both pull the
  Richardson value into the conditions map alongside best_duct_band.
- Scorer.composite_score/2 reads conditions[:bulk_richardson] and
  passes it into score_refractivity/5.
- Backward compat: 4-arity variant and nil Richardson keep the old
  unconditional-boost behaviour so existing callers don't regress.
2026-04-18 12:40:57 -05:00
06feda7110
perf: finish audit follow-ups (path conditions, recalibrator parallelism)
Path calculator conditions map was missing :latitude and
:best_duct_band_ghz. Scorer read them with bracket-notation so the
behaviour was defensively correct but we were leaving signal on the
floor:

- latitude: midpoint of src/dst so `score_season` picks up the right
  latitude-based seasonal shift for northern vs southern paths
- best_duct_band_ghz: queried via new Weather.nearest_native_duct_ghz/3
  at the midpoint so score_refractivity/4 applies the 1.15× boost
  when a native-resolution duct supports the target band

Recalibrator.fit/1 was serialising ~10k per-contact HRRR lookups
(5000 positives + 5000 negatives, one Weather.find_nearest_hrrr each).
Parallelised both sides with Task.async_stream at 20 concurrency —
well under the prod db pool of 30. Cuts a band recalibration from
~minutes of query round-trips to seconds.
2026-04-18 12:23:18 -05:00
2874e6a93b
perf+fix: parallelize path HRRR queries; fix meteor-shower year boundary
Audit pass found two real issues (most agent-flagged items were false
positives — the hrrr_native_profiles composite index exists,
pwat_mm/bl_depth_m are already in the path conditions map, and the
/1.0 in MechanismClassifier is the standard Elixir int→float coerce).

1. Path calculator's 9-point HRRR sampling was running sequentially,
   ~9× the latency of a single query. Task.async_stream with
   max_concurrency: 9 makes them concurrent so path calc pays roughly
   one query's worth of time instead of nine.

2. MechanismClassifyWorker's meteor-shower window used hardcoded 2024
   peak dates with `%{date | year: peak.year}` projection. That broke
   across year boundaries — a Dec 30 contact couldn't match a Jan 4
   Quadrantids peak even in-range of the ±3 day window. Switched to
   {month, day} tuples and check the peak projected onto year N-1, N,
   N+1 so the window works correctly near Dec 31 / Jan 1.
2026-04-18 12:08:42 -05:00
5dea91d43f
feat(beacons): plot-path link seeds TX chain to match beacon EIRP
The "Plot path to beacon" action now includes `tx_power_dbm` and
`src_gain_dbi` in the /path query string so the path calculator's TX
side emits the same EIRP as the beacon out of the box. The beacon
schema stores `power_mw` as EIRP already (per Beacons.RangeEstimate),
so we take 10·log10(power_mw), split against a default 30 dBi dish,
and clamp the TX power to a 0 dBm floor so very-low-EIRP beacons
produce a sane gain/power split instead of -10 dBm outputs.

Examples:
  10 W EIRP (40 dBm)  → 30 dBi dish + 10 dBm TX
  100 W EIRP (50 dBm) → 30 dBi dish + 20 dBm TX
  100 mW EIRP (20 dBm) → 20 dBi + 0 dBm (clamped)
2026-04-18 10:54:38 -05:00
fbec454917
feat(path): 9-point HRRR sampling + nearest sounding + km/mi display
Path calculator now samples nine HRRR grid cells evenly along the
source→destination line (was three: Source/Midpoint/Destination) so
mid-path ducts that endpoint-only queries missed show up in the duct
count. The UI reports "Duct at N / 9 HRRR points" instead of a binary
flag, and the table lists every sample with its percent-of-path label.

Independent RAOB signal: uses the new `Weather.nearest_sounding_to/3`
to find the closest sounding within 300 km / ±3 h of the midpoint. If
the sounding's `ducting_detected` is true we surface a "RAOB <code>
confirms duct" badge — HRRR under-reads thin surface ducts that
soundings resolve, so the two signals together are the right
cross-check for live duct prediction.

Distance formatting: all km values on the path results (top Distance
summary, sounding "km from midpoint") now render as
`123 km (76 mi)`; under 10 km we keep one decimal so near-LOS paths
don't round to zero.
2026-04-18 10:50:24 -05:00
18d16b0ad6
feat(weather): nearest_sounding_to helper for path calculator
Spatial + temporal nearest-RAOB lookup (defaults to 300 km / ±3 h)
joining weather_stations → soundings. Returns the full sounding row
so callers can read the derived ducting_detected / min_refractivity_
gradient fields that HRRR's pressure-level product systematically
under-reads for thin surface ducts.

Prep for the path-calculator enrichment: a sounding within range at
the path midpoint gives an independent duct signal to cross-check
the nine HRRR samples already taken along the path.
2026-04-18 10:46:35 -05:00
622edee180
feat(propagation): per-contact mechanism classification
Classifies every contact's likely non-LOS propagation mechanism and
persists the result on contacts.propagation_mechanism. Mechanism is
determined in priority order:

  1. user_declared_prop_mode (ADIF PROP_MODE from the operator log)
  2. EME — moon-ephemeris check, ≥2m band, >1800 km path
  3. aurora — Kp≥5 + high-lat path, 50-432 MHz
  4. sporadic-E — foEs × 5 ≥ band_mhz, 400-2500 km path
  5. meteor_scatter — ±3 days of a shower peak, VHF/UHF
  6. rain_scatter — common-volume radar heavy rain, 5-11 GHz ≤800 km
  7. tropo_duct — HRRR native_best_duct ≥ band or ducting_detected
  8. line_of_sight — ≤50 km path
  9. troposcatter — default

Persisted via MechanismClassifyWorker (queue: :mechanism, unique on
contact_id). Submit-time enqueue path includes :mechanism by default;
BackfillEnqueueWorker cron now handles :mechanism alongside existing
types so prod continuously backfills any contact with
propagation_mechanism_status in (:pending, :queued, :failed). Also
added :radar to the cron's type list so common-volume radar backfill
runs automatically rather than only via `mix radar_backfill`.

New modules:
- Microwaveprop.Propagation.MoonEphemeris — Meeus low-precision moon
  position, accuracy ±1° — enough for the mutual-visibility EME test
- Microwaveprop.Propagation.MechanismClassifier — plug-in priority
  chain over the evidence map
- Microwaveprop.Workers.MechanismClassifyWorker — assembles inputs
  from HRRR / native profiles / common-volume radar / solar_indices /
  ionosonde + calls the classifier

ADIF importer now reads PROP_MODE into user_declared_prop_mode so
operator-tagged mechanisms (EME/ES/MS/RS/AS/AUR) become ground truth.
2026-04-18 10:42:08 -05:00
f9af347485
chore(logs): skip HEAD requests in endpoint telemetry log 2026-04-18 10:22:48 -05:00
5fd37a17fd
feat(propagation): per-band weight calibration from full-corpus correlation
Ran recalibrate_algo.py against the full local prop_dev (81,994 contacts,
18.6M HRRR rows) and derived per-band composite weights for the nine
bands with >=200 matched contacts. Moisture (dewpoint/PWAT/surface N)
is consistently beneficial through 5.76 GHz, reverses at 24 GHz; rain
scales sqrt(rain_k) not linearly so 24 GHz gets 0.215 rain weight
instead of the linear-ratio 0.95. 10 GHz stays as the reference band
(defaults); 47+ GHz inherits defaults (n<200).

- BandConfig.weights/1 returns per-band override or default fallback
- @band_configs carries :weights on 222/432/902/1296/2304/3400/5760/24G
- Recalibrator.compute_factors/3 + fit(band_mhz:) band-aware fitting
- Scorer + ContactLive.Show pass band_config to weights/1
- algo.md Part 2d documents the 2026-04-18 analysis + derivation rule
- scripts/derive_band_weights.py turns correlations into weight maps
- report preserved at docs/algo-reports/2026-04-18-recalibration.md
2026-04-18 10:19:26 -05:00
fc91f204fd
feat(propagation): record per-forecast-hour chain step timings
Add a propagation_run_timings table so the wall-clock duration of each
(run_time, forecast_hour) step is queryable long after the run is over.
Keyed by (run_time, forecast_hour) with a status column that captures
whether the step succeeded or bailed out, and an error string on
failure.

PropagationGridWorker stamps every step (ok and failed) via
Propagation.record_run_timing/1. Timing inserts are wrapped in rescue +
changeset-error handling so the instrumentation can never brick the
chain.
2026-04-18 09:36:52 -05:00
00aad3cb98
feat(auth): self-service password reset
Add a "Forgot your password?" flow off the login page. A 24-hour
reset_password token is emailed on request, the landing page lets the
user pick a new password, and all other tokens for the user are
revoked on success.

The request endpoint returns the same flash regardless of whether the
email matches a user so that attackers can't enumerate accounts.
2026-04-18 08:42:35 -05:00
b8b78f7e57
feat(design): apply NTMS design system tokens
Adopt the handoff design system: navy-slate surfaces with indigo-violet
accent (#6366F1 dark / #5458E3 light), Inter as the UI font, hairline
1px borders, softer corner radii. Add propagation-scale and amateur-band
CSS vars so JS hooks can pull theme-aware colors via getComputedStyle.

Polish: drop table zebra striping for hover-on-base-200 rows with
uppercase small-caps headers, bump .header to text-2xl, tighten nav
spacing with a hairline divider, fix broken text-brand -> text-primary
on the auth screens.

Wire --band-* into contacts_map_hook: resolve once at mount, re-resolve
on data-theme mutation so polylines recolor when the theme toggles.
2026-04-17 17:34:15 -05:00
33f5d4edbe
feat(rainscatter): classify QSO propagation mechanism from common-volume radar
Adds a per-contact enrichment pipeline that determines whether a QSO was
most likely carried by rain scatter, tropospheric ducting, or ordinary
troposcatter — using IEM n0q composite reflectivity sampled inside the
lens-shaped intersection of 400 km-radius disks around each endpoint.

Pieces:
  * Microwaveprop.Propagation.CommonVolume — lens geometry (haversine,
    in-CV test, bbox, area).
  * contact_common_volume_radar table (1:1 per contact) storing
    aggregate dBZ stats inside the CV + radar_status column on contacts.
  * Microwaveprop.Workers.CommonVolumeRadarWorker — Oban :radar queue,
    fetches the n0q frame at QSO time, iterates pixels inside the CV
    bbox, aggregates rain/heavy/core-pixel counts, max/mean dBZ, and
    coverage percentage.
  * Microwaveprop.Propagation.RainScatterClassifier — rule-based mapper
    from (band, distance, radar stats, duct flags) to one of
    :likely_rainscatter | :rainscatter_possible | :tropo_duct |
    :troposcatter | :unknown.
  * ContactWeatherEnqueueWorker learns a :radar enrichment type and
    enqueues the CV worker on contact submission; pre-2014 contacts
    (outside IEM n0q coverage) are pinned to :unavailable.
  * `mix radar_backfill` bulk-enqueues historical contacts with
    --year / --limit / --dry-run.
  * Contact detail page renders a mechanism badge with supporting
    stats (common-volume area, max dBZ, heavy-rain pixel count,
    coverage %).
2026-04-17 15:57:59 -05:00
c09446a517
fix(weather-map): default timeline cursor to the hour closest to now
Previously the map loaded the latest cached valid_time from GridCache,
which can be any +Nh forecast hour the worker has written while walking
f00-f18. Users expect to see current conditions on load, not a forecast
that's ~11 hours out.
2026-04-17 15:57:59 -05:00
b6caffea3e
feat(nav): show last-deployed timestamp
Docker final stage bakes BUILD_TIMESTAMP into /app/BUILD_TIMESTAMP via
build-arg (only consumed after mix compile, so earlier layer caching is
preserved). Application.build_timestamp/0 reads the file, falls back to
DEPLOY_TIMESTAMP env, then to compile time. Navbar renders a compact
"Deployed Xm ago" with the full UTC time in the tooltip.
2026-04-17 13:08:51 -05:00