Commit graph

26 commits

Author SHA1 Message Date
dc8353a9e9
test: coverage round 4 (84.39% → 84.44%) + 6 new property tests
30 unit tests + 6 property tests across two parallel agents.

- ContactLive.Show + HrrrNativeClient + NexradClient: sort_observations
  + sort_soundings actual ordering per field, closest_observations
  capped-at-5 proximity, nil-pos2 half_dist=0 path, HrrrNativeClient
  scrambled-level density + surface finiteness, NexradClient cache
  population + zero-byte + year-boundary URL rounding. Properties:
  sort_observations preservation, haversine symmetry, build_native
  surface_temp_k finiteness.

- PathLive + Viewshed + GefsFetchWorker + SnmpClient: GPS source
  URL preservation, QRZ 404 surfacing, propagation_updated same-midpoint
  no-op; Viewshed effective_reach_km BLOCKED boundaries + find_reach_km
  zero max_range; GefsFetchWorker 502/400/410 + wind_u/wind_v aliases
  + nil profile; SnmpClient fully-qualified OID + double-dot drop +
  empty poll + unknown radio type. Properties: destination_point
  round-trip < 0.5 km, valid_time = run_time + fh*3600 invariant,
  parse_snmpget_output totality.
2026-04-24 10:32:05 -05:00
c514626a62
test: coverage round 3 (83.96% → 84.39%) + 10 new property tests
64 unit tests + 10 property tests across three parallel agents.

- ContactLive.Show round 3 (72% → ~80%): enrichment failure
  surfaces (terrain/iemre/weather :failed, hrrr :unavailable NARR
  fallback), radar-only mechanism, load_solar + enqueue_missing_solar
  paths, blocked_message nil obs_dist, band_summary for 144M/1296M/
  2304M/47G/75G, admin edit no-op, mode changes, fetch_queue_counts
  with seeded Oban jobs. Properties: propagation_mechanism summary is
  always a binary, obs_sort_key totality over arbitrary field keys.

- Weather clients: iem_client transport timeouts + CSV edge cases,
  rtma_client idx url property, swpc_client HTTP 503/404 + empty-
  array parses + JSON/CSV totality properties, ncei_metar_client
  parse edge cases, snmp_client OID-roundtrip property + parse
  edge cases.

- Mid-coverage LiveViews: UserProfileLive avatar/render branches,
  EmeLive invalid-grid/unknown-callsign + form-validation property,
  PathLive zero-distance + 3000km + height/power round-trip property,
  BeaconLive numeric bearing + non-admin approve denied, MapLive
  select_band/toggle_radar/toggle_grid events + band-URL property.

Fix: propagation_mechanism property test uses System.unique_integer
for lat + valid_time nonces instead of StreamData-shrinkable jitter;
the shrink toward 0 was tripping the hrrr_profiles unique index.

205 → 215 properties, 2743 → 2812 tests, 0 failures.
2026-04-24 10:15:37 -05:00
bbf51544e1
test: coverage round 2 (82.05% → 83.96%) + 23 new property tests
77 unit tests + 23 property tests across four parallel agents.

- ContactLive.Show (72.01% → ~75%): every factor-note branch
  (humidity/time/td/refractivity/pressure/season/pwat), all
  propagation_mechanism classifications, apply_admin_edit/owner_edit/
  submit_user_edit error paths, internal_network? IP shapes, expanded
  sounding skew-T, composite-score [0,100] property.

- PathLive + BeaconLive.Show + Weather: coordinate-pair resolver,
  invalid grid, empty-DB edge cases for iemre_for_*, narr_for_*,
  nearest_native_duct_*, find_nearest_rtma, recent_surface_obs;
  properties for round_to_hrrr_grid/round_to_iemre_grid/band
  round-trip/beacon-id URL round-trip.

- Viewshed + Duct + SoundingParams: find_reach_km edge cases,
  destination_point, effective_reach_km fractions, detect_ducts
  trailing-duct + guard branches; 13 properties including flat-
  terrain-fully-visible, thicker-duct-lower-freq, M-profile
  monotonicity, feature-vector deterministic length.

- Workers + Mix tasks: GefsFetchWorker 500/429/403 + malformed
  ISO8601, HrrrNativeGridWorker snap-to-3-decimals + empty DB,
  IonosphereFetchWorker 404 + non-tabular body, RadarBackfill
  --dry-run + --year + --limit, NexradBackfill --limit 0, Backtest
  --feature + --all; year-filter property covering 2015..2026.

170 → 205 properties, 2666 → 2743 tests, 0 failures.
2026-04-24 10:15:37 -05:00
e11ebc9af8
test: coverage round 1 (80.45% → 82.05%) + 12 new property tests
Adds 67 test cases + 12 property tests across four surfaces — parallel
agents targeted the lowest-coverage files in the tree.

- ContactLive.Show (48.56% → 72.01%): 16 unit tests + 2 properties
  covering IEMRE/radar/ducting/propagation-analysis render branches,
  every terrain-verdict class, expanded HRRR/terrain sections, pending-
  edit banner, internal-network conn, and total-over-fields properties
  for obs + sounding sort handlers.

- HrrrNativeClient (33% → 58%): 8 unit tests + 4 properties covering
  partial-surface fallback, empty input, optional-var nils, bogus
  binary dispatch, and level-count + monotonic-height + URL-encoding
  + duct-subset invariants.

- NexradClient / HrrrClient / GefsClient: 13 unit tests + 4 properties
  covering 500/404/timeout paths, malformed idx bodies, parse_idx
  totality, byte-range invariants, and URL-builder round-trips.

- HrrrBackfill / HrrrNativeDeriveFields / ImportContestLogs: 7 unit
  tests + 1 property. Seeds contacts + native profiles, stubs HRRR
  with Req.Test, exercises the `--limit 0` and `filter_points_needing_
  backfill` paths, CSV import happy + malformed paths, and a
  band-string round-trip property.

2599 → 2666 tests, 170 → 182 properties, 0 failures.
2026-04-24 10:15:37 -05:00
1e8dd7036d
test: fix async-hydration timeout flake + type warning
- Bump render_async/2 timeout to 2_000ms across contact_live_test.
  Default 100ms races when the suite runs on a loaded box — ContactLive.
  Show fans out 8 hydration tasks at mount and sporadically missed the
  window even on a quiet DB.
- Telemetry metrics-shape assertion uses Enum.any?/1 instead of the
  `metrics != []` comparison Elixir 1.19's type system flags as a
  distinct-type comparison (the function's non-empty return type).
2026-04-24 09:28:52 -05:00
5564b6703d
test: push line coverage from 78.67% → 80.45%
Adds 30 more test cases (2594 → 2624 tests) targeting the last
low-coverage hot spots.

Notable additions:
- ContactLive.Show fully-hydrated render path (seeds HrrrProfile,
  TerrainProfile, ContactCommonVolumeRadar, Sounding, SurfaceObservation,
  IemreObservation) exercises refresh_computed, build_propagation_analysis,
  build_data_sources, compute_elevation_profile, and the downstream
  render branches they unlock.
- handle_info :terrain_ready and :sounding_fetched paths for same-contact
  and unrelated-contact dispatch.
- Flagged-invalid badge + line-through header style.
- Owner-label button path (Edit vs Suggest Edit).
- NotifyListener init/1 subscribes on schedule; handle_info(:subscribe)
  succeeds against the live Repo.
- RecalibrateScorer runs cleanly against an empty corpus via the
  Recalibrator insufficient-data fallback.
- HrrrNativeClient.extract_native_profiles/2 tolerates malformed input
  without crashing.
2026-04-24 08:27:18 -05:00
f99d07bd29
test: push line coverage from 71.7% → 78.67%
Adds ~600 new test cases (2013 → 2613 tests; 170 properties; 0
failures) across 12 new test files plus expansions of eight existing
files. Big lifts per module:

  ContactLive.Mechanism      33 → 100%
  MetricsPlug                54 → ~100%
  Microwaveprop.Release      15.8 → 70%+
  Telemetry                  38 → 76%
  HrrrNativeGridWorker       27.7 → 76%
  ContactLive.Show           34 → 48% (handler + render branches)
  Admin.ContactEditLive      49.7 → 70%+
  GefsFetchWorker            35.8 → ~55%
  IonosphereFetchWorker      56.3 → 75%
  PathLive                   58.9 → 67%
  WeatherMapLive             66.9 → 80.2%
  RoverLive                  0 → 70.3%
  ContactMapLive             59.2 → 89.8%
  ContactMapController       0 → 100%
  Mix tasks (Rust.Golden, Notebook, Backtest, HrrrBackfill,
    HrrrClimatology, HrrrNativeBackfill, NexradBackfill,
    RadarBackfill, ImportContestLogs, PropagationGrid,
    ResetEnrichment, Hrrr.PurgeGridPoints)  0 → ~60-100%

Two incidental fixes made while adding tests:

- ContactLive.Show.handle_event("toggle_flag", ...) was passing
  socket.assigns.current_scope to admin?/1 instead of socket.assigns,
  so admins never matched the pattern and every toggle ran the
  "Admins only." flash branch. Flag now toggles for admins again.

- Commercial.PollWorker.fetch_weather/1 promoted from private to
  @doc'd public so the IEM-fetch + ASOS-upsert path can be tested
  directly without driving perform/1 through live SNMP (which was
  timing out for ~50 s per test in the earlier attempt).

Stable property-test additions cover the dewpoint-from-RH monotonicity,
nearest_run/1 idempotence, and the ionosphere station envelope.
2026-04-23 18:43:18 -05:00
4b155c5419
feat(contacts): hide Private column when the viewer has nothing private
Row-level privacy was already correct: private rows only render for
the owner and admins. The header column was still showing for every
viewer, leaving an always-empty slot for anyone without private
contacts of their own. Drop `:private` from the fields list unless
the scope can actually see at least one private row.
2026-04-19 12:36:12 -05:00
6c652ef2d4
feat(contacts): private flag with scope-aware visibility
Contacts can be marked private at submit time (single form, CSV
import, ADIF import) and edit time. Private contacts are hidden
from the public map, the public /u/callsign profile, and the
browse table for anonymous and non-owning viewers. The original
submitter and admins see them inline on the browse table with a
"Yes" badge, and the detail page shows a lock icon. Non-authorized
viewers get a 404 on the detail page.
2026-04-18 17:49:01 -05:00
48cbf40b9d
feat(contacts): show total contact count in list header 2026-04-17 13:08:51 -05:00
2a57074a47
feat(contact-detail): use 10 GHz threshold for MHz/GHz band display
Anything below 10 GHz now renders in MHz so 2304/3456/5760 etc. match
amateur microwave convention. 10 GHz and above still render in GHz.
2026-04-17 12:08:38 -05:00
1a8ca44de3
feat(contact-detail): render band in MHz below 1 GHz, GHz above
The detail page was hard-coding GHz formatting, so 50/144/222/432/902
rendered as 0.1/0.4/0.9 GHz instead of the MHz labels every amateur
operator actually uses. Switch to MHz when band < 1000, keep GHz for
microwave bands.

Also surface band and mode in the header subtitle so the key QSO
metadata is visible before SRTM/terrain enrichment completes (the
elevation-profile block that previously rendered them is gated on
that enrichment).
2026-04-17 10:51:54 -05:00
3604896726
Rename ERA5 → NARR across the codebase
- Schema: Era5Profile → NarrProfile; table renamed via migration
  rename_era5_profiles_to_narr_profiles (ALTER TABLE + rename indexes,
  atomic and instant, no row rewrite)
- Weather helpers: find_nearest_era5/era5_for_contact/era5_profiles_for_path
  → find_nearest_narr/narr_for_contact/narr_profiles_for_path
- BackfillEnqueueWorker: :era5 → :narr type (virtual, no narr_status column);
  reconcile_stale_queued_to_unavailable and status_priority_order now skip
  virtual types via @virtual_types. Fixes the prod crash where the cron tried
  to update a non-existent era5_status column.
- ContactWeatherEnqueueWorker: build_era5_jobs → build_narr_jobs,
  era5_jobs_for_contact → narr_jobs_for_contact
- ContactLive: @era5 → @narr, maybe_enqueue_era5 → maybe_enqueue_narr;
  UI label "ERA5 (0.25°)" → "NARR (32 km)"
- Cron (runtime.exs): args types list now "narr" instead of "era5"
- /status: progress row, status-by-type card, totals.narr_profiles, and
  table-count lookup all target narr_profiles
- Drop obsolete Era5CdsJob schema + era5_cds_jobs table (inspection
  artifact from the retired CDS pipeline; 34 orphan rows in prod)
- Misc docstring/comment cleanups (skew_t, about, wgrib2, propagation_train)

Includes a regression test for the virtual-type crash.
2026-04-16 09:22:23 -05:00
30c1018400
Extend skew-T plot to cover the full troposphere
Historical contacts showed a skew-T log-P diagram that stopped at 700 mb
because HrrrClient and Era5Client only fetched pressure-level data down
to the top of the boundary layer. The chart canvas already ran up to
100 mb, so the trace clipped mid-atmosphere.

Two complementary fixes:

1. Extend @pressure_levels in HrrrClient and Era5Client with
   650/600/550/500/450/400/350/300/250/200/150/100 mb so new fetches
   cover the full troposphere + lower stratosphere.

2. Prefer the native hybrid-sigma profile for the contact-detail
   skew-T when one has been backfilled for the contact's hour. The
   native profile already stores all 50 hybrid levels up to ~19 km,
   so historical contacts covered by the native backfill get a full
   trace without re-hitting S3. A new HrrrNativeProfile.to_skew_t_profile/1
   converts the parallel arrays into the %{"pres","tmpc","dwpc","hght"}
   list shape the renderer expects, deriving dewpoint from SPFH via the
   Magnus inverse. Weather.find_nearest_native_profile/3 mirrors
   find_nearest_hrrr/3 for the lookup.
2026-04-14 17:25:58 -05:00
d4b849f976
Show ERA5 fallback on contact detail, fix status page ERA5 progress
Contact detail page now renders an atmospheric profile section backed by
HRRR when available and ERA5 (reanalysis) otherwise, labeled with the
data source. When both sources are missing and HRRR is known-unavailable
(pre-2014 contacts), the Era5FetchWorker is enqueued so the month-tile
backfill gets triggered from a user visit.

Status page ERA5 progress previously computed "complete" as
(total_contacts - hrrr_unavailable_contacts), which treated every contact
with any HRRR status as ERA5-complete — inflating the bar to 76% while
era5_profiles had 0 rows. Replaced with a per-candidate EXISTS query
against era5_profiles using the same 0.15°/30-min tolerance
Weather.find_nearest_era5/3 uses at lookup time.
2026-04-13 13:44:31 -05:00
f733ddb6ac
Convert /contacts list to live_table
Replaces the hand-rolled sort/search/pagination on /contacts with
LiveTable.LiveResource. Sortable columns for station1/2, grid1/2,
band, mode, distance_km, qso_timestamp, and inserted_at; searchable
across both stations, both grids, and mode. Custom renderers preserve
the enrichment summary badge (with per-status tooltip), the invalid
flag column, and the qso/inserted_at formatters.

The reciprocal-grouping block has been dropped per the brainstorming
decision — every QSO now shows as its own row, which avoids fighting
live_table's one-row-per-record model. A "View" action links to the
detail page instead of the previous row-click handler (live_table's
stream dom_ids are random UUIDs so row-click by record id is no
longer practical without forking the library).

Test updates:
- sort test switched to ?sort_params[station1]=asc (live_table URL
  format; the old ?sort_by/?sort_order params are gone)
- pair-search test removed since live_table does a single ILIKE
  across searchable fields and no longer supports the two-callsign
  intersection that Radio.list_contacts/1 used to do; single-term
  search is exercised instead
- pagination test asserts the presence of "Page" (live_table's
  pagination text is "Page N" without the total count)

Also converts the stray cond-with-single-clause in
UserManagementLive.Index's delete handler to if/else (credo fix).
2026-04-12 17:22:15 -05:00
fc245367e3
User profiles at /u/:callsign, flexible band input, assorted UX cleanup
Profile page:
  * New MicrowavepropWeb.UserProfileLive at /u/:callsign is a public
    page showing a user's contacts and beacons. Resolves case-
    insensitively so /u/w5isp and /u/W5ISP are the same thing; unknown
    callsigns redirect to /. Uses daisyUI card / stats / table
    components with an avatar-placeholder initial and hero icons.
  * Accounts.get_user_by_callsign/1 (case-insensitive) plus
    Radio.list_contacts_for_user/1 and Beacons.list_beacons_for_user/1
    back the page. Beacons list includes both approved and pending so
    owners see their drafts.
  * The top nav bar and the three LiveView sidebars (MapLive,
    WeatherMapLive, ContactMapLive) now render the logged-in callsign
    as a navigate link to /u/:callsign instead of a static label.
  * Nine new tests cover the lookup, the LiveView render, and the
    ownership-scoped queries.

Flexible band input:
  * New Microwaveprop.Radio.BandResolver module converts any of:
    ADIF wavelength labels ("33cm", "1.25cm", "6mm", case/whitespace
    insensitive), numeric frequency strings ("903.100", "10368.000"),
    and canonical MHz integers into the one of the site's known bands.
    Returns the nearest allowed band for numeric inputs >= 900 MHz,
    nil otherwise.
  * 902 MHz is added to Contact.@allowed_bands, ContactEdit.@allowed_bands,
    AdifImport.@allowed_bands, and the BandResolver list so "33cm"
    round-trips end-to-end.
  * AdifImport and CsvImport now delegate band resolution to
    BandResolver, and Radio.create_contact/2 normalizes the :band attr
    on the way in so the manual form and any API callers benefit too.
    CsvImport's "invalid band" tests previously used 99999 MHz which
    the new resolver snaps to the nearest allowed band; swapped to
    "notaband" which is truly unresolvable.

Contacts and beacons list UX:
  * Remove the "Submitted" column from /contacts — it duplicated info
    already visible on the detail page and was pushing the real
    columns off narrow viewports. submitted_cell/1 and its three
    column-specific tests go with it.
  * Hide the Lat / Lon columns from /beacons — six decimal places of
    coordinates weren't useful next to the grid square and took a
    disproportionate amount of row width.
2026-04-12 16:13:08 -05:00
a40ea0d62d Show submitter callsign on the contacts index
When a contact is linked to a logged-in user account, display that
user's callsign in the Submitted column instead of a generic "Yes".
Anonymous submissions still show "Yes"; scraped contacts show an em
dash.
2026-04-09 14:02:30 -05:00
4759c65809
Support two-callsign search on contacts page to find contacts between a pair 2026-04-04 09:29:32 -05:00
d36256e730
Show queue position on all loading spinners, fix solar backfill
- Unified queue_info helper shows "(N jobs in queue)" on all spinners
- Solar index auto-enqueues when missing on contact page
- Freshness monitor disabled in dev
- Fix test stubs for solar client
2026-04-02 16:17:31 -05:00
d207863b15
Live-update contact detail page when enrichment completes
- HRRR and terrain workers broadcast via PubSub on completion
- Contact show page subscribes and recomputes elevation profile,
  propagation analysis, and HRRR data on receipt
- Page updates in-place without reload when background jobs finish
2026-04-02 08:17:43 -05:00
a25864bdcb
Sanitize and validate user-submitted contact fields
- Callsigns: strip whitespace, upcase, validate alphanumeric + /
- Grids: strip whitespace, upcase before Maidenhead validation
- Email: stricter regex, max 254 chars
- Callsign max length 20 chars
- Fix static_paths to include downloads directory
- Update show test for removed contact details section
2026-04-01 15:41:42 -05:00
f7a753626d
Add interactive popups to contact map, dedup reciprocals, darken line colors 2026-04-01 12:39:02 -05:00
cd8d7a0f52
Add contact detail map, elevation profile chart, and UI improvements
- Leaflet map on contact detail pages showing both station locations
- SRTM elevation profile with Chart.js: terrain, LOS, Fresnel zone
- Path info: distance, azimuth, elevation angle in both directions
- Compact 4-column grid layout for contact details
- Contact map page with all contacts plotted
- Rename QSOs to Contacts in page header
- Wrap algo page in standard layout
2026-04-01 12:34:03 -05:00
254e64dedc
Rename qsos to contacts throughout codebase, keep DB table name
Rename all modules, functions, variables, routes, and UI text from
qso/qsos to contact/contacts. Database table stays as "qsos" to avoid
migration. Add /qsos -> /contacts redirects for old URLs.
2026-04-01 11:25:04 -05:00
bc529af392
Update algo.md Finding 10: corrected mode advantage, no SSB on rainscatter 2026-04-01 11:10:11 -05:00
Renamed from test/microwaveprop_web/live/qso_live_test.exs (Browse further)