Commit graph

51 commits

Author SHA1 Message Date
Graham McInitre
49ade78766 fix: wire pending_edits_query as data_provider for contact edit review table
The LiveTable on /admin/contact-edits used the bare ContactEdit schema
as its data source, which caused three symptoms:
- '0 pending' counter but stale approved/rejected edits still visible
- Blank contact/submitted-by cells (select_columns stripped preloaded
  associations, cell renderers received flat maps with no :contact/:user)
- Approve/reject didn't remove the row from the table

Fix: assign {Radio, :pending_edits_query, []} as the data_provider in
mount so handle_params threads it to stream_resources. The query variant
of list_resources preserves preloaded associations and includes the
WHERE status = :pending filter.

Added two tests that verify the table rendering and edit removal.
2026-07-22 08:54:46 -05:00
fd976b0cd5
fix: resolve 391 Credo issues across codebase
- Add jump_credo_checks ~> 0.4 with all 20 checks enabled
- Fix all standard Credo issues: 139 @spec (113 done, 26 remain),
  4 refactoring, 3 alias usage, 9 System.cmd env, 5 unsafe_to_atom,
  2 max line length, 9 assert_receive timeout
- Fix 170+ jump_credo_checks warnings:
  - 117 TopLevelAliasImportRequire: move nested alias/import to module top
  - 32 UseObanProWorker: switch to Oban.Pro.Worker
  - 4 DoctestIExExamples: add doctests / create test file
  - ~20 WeakAssertion: strengthen type-check assertions
  - Various ConditionalAssertion, AssertReceiveTimeout fixes
- Exclude vendor/ from Credo analysis
- Remaining: 175 warnings (mostly opinionated WeakAssertion,
  AvoidSocketAssignsInTest), 26 @spec annotations
2026-06-12 13:51:32 -05:00
3bdc4b6a11
Fix 4 critical bugs and document remaining findings
- TOCTOU race: add partial unique index + rescue constraint violation on
  concurrent contact insert (radio.ex:737-768)
- Mass assignment: remove :user_id from @optional_fields (contact.ex:85)
- Stale path_live result: clear result when URL params change (path_live.ex:105)
- Blocking migrations: wrap Release.migrate() in Task.start (application.ex:79)
- Rate limiter: fix guard for monitor tokens (not is_nil vs is_binary)
- findings.md: document remaining non-critical bugs and improvements
2026-05-29 15:27:33 -05:00
a3b97f9e98
fix(security): mass-assignment on owner edit + 3 minor hardening fixes
- Radio.normalize_proposed/1 now Map.take/2 the allowed-edit keys
  before normalization. Owner / admin / pending-edit submit paths all
  funnel through this boundary, so a crafted form can no longer
  mass-assign user_id, flagged_invalid, inserted_at, etc. on a
  contact via String.to_existing_atom -> Ecto.Changeset.change.
- Accounts.revoke_api_token/2 rescues Ecto.Query.CastError and
  returns {:error, :not_found} so a malformed UUID in
  DELETE /api/v1/me/api-tokens/:id renders the API's clean 404
  problem+json instead of a 500.
- ProfilesFile.read_etf decodes with :erlang.binary_to_term(bin, [:safe]).
  Defense-in-depth against tampered on-disk profiles (atom-table
  exhaustion via untrusted ETF).
- Regression tests: contact_edit_test.exs covers the rejected
  mass-assignment fields; accounts_api_token_test.exs covers the
  malformed-UUID path.
2026-05-11 19:04:16 -05:00
da332d3174
test: 100% coverage push — Valkey.RedixAdapter, Buildings.BulkFetch, AprsRepo, Maidenhead, Canopy
- Valkey.RedixAdapter: 0% -> 100% (delegation lines exercised)
- Buildings.BulkFetch: 0% -> 56% (run/4 with stubbed empty index)
- AprsRepo: 50% -> exercises repo config + Ecto.Repo callbacks
- Maidenhead.valid?/1: covers non-binary non-nil arms
- Canopy.tile_filename/2: S/E hemisphere prefix branches
2026-05-08 10:01:28 -05:00
fe2ecf2618
feat(import): strip non-printables from CSV/ADIF uploads
Adds Microwaveprop.Radio.TextSanitizer to remove BOM, zero-width
spaces/joiners, C0 control bytes (except CR/LF/TAB), and DEL, and to
collapse NBSP and other unicode whitespace into a regular space.

CSV: applied to the whole upload at the top of preview/2.
ADIF: applied per-field-value after extraction so the byte-counted
length tags still slice the right substring.
2026-04-26 15:34:46 -05:00
6aa91e7656
fix: April 2026 codebase review — address 13 bugs across propagation chain
Each fix is covered by a regression test that fails on `main` and
passes on this commit.

Round 1 (initial review):

* propagation: thread `latitude` into the conditions map so
  `score_season/4` actually picks up regional multipliers
* hrrr_client / fetcher.rs: `nearest_hrrr_hour` rounds DOWN, never at
  a future cycle that NOAA hasn't published yet
* radio: spherical-vector great-circle midpoint replaces the
  arithmetic mean — anti-meridian paths no longer fold to Greenwich
* weather: `reconcile_weather_statuses` scales the longitude band by
  `1 / cos(lat)` so the bbox stays ~150 km wide at every latitude
* radio/maidenhead: clamp 90°/180° below the field-bucket overflow so
  `from_latlon` never emits invalid characters like 'S'
* prop_grid_rs/pipeline: merge HRRR + NEXRAD-derived rain rates and
  read `best_duct_freq_ghz` into `best_duct_band_ghz` so the Native
  Duct Boost actually fires
* propagation/region (Elixir + Rust): inclusive upper bounds so points
  exactly at lat_max get the regional multiplier
* weather/sounding_params (Elixir + Rust): drop the 10 m gradient
  floor so HRRR's thin near-surface layers stop hiding sharp ducts
* weather/sounding_params: when the profile ends inside a duct,
  finalize it with the highest sample as the top instead of throwing
  it away (Rust port already correct)

Round 2 (post-fix sweep):

* radio + commercial: single canonical haversine in Radio (atan2
  form); Commercial delegates instead of carrying a second copy that
  could disagree at threshold distances
* prop_grid_rs/profiles_file: `snap_coords` matches Elixir's
  step-aware snap (`round(coord/0.125) * 0.125`, then 3-dp round) so
  Rust-keyed and Elixir-keyed profile maps land on the same cell
* weather/grib2/wgrib2: `parse_lon_val_segment` uses `Float.parse`
  uniformly — wgrib2 dropping the trailing `.0` from a longitude no
  longer crashes the whole chain step
2026-04-25 10:52:51 -05:00
c17f912622
fix(radio): invalidate contact-map cache on edits, not just inserts
`apply_edit_to_contact/2` is the path for both owner direct-edits and
admin reviewed-edits. The public contact-map + total-count + gzipped
controller payload caches are built from `private == false` rows, but
only the insert path was clearing them — every other update left
/api/contacts/map and /contacts/map serving the previous snapshot for
up to 10 minutes. A `private: false -> true` flip on a contact already
present in the cached payload was a temporary privacy leak; grid
corrections, callsign changes, and flagged_invalid flips silently
diverged from the live row.

Extracts the three `Cache.invalidate` calls into
`invalidate_contact_map_caches/0` and runs it from both
`apply_edit_to_contact/2` and the existing insert path so the public
view stays in sync.
2026-04-25 10:08:14 -05:00
13996583bd
fix(radio): add private clause to current_value/2
The /contacts/:id edit form always submits the `private` checkbox, but
`current_value/2` had no `"private"` clause and fell through to the
catch-all `nil`. The Contact schema defaults `private: false`, so an
unchanged box compared `false != nil` and was always tagged as a
change — for non-owners that meant a spurious pending review queued
on every save, and for owners/admins it triggered a no-op direct
update.

Adds the missing clause so the diff sees boolean-vs-boolean and
correctly returns `:no_changes` when the box is untouched.
2026-04-25 10:04:39 -05:00
2365c19321
fix(radio): coerce qso_timestamp string in normalize_proposed
The /contacts/:id edit form pre-fills qso_timestamp via
`Calendar.strftime(ts, "%Y-%m-%d %H:%M")` and submits whatever the
user typed back unchanged. Two failure modes resulted:

1. `diff_against_contact/2` compared that string against the stored
   `%DateTime{}`, never matched, and treated every untouched submit
   as a modification.
2. `build_contact_changes/2` then ran `DateTime.from_iso8601/1` on
   the space-separated, no-seconds, no-Z form and crashed with
   `MatchError` on `{:error, :invalid_format}` — direct owner/admin
   submits raised instead of saving.

Adds `normalize_timestamp_field/2` to `normalize_proposed/1` that
parses both the form's `"YYYY-MM-DD HH:MM[:SS]"` and the strict
`"YYYY-MM-DDTHH:MM:SSZ"` shapes into a UTC `%DateTime{}` (with
`NaiveDateTime.from_iso8601` doing the heavy lifting after a small
seconds-padding pass). Empty strings drop the field; unparseable
input drops it too so a typo becomes "no changes" rather than a
500.
2026-04-25 10:03:26 -05:00
0a9058bfa0
test: broaden coverage across parsers, schemas, and pure helpers
- SwpcClient: unparseable time_tag rows drop silently, integer / string
  numerics both cast, already-decoded list bodies (Req auto-decode) are
  accepted, non-JSON-array bodies return {:error, :not_a_list}, X-ray
  rows whose energy isn't the long wavelength are filtered, and the
  misspelled-upstream electron_contaminaton key flags
  electron_contaminated.
- UwyoSoundingClient: every month abbreviation routes to its correct
  number, bogus month tokens fall through to an empty result, and
  short-of-fixed-width lines are skipped without raising.
- Grid.wgrib2_grid_spec/0: lon/lat_start anchor to the SW corner,
  steps match Grid.step/0, cell counts span CONUS inclusive, and the
  spec's cell product equals length(Grid.conus_points/0).
- PollWorker: empty-link list is a no-op, and poll_fn receives host /
  community / radio_type straight from the link row.
- Contact submission_changeset: antenna heights are bounded to
  [0, 1000] ft, whitespace-only mode normalises without firing an
  invalid-mode error, plus a pair of StreamData properties — every
  sanctioned band validates and every out-of-list band string is
  rejected.

Suite: 2,380 tests + 148 properties (was 2,370 + 146); coverage
70.38% → 71.08% total; credo strict clean.
2026-04-23 13:56:29 -05:00
cfb0c51d2f
test: add property coverage for grid snaps, backoff, notes
Pure-function invariants that example tests can't express at scale:

- Weather.round_to_iemre_grid/2, HrrrClient.nearest_hrrr_hour/1,
  NexradClient.round_to_5min/1, NarrClient.snap_to_analysis_hour/1:
  each is idempotent, its output satisfies a grid/slot-membership
  predicate, and lies within a bounded distance of the input.
- NarrClient.in_coverage?/1: coverage_end is an exclusive upper
  bound; any timestamp after it is out of coverage.
- IemreFetchWorker.backoff/1: result is in [120, 21_600], monotonic
  non-decreasing in attempt, equals 120*2^(attempt-1) pre-cap.
- Contact submission_changeset notes field: every non-blank string
  up to 2000 chars validates; any longer string is rejected.

22 new properties (139 → 161 including existing); all 2,300 tests
green, credo clean.
2026-04-23 13:05:06 -05:00
b2ff27945d
feat(adif): carry NOTES / COMMENT into contact notes
ADIF records frequently include operator commentary in the
<NOTES> field (multi-line, detailed) or <COMMENT> (short
one-liner). Now that contacts have a notes column, pass that
value through the import pipeline instead of dropping it.

Preference is NOTES over COMMENT to match the ADIF 3.1.4 spec's
intended split — NOTES is the richer field. Whitespace-only
values collapse to nil so a program that always emits a blank
NOTES tag doesn't fill every row with an empty string.

Tests cover NOTES capture, COMMENT fallback, NOTES-over-COMMENT
preference, and the absent-field nil case. ADIF upload blurb
lists the new carry-over so users see it before confirming.
2026-04-23 11:05:39 -05:00
cd3950f366
feat(contacts): add free-form notes field
Operators want a place to jot observations that aren't captured by
the structured fields — weather anecdotes, propagation mode
commentary, equipment details, band conditions. Add a text column
on contacts plus the two ingest paths users submit through:

  * /submit single-contact form gets a 3-row textarea under the
    other fields. Optional, max 2000 chars, with a live length cap
    via maxlength. Whitespace-only input collapses to NULL so the
    column reflects "no notes" rather than an empty string.
  * CSV importer recognises a `notes` (or `note`) column via the
    existing header_aliases table and flows values straight through
    submission_changeset. Sample template gets a matching example
    row with embedded commas so the quoted-field round-trip is
    exercised on download.

ADIF import and the refinement/refinement-notify paths are out of
scope — users specifically asked for CSV + single-contact. Tests
cover the changeset (accept/blank/over-length), CSV header parsing
(plain + RFC-4180 quoted), and LiveView form submit end-to-end.
2026-04-23 11:02:27 -05:00
32ba5de456
test(property): add property tests for parsing + domain conversions 2026-04-21 13:57:47 -05:00
71e8f53142
fix: stabilize flaky tests + silence PromEx Oban poller in test
- HrrrClient idx-cache test only invalidated the surface idx URL, but
  fetch_profile also fetches a pressure idx. Previous runs' state for
  the pressure key decided whether the counter landed at 1 (prior run
  cached it, pass) or 2 (cold, fail). Invalidate both + assert 2 total
  fetches to reflect the actual code path.
- CsvImportTest deadlocked against other async DataCase tests when
  inline Oban child jobs upserted iemre_observations/terrain_profiles
  with a shared conflict target. Flip to async: false — same fix as
  ContactWeatherEnqueueWorkerTest earlier this session.
- PromEx.Plugins.Oban runs a 5s telemetry_poller that queries the DB,
  but its poller PID has no sandbox connection in test and crashed
  with DBConnection.OwnershipError on every tick, spamming the log.
  Gate the plugin on a config flag and skip it in config/test.exs;
  prod behaviour unchanged.
2026-04-21 13:49:07 -05:00
6c652ef2d4
feat(contacts): private flag with scope-aware visibility
Contacts can be marked private at submit time (single form, CSV
import, ADIF import) and edit time. Private contacts are hidden
from the public map, the public /u/callsign profile, and the
browse table for anonymous and non-owning viewers. The original
submitter and admins see them inline on the browse table with a
"Yes" badge, and the detail page shows a lock icon. Non-authorized
viewers get a 404 on the detail page.
2026-04-18 17:49:01 -05:00
ca07c0288d
feat(contacts): antenna height fields feed terrain analysis
Add optional height1_ft/height2_ft to contact schema, submit form, and
edit form (direct + suggest-edit paths). Heights flow through to the
elevation-profile terrain analysis so clearance and diffraction are
computed from the actual antenna height instead of a 10-ft default.
Editing heights resets terrain_status and re-enqueues
TerrainProfileWorker so the stored path analysis picks up the new
geometry.

Also fix mark_likely_duct/3: the pattern was grabbing the element
instead of the index, so no duct ever got flagged as the likely
propagation path even when extract_ducts returned a valid layer.
2026-04-18 15:48:14 -05:00
622edee180
feat(propagation): per-contact mechanism classification
Classifies every contact's likely non-LOS propagation mechanism and
persists the result on contacts.propagation_mechanism. Mechanism is
determined in priority order:

  1. user_declared_prop_mode (ADIF PROP_MODE from the operator log)
  2. EME — moon-ephemeris check, ≥2m band, >1800 km path
  3. aurora — Kp≥5 + high-lat path, 50-432 MHz
  4. sporadic-E — foEs × 5 ≥ band_mhz, 400-2500 km path
  5. meteor_scatter — ±3 days of a shower peak, VHF/UHF
  6. rain_scatter — common-volume radar heavy rain, 5-11 GHz ≤800 km
  7. tropo_duct — HRRR native_best_duct ≥ band or ducting_detected
  8. line_of_sight — ≤50 km path
  9. troposcatter — default

Persisted via MechanismClassifyWorker (queue: :mechanism, unique on
contact_id). Submit-time enqueue path includes :mechanism by default;
BackfillEnqueueWorker cron now handles :mechanism alongside existing
types so prod continuously backfills any contact with
propagation_mechanism_status in (:pending, :queued, :failed). Also
added :radar to the cron's type list so common-volume radar backfill
runs automatically rather than only via `mix radar_backfill`.

New modules:
- Microwaveprop.Propagation.MoonEphemeris — Meeus low-precision moon
  position, accuracy ±1° — enough for the mutual-visibility EME test
- Microwaveprop.Propagation.MechanismClassifier — plug-in priority
  chain over the evidence map
- Microwaveprop.Workers.MechanismClassifyWorker — assembles inputs
  from HRRR / native profiles / common-volume radar / solar_indices /
  ionosonde + calls the classifier

ADIF importer now reads PROP_MODE into user_declared_prop_mode so
operator-tagged mechanisms (EME/ES/MS/RS/AS/AUR) become ground truth.
2026-04-18 10:42:08 -05:00
48833f15be
feat(import): async CSV import with progress-tracking schema
For large CSVs (e.g. the 34k-row ARRL dump), the synchronous commit
path blocked for minutes and a crash mid-commit left no audit trail.
Refactor to run inserts + enrichment enqueues via Oban.

- New import_runs table: stores preview rows + running counters (total,
  processed, imported, refined, error) + status + errors map + timing.
- New :contact_import Oban queue (limit 4) + ContactImportWorker.
  Args: {import_run_id, offset, limit}. Each chunk (100 rows) inserts
  its slice via CsvImport.commit_rows/1, atomically increments counters
  in one update_all, flips status, and broadcasts progress on
  'csv_import:<id>' PubSub topic.
- CsvImport.commit_rows/1 extracted from commit/1 (back-compat preserved).
  New CsvImport.enqueue/2 persists the run + dispatches N chunk jobs.

Also: submit page copy updated from '902 MHz and up' to '50 MHz and up'
matching the band allowlist expansion done earlier.

LiveView UI for /imports/:id is a separate commit.
2026-04-17 09:31:22 -05:00
8a969e315c
refactor: normalize pos1/pos2 JSONB key to 'lon' everywhere
57,186 prod contacts stored pos1/pos2 with 'lng'; 1,133 used 'lon'.
Every Elixir caller carried a `pos["lon"] || pos["lng"]` fallback
— which just caused a SQL widget to silently miscount 98% of contacts
(count_narr_done used `pos1->>'lon'` directly, no fallback, so every
lng-keyed row returned NULL and failed the coverage check).

- Migration rewrites every pos1/pos2 JSONB in place, renaming 'lng' to
  'lon' and dropping 'lng'.
- Removes all 20+ `|| pos["lng"]` fallbacks across lib/, workers,
  scorer, weather, radio.ex, contact show view, and recalibrator.
- lib_ml/propagation_analyze.ex SQL now reads pos1->>'lon' directly
  (was reading 'lng' only, which would have broken after migration).
- priv/repo/import_contacts.exs one-time seed script now emits 'lon'
  with string keys, matching production shape.
- Test fixtures in 4 test files normalized to 'lon'.
- Two lng-characterization tests deleted — nonsensical post-normalize.
- Updated notebook + old import_weather script to match.
- JS hook contact_map_hook.ts TypeScript type narrowed to 'lon'.
2026-04-17 09:10:32 -05:00
4f956e4606
test: cover radio/edit_notifier
13 characterization tests for deliver_edit_approved + deliver_edit_rejected:
to/from/reply-to, subject format, callsign greeting, APPROVED/REJECTED tags,
band formatting (MHz vs GHz), change rendering, admin-note present/nil/empty,
endpoint URL link, preload tolerance.
2026-04-16 14:58:55 -05:00
cb2937f12c
test: cover geo, qrz record, callsign client
34 characterization tests for pure/wrapper modules. Documents existing
behavior — no production code changes.
2026-04-16 14:09:44 -05:00
f93f5c9a57
Rename 9cm band 3456 → 3400
The US 9cm amateur allocation was cut from 3300-3500 to 3300-3450 MHz
in 2020, so 3456 (the legacy weak-signal calling frequency) is no
longer in-band. Move the canonical key to 3400, migrate existing
contacts, and keep 3456 input resolving to 3400 via the
nearest-band snap so historical ADIF/CSV logs still import cleanly.
2026-04-16 12:40:38 -05:00
36adf875b7
Add 50/144/222 MHz bands, rename 440→432
Expands submittable-contact bands to include 6m, 2m, 1.25m, and 70cm
(as 432 rather than the old 440 placeholder). Each new band gets an
explicit allocation window in BandResolver.nearest_band so ADIF FREQ
fields near the amateur allocations resolve correctly while 60-900 MHz
frequencies outside those windows are still rejected. Microwave
(>= 900 MHz) snapping is unchanged — nearest-band match across the
full @allowed_bands list.

Also adds BandConfig entries for 50 and 222 (tropo-only config, same
pattern as 144/432). Sporadic-E / F2 / meteor scatter modeling is not
yet in scope — ionosphere data is only used to compute an Es readout
on /path for 50/144/222/432.
2026-04-16 12:36:10 -05:00
bbeb95ff5d
CSV/ADIF import: upsert existing contacts on grid/mode refinement
Widen dedup match to 4-char grid prefix so an upload with a longer grid
(e.g. FN42aa25) finds the existing FN42 contact. Within a match, classify
as a refinement when the upload strictly extends the existing grid or
fills a missing mode; as a contradiction (still skipped) when grids or
modes genuinely disagree. Refinements flow through a new preview bucket,
update the existing row in place, and re-enqueue enrichment when the
position changed.
2026-04-16 10:57:37 -05:00
be6f1a1d28
CsvImport: header-driven column mapping
The /contacts LiveTable export produces a 12-column CSV with label-style
headers (Station 1 / Grid 1 / QSO (UTC) / ...) in a different column
order plus extras (id, distance_km, hrrr_status, flagged_invalid,
inserted_at). The old positional import rejected it with "expected 6 or
7 columns, got 12".

Replace the positional parser with header-driven mapping:
- Parse the header row, normalize each cell (downcase + trim), and look
  it up in an alias map that covers both the raw field names and the
  export labels.
- Required columns: station1, station2, grid1, grid2, band, qso_timestamp
  (mode stays optional). If any are missing the header, return
  {:error, {:missing_required_columns, [...]}}.
- Unknown columns are silently ignored so the export's extras don't break
  round-trip.
- Per-row validation stays the same (changeset errors still reported by
  row number).

Existing 6- or 7-column positional CSVs that already have a header still
work — the header parser recognizes `station1`, `station2`, etc.
directly.
2026-04-16 09:53:14 -05:00
fc245367e3
User profiles at /u/:callsign, flexible band input, assorted UX cleanup
Profile page:
  * New MicrowavepropWeb.UserProfileLive at /u/:callsign is a public
    page showing a user's contacts and beacons. Resolves case-
    insensitively so /u/w5isp and /u/W5ISP are the same thing; unknown
    callsigns redirect to /. Uses daisyUI card / stats / table
    components with an avatar-placeholder initial and hero icons.
  * Accounts.get_user_by_callsign/1 (case-insensitive) plus
    Radio.list_contacts_for_user/1 and Beacons.list_beacons_for_user/1
    back the page. Beacons list includes both approved and pending so
    owners see their drafts.
  * The top nav bar and the three LiveView sidebars (MapLive,
    WeatherMapLive, ContactMapLive) now render the logged-in callsign
    as a navigate link to /u/:callsign instead of a static label.
  * Nine new tests cover the lookup, the LiveView render, and the
    ownership-scoped queries.

Flexible band input:
  * New Microwaveprop.Radio.BandResolver module converts any of:
    ADIF wavelength labels ("33cm", "1.25cm", "6mm", case/whitespace
    insensitive), numeric frequency strings ("903.100", "10368.000"),
    and canonical MHz integers into the one of the site's known bands.
    Returns the nearest allowed band for numeric inputs >= 900 MHz,
    nil otherwise.
  * 902 MHz is added to Contact.@allowed_bands, ContactEdit.@allowed_bands,
    AdifImport.@allowed_bands, and the BandResolver list so "33cm"
    round-trips end-to-end.
  * AdifImport and CsvImport now delegate band resolution to
    BandResolver, and Radio.create_contact/2 normalizes the :band attr
    on the way in so the manual form and any API callers benefit too.
    CsvImport's "invalid band" tests previously used 99999 MHz which
    the new resolver snaps to the nearest allowed band; swapped to
    "notaband" which is truly unresolvable.

Contacts and beacons list UX:
  * Remove the "Submitted" column from /contacts — it duplicated info
    already visible on the detail page and was pushing the real
    columns off narrow viewports. submitted_cell/1 and its three
    column-specific tests go with it.
  * Hide the Lat / Lon columns from /beacons — six decimal places of
    coordinates weren't useful next to the grid square and took a
    disproportionate amount of row width.
2026-04-12 16:13:08 -05:00
b49c08914e
Force 24h timestamps and let logged-in owners edit their contacts directly
Submit and contact-detail forms now use a plain text input for the QSO
timestamp with a "YYYY-MM-DD HH:MM" placeholder and regex pattern. The
native datetime-local input ignores lang="en-GB" on macOS/iOS when the
system clock is 12-hour, so it was rendering AM/PM against the user's
wishes. Ecto's :utc_datetime cast already accepts this format (verified
space+seconds, T+seconds, and both with and without seconds/Z).

Contact edit workflow grows a third branch: admins still apply directly,
non-owners still go through the admin review queue, and now logged-in
owners of a contact (user_id == current_user.id) can apply edits
directly too. New helpers Radio.owner?/2 and Radio.apply_owner_edit/3
reuse the existing normalize + diff + apply_edit_to_contact path so the
grid-change → enrichment re-enqueue pipeline kicks in automatically.
Anonymous contacts (user_id nil) and other users' contacts both return
{:error, :not_owner}.

Nine new tests cover owner?/2 and apply_owner_edit/3 including the
no-changes, wrong-user, and anonymous-contact paths.
2026-04-12 15:18:47 -05:00
265669fc3a
Fix ADIF parser crash on records with multi-byte UTF-8 characters
Regex.scan with `return: :index` reports BYTE offsets, but String.slice/3
uses CHARACTER offsets. When a record contained a multi-byte UTF-8
character in an earlier field (e.g. "café" in NOTES), every subsequent
field's slice shifted one byte left, eventually landing on a literal
">" that crashed String.to_integer/1 with ArgumentError.

Switch to :binary.part/3 which is byte-based and matches what Regex.scan
reports. Also simplify the value offset calculation to use the scanned
tag_len directly instead of re-splitting the tag text. Reproduced in a
new regression test.

Prod stack trace: AdifImport.parse_fields/1 crashed on upload with
binary_to_integer(">") -- a FreeDV/N1MM logger that embedded UTF-8
characters in a NOTES field triggered it.
2026-04-12 15:13:59 -05:00
eb1e80958c Normalize ADIF MODE/SUBMODE and document the mapping
- AdifImport.normalize_mode/2 maps ADIF MODE (and SUBMODE when present)
  onto the fixed set of modes the site accepts: CW/SSB/FM/FT8/FT4/Q65.
  SUBMODE takes precedence so `MODE=MFSK, SUBMODE=FT8` correctly imports
  as FT8. USB/LSB/DSB all become SSB; NBFM/WFM become FM.
- Unknown modes (RTTY, PSK31, etc.) map to nil so the row still imports
  — mode is optional on the submission path now.
- AdifImport.mode_mapping_reference/0 returns the user-facing list of
  recognized inputs and their mappings.
- Submit page's ADIF tab renders that list in a table so users can see
  upfront what will happen to their file before uploading.
2026-04-12 13:15:31 -05:00
6c334d6e18 Cache /contacts, fix map blink, make mode optional
- Add contacts_inserted_at_desc_id_desc_idx so /contacts list runs as an
  Index Scan (~0.4ms) instead of a Seq Scan + top-N heapsort (~77ms on
  58k rows)
- Add Microwaveprop.Cache: tiny ETS-backed TTL cache for memoizing
  expensive-but-stale-tolerant values
- Cache total_entries for unsearched /contacts page loads (30s TTL,
  invalidated on insert)
- Cache the entire /contacts/map payload (pre-encoded JSON + band list),
  moving load_contacts into Radio.contact_map_payload; invalidated on
  new contact insert. Mount goes from ~150-300ms to ~5ms.
- Fix /map overlay blinking on load: reuse the Leaflet GridLayer across
  renderScores calls and just redraw() against the updated ScoreGrid,
  instead of removeLayer + new layer which flashed between tile regens
- Make mode optional on submission: schema change (null: true),
  submission_changeset no longer requires it, blank strings normalise to
  nil, CSV import accepts 6-column (no mode) or 7-column layouts
- core_components .input adds a red asterisk to labels when required,
  giving users a visual cue for which fields must be filled on /submit
2026-04-12 12:47:25 -05:00
9abbb83469 Fix credo warnings: struct specs, length/1, and test patterns
- Replace %Struct{} with Struct.t() in all @spec annotations
- Replace length(x) > 0 with x != [] in test assertions
- Fix multi-line spec struct references in weather.ex
2026-04-12 10:26:53 -05:00
d5d0c32745 Use daisyUI components for stats, alerts, cards, and tabs
- about: stat cards → daisyUI stat, warning/donate boxes → alert
- submit: tab buttons → daisyUI tabs-box, info/instruction boxes → alert
- backfill: enrichment/DB stat boxes → daisyUI stat
- beacon show: stats container → daisyUI card
- contact show: info/status boxes → alert, data source cards → card
- ADIF: strip APP_ prefix from app-defined fields, normalize grid case
2026-04-12 09:20:20 -05:00
c3becdac9f Show beacon submitter callsign, add donate link, prefer 8-char grids
- Add belongs_to :user on Beacon schema, preload in get_beacon!/1
- Show "Submitted by {callsign}" on beacon detail page
- Add NTMS donation link on the about page
- Change grid preference text to 8 characters (e.g. EM12kp37)
2026-04-11 18:08:18 -05:00
5bcf579009 Add ADIF file upload for contact submission
Parse ADIF tagged-data format with fuzzy frequency-to-band matching
(nearest microwave band for any freq >= 900 MHz). Supports STATION_CALLSIGN,
OPERATOR, CALL, GRIDSQUARE, MY_GRIDSQUARE, FREQ, BAND, QSO_DATE, TIME_ON.
Same dedup and preview/commit flow as CSV upload.

Also fix select dropdown text alignment in daisyUI.
2026-04-11 17:39:05 -05:00
25c703401b Link to existing contact on duplicate submission
Return the conflicting contact from create_contact so the UI can redirect
to it with a flash message instead of just showing a form error.
2026-04-11 17:30:46 -05:00
cdb59be6d7 Deduplicate single contact submissions same as CSV import
Check for existing contacts with the same station pair (order-independent),
band, and grids within a 1-hour window before inserting.
2026-04-11 17:27:03 -05:00
4ca8c5d3bc Add Q65 contact mode 2026-04-11 17:21:45 -05:00
a66d3094ca Add contact edit approval system with admin review queue
Registered users can suggest edits to any contact's core fields
(callsigns, grids, band, mode, timestamp). Edits enter an admin
approval queue with field-by-field diff view. On approve, changes
are applied and enrichment re-enqueued if grids/band changed.
Users receive email notification on approve or reject.

Also updates dependabot.yml for mix ecosystem.
2026-04-11 16:15:49 -05:00
44b169b234 CSV upload preview with duplicate detection before commit
Uploads now show a review page with three summary cards (valid,
duplicates, invalid), tables for invalid and duplicate rows, and a
sample of the first 20 valid rows. Nothing is written to the database
until the user clicks "Looks good — submit N contacts".

Duplicate detection treats two rows as the same contact if they share
the same pair of (callsign, grid) tuples (direction-agnostic), the
same band, and timestamps within 60 minutes. Dedup runs against both
existing DB contacts (single band/time-scoped query) and earlier rows
in the same upload, and the preview labels which side matched.

CsvImport grows preview/2 + commit/1; the old import/2 is kept as a
thin backward-compat wrapper that skips dedup. Added 11 new CsvImport
tests and reworked the SubmitLive CSV tests for the two-step flow.
2026-04-09 12:30:10 -05:00
ee9275e0b9 Add /beacons CRUD and /users admin page
Beacons:
- Scaffolded with phx.gen.live then reworked so reads are public
  and mutations go through a live_session gated by the new
  :require_admin on_mount hook in UserAuth
- Beacon schema stores frequency (MHz), callsign, grid, lat/lon,
  power (W), and height above ground (m); grid auto-derives from
  lat/lon when left blank via Maidenhead.from_latlon
- Adds Maidenhead.from_latlon/3 so we can compute grids locally
  instead of hitting an external API

Users admin page:
- /users and /users/:id/edit (admin-only) for listing, editing
  (callsign/name/email/is_admin), and deleting other users
- Adds Accounts.list_users, admin_update_user, delete_user, and
  a dedicated admin_changeset on the User schema
- Nav gains a "Users" link for admins and a "Beacons" link for
  everyone
2026-04-08 12:01:45 -05:00
7e52c6660d Add Tier 1 test coverage: EnrichmentStatus, Markdown, health, RemoteIp, Maidenhead
EnrichmentStatus 0→100%, HealthController 0→100%, Maidenhead 74→96%,
RemoteIp 56→94%, Markdown 0→92%. Total coverage 53→55%.
2026-04-06 10:15:38 -05:00
d275e9e7c4
fix enrichment 2026-04-02 15:30:41 -05:00
93f2c32971
Remove timex, earmark, dns_cluster deps; add local implementations
- Replace Timex with regex-based timestamp parser (broader format support)
- Replace Earmark with local Markdown.to_html! (headings, code blocks,
  tables, lists, inline formatting, links, horizontal rules)
- Remove dns_cluster (unused, single-instance deployment)
- Add stream_data property tests for timestamp parsing (5 properties
  covering ISO, US dates, AM/PM, format roundtrips, garbage rejection)
- Removes transitive deps: combine, tzdata, hackney, certifi, metrics,
  mimerl, parse_trans, ssl_verify_fun
2026-04-02 12:17:37 -05:00
ec388e4bf5
Use Timex for flexible CSV timestamp parsing with tests
- Add timex ~> 3.7, downgrade gettext to ~> 0.26 for compatibility
- Parse ISO 8601, US dates (M/D/YYYY), AM/PM, 24h, compact formats
- 22 tests covering all accepted timestamp formats and edge cases
- Integration tests for CSV import with US date and space-separated formats
2026-04-01 15:39:02 -05:00
18a291555c
Add CSV bulk upload to /submit page
Adds a tabbed UI with single-contact form and CSV upload. Users can
download a sample CSV, upload their file, and get partial import with
per-row error reporting. Enrichment jobs enqueue for each imported contact.
2026-04-01 15:05:51 -05:00
254e64dedc
Rename qsos to contacts throughout codebase, keep DB table name
Rename all modules, functions, variables, routes, and UI text from
qso/qsos to contact/contacts. Database table stays as "qsos" to avoid
migration. Add /qsos -> /contacts redirects for old URLs.
2026-04-01 11:25:04 -05:00
bc529af392
Update algo.md Finding 10: corrected mode advantage, no SSB on rainscatter 2026-04-01 11:10:11 -05:00
7d0978f7d5
Add /submit route for user QSO submission
Maidenhead grid module converts grid squares to lat/lon coordinates.
Submission form validates grids, bands, modes, and email, computes
positions and distance, then triggers the weather/HRRR/terrain
processing pipeline via Oban.
2026-03-29 16:46:35 -05:00