Commit graph

30 commits

Author SHA1 Message Date
51e390959c Add dialyzer specs and types across the codebase
277 @spec/@type annotations added to 58 files covering all public
APIs: contexts (propagation, radio, weather, terrain, beacons,
commercial), GRIB2 decoders, terrain analysis, duct detection,
rain scatter, CSV/ADIF import, weather clients, and all Ecto
schemas. Dialyzer passes with 0 errors.
2026-04-12 08:55:04 -05:00
c3becdac9f Show beacon submitter callsign, add donate link, prefer 8-char grids
- Add belongs_to :user on Beacon schema, preload in get_beacon!/1
- Show "Submitted by {callsign}" on beacon detail page
- Add NTMS donation link on the about page
- Change grid preference text to 8 characters (e.g. EM12kp37)
2026-04-11 18:08:18 -05:00
5bcf579009 Add ADIF file upload for contact submission
Parse ADIF tagged-data format with fuzzy frequency-to-band matching
(nearest microwave band for any freq >= 900 MHz). Supports STATION_CALLSIGN,
OPERATOR, CALL, GRIDSQUARE, MY_GRIDSQUARE, FREQ, BAND, QSO_DATE, TIME_ON.
Same dedup and preview/commit flow as CSV upload.

Also fix select dropdown text alignment in daisyUI.
2026-04-11 17:39:05 -05:00
4ca8c5d3bc Add Q65 contact mode 2026-04-11 17:21:45 -05:00
a66d3094ca Add contact edit approval system with admin review queue
Registered users can suggest edits to any contact's core fields
(callsigns, grids, band, mode, timestamp). Edits enter an admin
approval queue with field-by-field diff view. On approve, changes
are applied and enrichment re-enqueued if grids/band changed.
Users receive email notification on approve or reject.

Also updates dependabot.yml for mix ecosystem.
2026-04-11 16:15:49 -05:00
a40ea0d62d Show submitter callsign on the contacts index
When a contact is linked to a logged-in user account, display that
user's callsign in the Submitted column instead of a generic "Yes".
Anonymous submissions still show "Yes"; scraped contacts show an em
dash.
2026-04-09 14:02:30 -05:00
5724b33e8f Link submissions to the logged-in user on /submit
Contacts now carry a user_id FK to users. When a logged-in user
submits a contact (single or CSV upload), the server injects their
user_id and the email from their account into the submission params,
and the visible email input is replaced with a hidden input holding
that value — logged-in users don't have to retype their address.

The submission changeset now requires at least one of user_id or
submitter_email (rather than hard-requiring email), and email format
is only validated when one is actually present.

Reworded the submit-page blurb from "10 GHz to 241 GHz" to "902 MHz
and up" so it matches the actual band coverage of the project.
2026-04-09 13:01:49 -05:00
44b169b234 CSV upload preview with duplicate detection before commit
Uploads now show a review page with three summary cards (valid,
duplicates, invalid), tables for invalid and duplicate rows, and a
sample of the first 20 valid rows. Nothing is written to the database
until the user clicks "Looks good — submit N contacts".

Duplicate detection treats two rows as the same contact if they share
the same pair of (callsign, grid) tuples (direction-agnostic), the
same band, and timestamps within 60 minutes. Dedup runs against both
existing DB contacts (single band/time-scoped query) and earlier rows
in the same upload, and the preview labels which side matched.

CsvImport grows preview/2 + commit/1; the old import/2 is kept as a
thin backward-compat wrapper that skips dedup. Added 11 new CsvImport
tests and reworked the SubmitLive CSV tests for the two-step flow.
2026-04-09 12:30:10 -05:00
ee9275e0b9 Add /beacons CRUD and /users admin page
Beacons:
- Scaffolded with phx.gen.live then reworked so reads are public
  and mutations go through a live_session gated by the new
  :require_admin on_mount hook in UserAuth
- Beacon schema stores frequency (MHz), callsign, grid, lat/lon,
  power (W), and height above ground (m); grid auto-derives from
  lat/lon when left blank via Maidenhead.from_latlon
- Adds Maidenhead.from_latlon/3 so we can compute grids locally
  instead of hitting an external API

Users admin page:
- /users and /users/:id/edit (admin-only) for listing, editing
  (callsign/name/email/is_admin), and deleting other users
- Adds Accounts.list_users, admin_update_user, delete_user, and
  a dedicated admin_changeset on the User schema
- Nav gains a "Users" link for admins and a "Beacons" link for
  everyone
2026-04-08 12:01:45 -05:00
95351a2358 Add /path calculator page
LiveView page for analyzing microwave propagation between two points.
Features:
- Source/destination input: callsign (via gridmap.org API) or Maidenhead grid
- Band selector for all 8 microwave bands (10-241 GHz)
- Leaflet map showing path line (reuses ContactMap hook)
- Terrain elevation cross-section (reuses ElevationProfile hook)
- Link summary: distance, bearing, terrain verdict
- Loss budget: FSPL + O2 + H2O + rain + diffraction
- Propagation score with 10-factor breakdown
- Current atmospheric conditions from path-integrated HRRR

New CallsignClient for gridmap.org location lookup.
Navigation links added to main navbar and map control panel.
2026-04-07 13:25:04 -05:00
da75977a55
Rename qsos table to contacts, qso_id to contact_id
- Migration renames table, column, all indexes, and FK constraints
- Contact schema now references "contacts" table
- TerrainProfile foreign key renamed from qso_id to contact_id
- All code and tests updated to use contact_id
2026-04-02 15:52:19 -05:00
d275e9e7c4
fix enrichment 2026-04-02 15:30:41 -05:00
2b14e836f1
Fix CSV parsing and page number input validation
- RFC 4180 CSV parser: handles quoted fields with commas and escaped
  quotes ("Smith, John" no longer breaks parsing)
- Safe page number parsing: non-numeric input defaults to page 1
  instead of raising ArgumentError
2026-04-02 12:32:59 -05:00
93f2c32971
Remove timex, earmark, dns_cluster deps; add local implementations
- Replace Timex with regex-based timestamp parser (broader format support)
- Replace Earmark with local Markdown.to_html! (headings, code blocks,
  tables, lists, inline formatting, links, horizontal rules)
- Remove dns_cluster (unused, single-instance deployment)
- Add stream_data property tests for timestamp parsing (5 properties
  covering ISO, US dates, AM/PM, format roundtrips, garbage rejection)
- Removes transitive deps: combine, tzdata, hackney, certifi, metrics,
  mimerl, parse_trans, ssl_verify_fun
2026-04-02 12:17:37 -05:00
55a51f69ab
Replace boolean enrichment flags with enum status fields
- New fields: hrrr_status, weather_status, terrain_status, iemre_status
  with values: pending, queued, processing, complete, failed, unavailable
- EnrichmentStatus module defines valid state transitions
- Migration backfills: queued=true → complete, false → pending
- Workers set status on transitions (queued on enqueue, complete on finish)
- Show page reads status from contact struct, not computed dynamically
- Backfill dashboard queries status fields for accurate progress counts
- Remove cron-scheduled ContactWeatherEnqueueWorker (enrichment only on
  submission, page view, or manual backfill)
- Partial indexes on status != 'complete' for fast unprocessed lookups
2026-04-02 12:07:51 -05:00
737de10bf4
Support arbitrary-precision Maidenhead grid squares
Extend grid validation and decoding from 4/6 chars to unlimited
even-length grids (8, 10, etc). Recursive decoder handles alternating
letter(/24) and digit(/10) subdivision pairs after the field+square.
Validated against gridmap-web's Maidenhead implementation.
2026-04-02 07:31:52 -05:00
a25864bdcb
Sanitize and validate user-submitted contact fields
- Callsigns: strip whitespace, upcase, validate alphanumeric + /
- Grids: strip whitespace, upcase before Maidenhead validation
- Email: stricter regex, max 254 chars
- Callsign max length 20 chars
- Fix static_paths to include downloads directory
- Update show test for removed contact details section
2026-04-01 15:41:42 -05:00
ec388e4bf5
Use Timex for flexible CSV timestamp parsing with tests
- Add timex ~> 3.7, downgrade gettext to ~> 0.26 for compatibility
- Parse ISO 8601, US dates (M/D/YYYY), AM/PM, 24h, compact formats
- 22 tests covering all accepted timestamp formats and edge cases
- Integration tests for CSV import with US date and space-separated formats
2026-04-01 15:39:02 -05:00
05654289af
Accept various date/time formats in CSV import
Parse US dates (6/15/2024 2:30 PM), 24h (06/15/2024 14:30),
dashes (06-15-2024 14:30), compact (20240615 1430), date-only
(2024-06-15), and ISO 8601 — all normalized to UTC before insert.
2026-04-01 15:35:35 -05:00
18a291555c
Add CSV bulk upload to /submit page
Adds a tabbed UI with single-contact form and CSV upload. Users can
download a sample CSV, upload their file, and get partial import with
per-row error reporting. Enrichment jobs enqueue for each imported contact.
2026-04-01 15:05:51 -05:00
7ba82f4601
Duct visualization, flag invalid contacts, fix HRRR pruning
- Overlay HRRR duct layers on elevation profile chart (green bands)
- Add flagged_invalid field to contacts with toggle button on detail page
- Fix prune_old_grid_profiles to only delete 0.125° grid profiles,
  preserving QSO-linked HRRR data at arbitrary positions
- Auto-enqueue HRRR fetch when viewing contact without HRRR data (<48h)
- Compact solar conditions and contact details layout
- Add band/mode/distance info line to elevation profile section
2026-04-01 13:09:09 -05:00
ea96c93e2d
Rename QSO to Contact in UI, add higher bands, improve submit page
- All menus and UI text now say Contacts instead of QSOs
- Add 68, 122, 134, 241 GHz bands to submit form and validation
- Add info box on submit page explaining why contacts matter
- Larger submit button with icon
- Make HRRR partition migration idempotent for partial re-runs
2026-04-01 12:28:24 -05:00
254e64dedc
Rename qsos to contacts throughout codebase, keep DB table name
Rename all modules, functions, variables, routes, and UI text from
qso/qsos to contact/contacts. Database table stays as "qsos" to avoid
migration. Add /qsos -> /contacts redirects for old URLs.
2026-04-01 11:25:04 -05:00
bc529af392
Update algo.md Finding 10: corrected mode advantage, no SSB on rainscatter 2026-04-01 11:10:11 -05:00
3228835636
Add IEM precipitation data to QSO weather pipeline
- Add precip_1h_in and wx_codes fields to ASOS surface observations
- Add IEMRE reanalysis schema for radar-derived hourly precipitation
- Add IemreFetchWorker with exponential backoff and idempotency
- Integrate IEMRE enqueue into cron weather backfill pipeline
- All existing QSOs marked iemre_queued=false for automatic backfill
2026-03-30 13:18:11 -05:00
7d0978f7d5
Add /submit route for user QSO submission
Maidenhead grid module converts grid squares to lat/lon coordinates.
Submission form validates grids, bands, modes, and email, computes
positions and distance, then triggers the weather/HRRR/terrain
processing pipeline via Oban.
2026-03-29 16:46:35 -05:00
45e2e69361
Add SRTM terrain path profiles for QSOs
Fetch elevation data along the path between two stations via the
Open-Meteo Elevation API (with Open-Topo-Data fallback), compute
Fresnel zone clearance, earth bulge, and knife-edge diffraction loss,
and store the results per QSO.

- terrain_profiles table with migration and TerrainProfile schema
- ElevationClient with batched API calls and fallback
- TerrainAnalysis with Fresnel/diffraction physics (ITU-R P.526-15)
- TerrainProfileWorker on Oban :terrain queue
- QsoWeatherEnqueueWorker enqueues terrain jobs automatically
- QSO show page displays verdict badge and collapsible elevation table
- Reorder show page: terrain, soundings, solar, HRRR, surface obs
- Fix Dockerfile wgrib2 build (add cmake dependency)
2026-03-29 16:22:29 -05:00
8bd0989fbe
Add HRRR model profile fetching for QSOs
HRRR provides hourly 3km-resolution atmospheric profiles, filling the
temporal gaps (12-hourly soundings) and spatial gaps (only 9 sounding
stations) in our current weather data.

- Add hrrr_profiles table and hrrr_queued flag on QSOs
- HrrrClient fetches GRIB2 data via HTTP Range requests + wgrib2
- HrrrFetchWorker derives refractivity/ducting params via SoundingParams
- QsoWeatherEnqueueWorker now also enqueues HRRR jobs
- QSO show page displays HRRR section with collapsible profile
- Dockerfile builds wgrib2 from source for production
2026-03-29 15:54:22 -05:00
c38596cc36
Add QsoWeatherEnqueueWorker cron job to auto-fetch weather for new QSOs
Oban cron worker runs every 4 hours, finds QSOs without weather data,
discovers nearby ASOS/sounding stations, and enqueues WeatherFetchWorker
jobs. Migrations run automatically on app start in production.
2026-03-29 14:11:03 -05:00
6f16395f44
Add QSO import, solar indices, Oban workers, LiveView UI, and parallel weather import
- Radio context with QSO schema and CSV import script (58K contacts)
- Solar index schema, GFZ client, and daily Oban cron worker
- LiveView dashboard with QSO table and weather correlation views
- Parallelize weather import script using Task.async_stream (10 concurrent workers)
- Replace sequential rate_limit sleeps with concurrency-based backpressure
- Atomic progress counter for interleave-safe reporting
2026-03-29 13:04:55 -05:00