Commit graph

29 commits

Author SHA1 Message Date
51e390959c Add dialyzer specs and types across the codebase
277 @spec/@type annotations added to 58 files covering all public
APIs: contexts (propagation, radio, weather, terrain, beacons,
commercial), GRIB2 decoders, terrain analysis, duct detection,
rain scatter, CSV/ADIF import, weather clients, and all Ecto
schemas. Dialyzer passes with 0 errors.
2026-04-12 08:55:04 -05:00
c3becdac9f Show beacon submitter callsign, add donate link, prefer 8-char grids
- Add belongs_to :user on Beacon schema, preload in get_beacon!/1
- Show "Submitted by {callsign}" on beacon detail page
- Add NTMS donation link on the about page
- Change grid preference text to 8 characters (e.g. EM12kp37)
2026-04-11 18:08:18 -05:00
25c703401b Link to existing contact on duplicate submission
Return the conflicting contact from create_contact so the UI can redirect
to it with a flash message instead of just showing a form error.
2026-04-11 17:30:46 -05:00
cdb59be6d7 Deduplicate single contact submissions same as CSV import
Check for existing contacts with the same station pair (order-independent),
band, and grids within a 1-hour window before inserting.
2026-04-11 17:27:03 -05:00
08ffc7e028 Admin direct edit on contact show page, suggest edit for non-admins 2026-04-11 16:46:46 -05:00
a66d3094ca Add contact edit approval system with admin review queue
Registered users can suggest edits to any contact's core fields
(callsigns, grids, band, mode, timestamp). Edits enter an admin
approval queue with field-by-field diff view. On approve, changes
are applied and enrichment re-enqueued if grids/band changed.
Users receive email notification on approve or reject.

Also updates dependabot.yml for mix ecosystem.
2026-04-11 16:15:49 -05:00
a40ea0d62d Show submitter callsign on the contacts index
When a contact is linked to a logged-in user account, display that
user's callsign in the Submitted column instead of a generic "Yes".
Anonymous submissions still show "Yes"; scraped contacts show an em
dash.
2026-04-09 14:02:30 -05:00
7fd8f3165c
Add deterministic tiebreaker to contacts sort order 2026-04-05 09:27:34 -05:00
16883591b4
Database performance fixes and async backfill enqueue
- Fix score_pressure crash on nil pressure_mb (coastal HRRR points)
- Set 10-min timeout on grid score upsert transaction (was :infinity)
- Single DELETE for prune_old_scores instead of N queries in a loop
- Remove dead load_hrrr_refractivity that loaded 95k rows into nil map
- Pass selected_time to point_detail to skip latest_valid_time sub-query
- Batch station existence checks (1 query per path point, not per station)
- Batch solar index upserts via insert_all in chunks of 500
- Batch backfill_distances via single UPDATE FROM VALUES statement
- Add is_grid_point boolean + partial index to hrrr_profiles (replaces
  non-sargable modular arithmetic filter on every weather map query)
- Add partial index on contacts(qso_timestamp) WHERE pos1 IS NOT NULL
- Move backfill enqueue to Oban worker so UI returns immediately
2026-04-04 19:19:18 -05:00
dc66252fdc
Reduce dead tuples by using conditional upserts and skip-on-conflict
- HRRR, IEMRE, terrain: on_conflict: :nothing (immutable data)
- Surface obs, soundings, solar: conditional WHERE (only update when values differ)
- set_enrichment_status!: skip rows already at target status
2026-04-04 17:32:57 -05:00
4759c65809
Support two-callsign search on contacts page to find contacts between a pair 2026-04-04 09:29:32 -05:00
a8920e9480
Fix position computation, grid truncation, and status resets
- Truncate odd-length grids to nearest valid even length (FN03c → FN03)
- Handle lng/lon key inconsistency in haversine distance calculation
- Reset unavailable statuses to pending when positions are computed
- Rename backfill queue column to "Pending"
2026-04-02 16:31:34 -05:00
d275e9e7c4
fix enrichment 2026-04-02 15:30:41 -05:00
1da1cbed43
Fix nil access bugs found in security/bug audit
- TerrainProfileWorker: guard pos1/pos2 nil before accessing lat/lon,
  mark unavailable if missing instead of crashing
- Propagation analysis: guard pos1 nil in compute_factors
- contact_path_points: return [] instead of [{nil, nil}] on missing coords
- JS elevation hook: null check canvas before getContext
2026-04-02 12:38:49 -05:00
79af4e4959
Ensure positions computed from grids before any enrichment
- Radio.ensure_positions!/1 computes pos1/pos2/distance from grids
- Called in contact show page mount and enqueue_for_contact
- Migration backfills positions for existing contacts missing them
- Migration corrects enrichment statuses based on actual data presence
- Backfill dashboard includes contacts with grids but no positions
2026-04-02 12:28:48 -05:00
55a51f69ab
Replace boolean enrichment flags with enum status fields
- New fields: hrrr_status, weather_status, terrain_status, iemre_status
  with values: pending, queued, processing, complete, failed, unavailable
- EnrichmentStatus module defines valid state transitions
- Migration backfills: queued=true → complete, false → pending
- Workers set status on transitions (queued on enqueue, complete on finish)
- Show page reads status from contact struct, not computed dynamically
- Backfill dashboard queries status fields for accurate progress counts
- Remove cron-scheduled ContactWeatherEnqueueWorker (enrichment only on
  submission, page view, or manual backfill)
- Partial indexes on status != 'complete' for fast unprocessed lookups
2026-04-02 12:07:51 -05:00
7ba82f4601
Duct visualization, flag invalid contacts, fix HRRR pruning
- Overlay HRRR duct layers on elevation profile chart (green bands)
- Add flagged_invalid field to contacts with toggle button on detail page
- Fix prune_old_grid_profiles to only delete 0.125° grid profiles,
  preserving QSO-linked HRRR data at arbitrary positions
- Auto-enqueue HRRR fetch when viewing contact without HRRR data (<48h)
- Compact solar conditions and contact details layout
- Add band/mode/distance info line to elevation profile section
2026-04-01 13:09:09 -05:00
254e64dedc
Rename qsos to contacts throughout codebase, keep DB table name
Rename all modules, functions, variables, routes, and UI text from
qso/qsos to contact/contacts. Database table stays as "qsos" to avoid
migration. Add /qsos -> /contacts redirects for old URLs.
2026-04-01 11:25:04 -05:00
02cb4fd67b
Integrate ML model into grid worker, QSO search, and UI improvements
ML Integration:
- Load trained model at app startup, cache compiled predict fn in persistent_term
- Grid worker uses batched ML prediction (10K chunks) when model loaded,
  falls back to algorithm scorer when not
- ML score replaces composite, algorithm factor scores preserved for detail view
- Fix process explosion: single EXLA call per chunk instead of per-grid-point

QSO Features:
- Callsign search (ILIKE on station1/station2) with trigram indexes
- Reciprocal QSO grouping (same pair, same band, same hour)
- Wider layout (max-w-7xl) for data table pages
- QSO Training Data link on map page

Infrastructure:
- Re-enable hourly propagation grid worker in dev
- Track ML model weights in git for Docker builds
- Add btree indexes on qsos (timestamp, band, distance_km)
- Remove nav icons from layout header
2026-04-01 10:14:22 -05:00
ded9c054ac
Multi-point path enrichment for HRRR, IEMRE, and weather data
Enqueue worker now gathers atmospheric data at pos1, midpoint, and pos2
along each QSO path instead of only pos1. Existing has_* guards prevent
duplicate fetches at each grid point.

- Add Radio.qso_path_points/1 for path point extraction
- Update hrrr_job_for_qso, iemre_job_for_qso, jobs_for_qso to iterate path points
- Refactor Weather into find_nearest_hrrr/3 and find_nearest_iemre/3
- Add hrrr_profiles_for_path/1 and iemre_for_path/1 query functions
- Add mix reset_enrichment task to trigger re-processing
2026-03-30 16:09:01 -05:00
3228835636
Add IEM precipitation data to QSO weather pipeline
- Add precip_1h_in and wx_codes fields to ASOS surface observations
- Add IEMRE reanalysis schema for radar-derived hourly precipitation
- Add IemreFetchWorker with exponential backoff and idempotency
- Integrate IEMRE enqueue into cron weather backfill pipeline
- All existing QSOs marked iemre_queued=false for automatic backfill
2026-03-30 13:18:11 -05:00
47456630ff
Add Committed column to QSOs table showing inserted_at 2026-03-29 17:38:35 -05:00
fe3c8812a0
Harden /submit route for production
- Defensive grid-to-coordinate resolution in create_qso (case instead of crash on match failure)
- Log warning when Oban weather enqueue fails instead of silently dropping
- Add 30s per-session submission cooldown to prevent rapid-fire submissions
2026-03-29 17:24:12 -05:00
7d0978f7d5
Add /submit route for user QSO submission
Maidenhead grid module converts grid squares to lat/lon coordinates.
Submission form validates grids, bands, modes, and email, computes
positions and distance, then triggers the weather/HRRR/terrain
processing pipeline via Oban.
2026-03-29 16:46:35 -05:00
45e2e69361
Add SRTM terrain path profiles for QSOs
Fetch elevation data along the path between two stations via the
Open-Meteo Elevation API (with Open-Topo-Data fallback), compute
Fresnel zone clearance, earth bulge, and knife-edge diffraction loss,
and store the results per QSO.

- terrain_profiles table with migration and TerrainProfile schema
- ElevationClient with batched API calls and fallback
- TerrainAnalysis with Fresnel/diffraction physics (ITU-R P.526-15)
- TerrainProfileWorker on Oban :terrain queue
- QsoWeatherEnqueueWorker enqueues terrain jobs automatically
- QSO show page displays verdict badge and collapsible elevation table
- Reorder show page: terrain, soundings, solar, HRRR, surface obs
- Fix Dockerfile wgrib2 build (add cmake dependency)
2026-03-29 16:22:29 -05:00
8bd0989fbe
Add HRRR model profile fetching for QSOs
HRRR provides hourly 3km-resolution atmospheric profiles, filling the
temporal gaps (12-hourly soundings) and spatial gaps (only 9 sounding
stations) in our current weather data.

- Add hrrr_profiles table and hrrr_queued flag on QSOs
- HrrrClient fetches GRIB2 data via HTTP Range requests + wgrib2
- HrrrFetchWorker derives refractivity/ducting params via SoundingParams
- QsoWeatherEnqueueWorker now also enqueues HRRR jobs
- QSO show page displays HRRR section with collapsible profile
- Dockerfile builds wgrib2 from source for production
2026-03-29 15:54:22 -05:00
d50c78102d
Backfill missing QSO distances during weather enqueue scan
Adds haversine_km/4 to Radio context and backfill_distances/1 which
calculates distance_km from pos1/pos2 for QSOs missing it. Called
by QsoWeatherEnqueueWorker before enqueueing weather fetches.
2026-03-29 14:16:38 -05:00
c38596cc36
Add QsoWeatherEnqueueWorker cron job to auto-fetch weather for new QSOs
Oban cron worker runs every 4 hours, finds QSOs without weather data,
discovers nearby ASOS/sounding stations, and enqueues WeatherFetchWorker
jobs. Migrations run automatically on app start in production.
2026-03-29 14:11:03 -05:00
6f16395f44
Add QSO import, solar indices, Oban workers, LiveView UI, and parallel weather import
- Radio context with QSO schema and CSV import script (58K contacts)
- Solar index schema, GFZ client, and daily Oban cron worker
- LiveView dashboard with QSO table and weather correlation views
- Parallelize weather import script using Task.async_stream (10 concurrent workers)
- Replace sequential rate_limit sleeps with concurrency-based backpressure
- Atomic progress counter for interleave-safe reporting
2026-03-29 13:04:55 -05:00