Commit graph

20 commits

Author SHA1 Message Date
da75977a55
Rename qsos table to contacts, qso_id to contact_id
- Migration renames table, column, all indexes, and FK constraints
- Contact schema now references "contacts" table
- TerrainProfile foreign key renamed from qso_id to contact_id
- All code and tests updated to use contact_id
2026-04-02 15:52:19 -05:00
d275e9e7c4
fix enrichment 2026-04-02 15:30:41 -05:00
2b14e836f1
Fix CSV parsing and page number input validation
- RFC 4180 CSV parser: handles quoted fields with commas and escaped
  quotes ("Smith, John" no longer breaks parsing)
- Safe page number parsing: non-numeric input defaults to page 1
  instead of raising ArgumentError
2026-04-02 12:32:59 -05:00
93f2c32971
Remove timex, earmark, dns_cluster deps; add local implementations
- Replace Timex with regex-based timestamp parser (broader format support)
- Replace Earmark with local Markdown.to_html! (headings, code blocks,
  tables, lists, inline formatting, links, horizontal rules)
- Remove dns_cluster (unused, single-instance deployment)
- Add stream_data property tests for timestamp parsing (5 properties
  covering ISO, US dates, AM/PM, format roundtrips, garbage rejection)
- Removes transitive deps: combine, tzdata, hackney, certifi, metrics,
  mimerl, parse_trans, ssl_verify_fun
2026-04-02 12:17:37 -05:00
55a51f69ab
Replace boolean enrichment flags with enum status fields
- New fields: hrrr_status, weather_status, terrain_status, iemre_status
  with values: pending, queued, processing, complete, failed, unavailable
- EnrichmentStatus module defines valid state transitions
- Migration backfills: queued=true → complete, false → pending
- Workers set status on transitions (queued on enqueue, complete on finish)
- Show page reads status from contact struct, not computed dynamically
- Backfill dashboard queries status fields for accurate progress counts
- Remove cron-scheduled ContactWeatherEnqueueWorker (enrichment only on
  submission, page view, or manual backfill)
- Partial indexes on status != 'complete' for fast unprocessed lookups
2026-04-02 12:07:51 -05:00
737de10bf4
Support arbitrary-precision Maidenhead grid squares
Extend grid validation and decoding from 4/6 chars to unlimited
even-length grids (8, 10, etc). Recursive decoder handles alternating
letter(/24) and digit(/10) subdivision pairs after the field+square.
Validated against gridmap-web's Maidenhead implementation.
2026-04-02 07:31:52 -05:00
a25864bdcb
Sanitize and validate user-submitted contact fields
- Callsigns: strip whitespace, upcase, validate alphanumeric + /
- Grids: strip whitespace, upcase before Maidenhead validation
- Email: stricter regex, max 254 chars
- Callsign max length 20 chars
- Fix static_paths to include downloads directory
- Update show test for removed contact details section
2026-04-01 15:41:42 -05:00
ec388e4bf5
Use Timex for flexible CSV timestamp parsing with tests
- Add timex ~> 3.7, downgrade gettext to ~> 0.26 for compatibility
- Parse ISO 8601, US dates (M/D/YYYY), AM/PM, 24h, compact formats
- 22 tests covering all accepted timestamp formats and edge cases
- Integration tests for CSV import with US date and space-separated formats
2026-04-01 15:39:02 -05:00
05654289af
Accept various date/time formats in CSV import
Parse US dates (6/15/2024 2:30 PM), 24h (06/15/2024 14:30),
dashes (06-15-2024 14:30), compact (20240615 1430), date-only
(2024-06-15), and ISO 8601 — all normalized to UTC before insert.
2026-04-01 15:35:35 -05:00
18a291555c
Add CSV bulk upload to /submit page
Adds a tabbed UI with single-contact form and CSV upload. Users can
download a sample CSV, upload their file, and get partial import with
per-row error reporting. Enrichment jobs enqueue for each imported contact.
2026-04-01 15:05:51 -05:00
7ba82f4601
Duct visualization, flag invalid contacts, fix HRRR pruning
- Overlay HRRR duct layers on elevation profile chart (green bands)
- Add flagged_invalid field to contacts with toggle button on detail page
- Fix prune_old_grid_profiles to only delete 0.125° grid profiles,
  preserving QSO-linked HRRR data at arbitrary positions
- Auto-enqueue HRRR fetch when viewing contact without HRRR data (<48h)
- Compact solar conditions and contact details layout
- Add band/mode/distance info line to elevation profile section
2026-04-01 13:09:09 -05:00
ea96c93e2d
Rename QSO to Contact in UI, add higher bands, improve submit page
- All menus and UI text now say Contacts instead of QSOs
- Add 68, 122, 134, 241 GHz bands to submit form and validation
- Add info box on submit page explaining why contacts matter
- Larger submit button with icon
- Make HRRR partition migration idempotent for partial re-runs
2026-04-01 12:28:24 -05:00
254e64dedc
Rename qsos to contacts throughout codebase, keep DB table name
Rename all modules, functions, variables, routes, and UI text from
qso/qsos to contact/contacts. Database table stays as "qsos" to avoid
migration. Add /qsos -> /contacts redirects for old URLs.
2026-04-01 11:25:04 -05:00
bc529af392
Update algo.md Finding 10: corrected mode advantage, no SSB on rainscatter 2026-04-01 11:10:11 -05:00
3228835636
Add IEM precipitation data to QSO weather pipeline
- Add precip_1h_in and wx_codes fields to ASOS surface observations
- Add IEMRE reanalysis schema for radar-derived hourly precipitation
- Add IemreFetchWorker with exponential backoff and idempotency
- Integrate IEMRE enqueue into cron weather backfill pipeline
- All existing QSOs marked iemre_queued=false for automatic backfill
2026-03-30 13:18:11 -05:00
7d0978f7d5
Add /submit route for user QSO submission
Maidenhead grid module converts grid squares to lat/lon coordinates.
Submission form validates grids, bands, modes, and email, computes
positions and distance, then triggers the weather/HRRR/terrain
processing pipeline via Oban.
2026-03-29 16:46:35 -05:00
45e2e69361
Add SRTM terrain path profiles for QSOs
Fetch elevation data along the path between two stations via the
Open-Meteo Elevation API (with Open-Topo-Data fallback), compute
Fresnel zone clearance, earth bulge, and knife-edge diffraction loss,
and store the results per QSO.

- terrain_profiles table with migration and TerrainProfile schema
- ElevationClient with batched API calls and fallback
- TerrainAnalysis with Fresnel/diffraction physics (ITU-R P.526-15)
- TerrainProfileWorker on Oban :terrain queue
- QsoWeatherEnqueueWorker enqueues terrain jobs automatically
- QSO show page displays verdict badge and collapsible elevation table
- Reorder show page: terrain, soundings, solar, HRRR, surface obs
- Fix Dockerfile wgrib2 build (add cmake dependency)
2026-03-29 16:22:29 -05:00
8bd0989fbe
Add HRRR model profile fetching for QSOs
HRRR provides hourly 3km-resolution atmospheric profiles, filling the
temporal gaps (12-hourly soundings) and spatial gaps (only 9 sounding
stations) in our current weather data.

- Add hrrr_profiles table and hrrr_queued flag on QSOs
- HrrrClient fetches GRIB2 data via HTTP Range requests + wgrib2
- HrrrFetchWorker derives refractivity/ducting params via SoundingParams
- QsoWeatherEnqueueWorker now also enqueues HRRR jobs
- QSO show page displays HRRR section with collapsible profile
- Dockerfile builds wgrib2 from source for production
2026-03-29 15:54:22 -05:00
c38596cc36
Add QsoWeatherEnqueueWorker cron job to auto-fetch weather for new QSOs
Oban cron worker runs every 4 hours, finds QSOs without weather data,
discovers nearby ASOS/sounding stations, and enqueues WeatherFetchWorker
jobs. Migrations run automatically on app start in production.
2026-03-29 14:11:03 -05:00
6f16395f44
Add QSO import, solar indices, Oban workers, LiveView UI, and parallel weather import
- Radio context with QSO schema and CSV import script (58K contacts)
- Solar index schema, GFZ client, and daily Oban cron worker
- LiveView dashboard with QSO table and weather correlation views
- Parallelize weather import script using Task.async_stream (10 concurrent workers)
- Replace sequential rate_limit sleeps with concurrency-based backpressure
- Atomic progress counter for interleave-safe reporting
2026-03-29 13:04:55 -05:00