Commit graph

95 commits

Author SHA1 Message Date
da75977a55
Rename qsos table to contacts, qso_id to contact_id
- Migration renames table, column, all indexes, and FK constraints
- Contact schema now references "contacts" table
- TerrainProfile foreign key renamed from qso_id to contact_id
- All code and tests updated to use contact_id
2026-04-02 15:52:19 -05:00
ae0e667cab
Fix backfill LiveStash recovery: always set all required assigns 2026-04-02 15:37:54 -05:00
d275e9e7c4
fix enrichment 2026-04-02 15:30:41 -05:00
15a09f0d7b
Fix enrichment: check actual data presence, not just status flag 2026-04-02 15:27:18 -05:00
f897a9a321
Re-enqueue enrichment jobs when status is queued but data missing
Previously, if a job was cancelled or lost while status was :queued,
viewing the contact detail page would not re-enqueue. Now HRRR,
weather/soundings, and terrain all re-enqueue when :queued with no
data, relying on Oban uniqueness for deduplication.
2026-04-02 15:07:49 -05:00
bb1c46fd57
Fix backfill page 500 by using pg_stat estimates instead of count(*)
The 6 sequential count(*) queries on large tables were timing out
at 15s when concurrent HRRR batch inserts held locks. Reuse the
n_live_tup estimates already fetched from pg_stat_user_tables.
2026-04-02 14:55:53 -05:00
66de041886
Show HRRR queue position on contact detail page
When HRRR data is loading, display how many jobs are ahead
in the queue so users can estimate wait time.
2026-04-02 14:42:45 -05:00
1c2879c466
Fix ambiguous column reference in backfill DB stats query 2026-04-02 13:29:15 -05:00
faa23a9693
Add Data Sources section to contact detail page
Shows provenance of all data used in the analysis:
- HRRR: path profile count, pressure levels, valid time, grid point
- Elevation: sample count, path distance, source (SRTM), duct layers
- Terrain: sample count, verdict, max elevation, diffraction loss
- Surface observations: station count and time range
- Soundings: profile count, stations, levels, ducting detection
- Solar indices: SFI, SSN, Ap
2026-04-02 13:28:58 -05:00
2452f1f397
Add detailed database stats to backfill dashboard
- Enrichment status breakdown per type (HRRR/Weather/Terrain/IEMRE)
  with color-coded status counts (complete/pending/queued/failed)
- Database overview: total DB size, HRRR profiles, observations,
  soundings with per-contact averages
- Table sizes: rows and disk size per table, sorted by size
- All stats auto-refresh every 2 seconds
2026-04-02 13:24:43 -05:00
59665a5009
Add LiveStash to backfill page, recover limit on reconnect 2026-04-02 13:13:23 -05:00
c1da0f20b1
Add LiveStash for state recovery across reconnects
- Installed live_stash ~> 0.1 with browser memory adapter
- Map page: recovers selected_band and selected_time on reconnect
- Submit page: recovers active_tab (single/csv) on reconnect
- JS params initialized with initLiveStash wrapper
2026-04-02 13:12:17 -05:00
2f65c4648c
Add /health endpoint and CHECKS file for zero-downtime deploys 2026-04-02 13:01:42 -05:00
20ed47397b
Improve map page mobile usability
Collapsible control panel with hamburger toggle, detail panel as
bottom sheet, compact legend moved to top-right, and scaled timeline
buttons for narrow viewports.
2026-04-02 12:59:55 -05:00
1da1cbed43
Fix nil access bugs found in security/bug audit
- TerrainProfileWorker: guard pos1/pos2 nil before accessing lat/lon,
  mark unavailable if missing instead of crashing
- Propagation analysis: guard pos1 nil in compute_factors
- contact_path_points: return [] instead of [{nil, nil}] on missing coords
- JS elevation hook: null check canvas before getContext
2026-04-02 12:38:49 -05:00
2b14e836f1
Fix CSV parsing and page number input validation
- RFC 4180 CSV parser: handles quoted fields with commas and escaped
  quotes ("Smith, John" no longer breaks parsing)
- Safe page number parsing: non-numeric input defaults to page 1
  instead of raising ArgumentError
2026-04-02 12:32:59 -05:00
79af4e4959
Ensure positions computed from grids before any enrichment
- Radio.ensure_positions!/1 computes pos1/pos2/distance from grids
- Called in contact show page mount and enqueue_for_contact
- Migration backfills positions for existing contacts missing them
- Migration corrects enrichment statuses based on actual data presence
- Backfill dashboard includes contacts with grids but no positions
2026-04-02 12:28:48 -05:00
93f2c32971
Remove timex, earmark, dns_cluster deps; add local implementations
- Replace Timex with regex-based timestamp parser (broader format support)
- Replace Earmark with local Markdown.to_html! (headings, code blocks,
  tables, lists, inline formatting, links, horizontal rules)
- Remove dns_cluster (unused, single-instance deployment)
- Add stream_data property tests for timestamp parsing (5 properties
  covering ISO, US dates, AM/PM, format roundtrips, garbage rejection)
- Removes transitive deps: combine, tzdata, hackney, certifi, metrics,
  mimerl, parse_trans, ssl_verify_fun
2026-04-02 12:17:37 -05:00
55a51f69ab
Replace boolean enrichment flags with enum status fields
- New fields: hrrr_status, weather_status, terrain_status, iemre_status
  with values: pending, queued, processing, complete, failed, unavailable
- EnrichmentStatus module defines valid state transitions
- Migration backfills: queued=true → complete, false → pending
- Workers set status on transitions (queued on enqueue, complete on finish)
- Show page reads status from contact struct, not computed dynamically
- Backfill dashboard queries status fields for accurate progress counts
- Remove cron-scheduled ContactWeatherEnqueueWorker (enrichment only on
  submission, page view, or manual backfill)
- Partial indexes on status != 'complete' for fast unprocessed lookups
2026-04-02 12:07:51 -05:00
fa5fabe02e
Fix stat card dotted border, use job queue counts for progress 2026-04-02 11:55:01 -05:00
4999548a0c
Show actual missing data counts instead of queued flags on backfill page 2026-04-02 11:50:47 -05:00
9485df4b8d
Read Cf-Connecting-Ip header for real client IP through Cloudflare tunnel 2026-04-02 11:49:11 -05:00
b6a5e709b7
Show enrichment progress bar with remaining/total contacts 2026-04-02 11:47:20 -05:00
c95da8ccdb
Add /backfill dashboard for enqueuing and monitoring enrichment jobs
- Input field to enqueue N contacts for backfill
- Live stats: unprocessed contacts, active/queued jobs, completed/hour
- Per-worker breakdown: executing (with spinner), available, retryable
- Auto-refreshes every 2 seconds
2026-04-02 11:46:49 -05:00
e13519bc3e
Add timing note to HRRR loading message 2026-04-02 11:07:23 -05:00
c42c5d8c4c
Enqueue weather (ASOS/RAOB) fetch on page view when missing 2026-04-02 11:02:05 -05:00
43f7bcf3c8
Recompute elevation profile with duct data when soundings arrive 2026-04-02 10:25:42 -05:00
17b66d6903
Live-update soundings and surface observations on contact page
- WeatherFetchWorker broadcasts on ASOS and RAOB completion
- Contact show page subscribes and reloads weather data on receipt
- Loading spinners for soundings and surface observations when queued
- Propagation analysis recomputed with new sounding duct data
2026-04-02 08:22:43 -05:00
a3ac8cff8d
Show loading spinners while enrichment jobs are processing 2026-04-02 08:19:52 -05:00
d207863b15
Live-update contact detail page when enrichment completes
- HRRR and terrain workers broadcast via PubSub on completion
- Contact show page subscribes and recomputes elevation profile,
  propagation analysis, and HRRR data on receipt
- Page updates in-place without reload when background jobs finish
2026-04-02 08:17:43 -05:00
0503fb3799
Re-enable HRRR auto-enqueue on contact detail page view 2026-04-02 08:08:28 -05:00
fd00de7bc6
Never download SRTM tiles during page load, only in Oban workers
- Srtm.lookup no longer auto-downloads by default, requires download: true
- TerrainProfileWorker passes download: true for background tile fetch
- Contact detail page enqueues TerrainProfileWorker if profile missing
- Page load gracefully handles missing tiles via API fallback or nil
2026-04-02 08:07:26 -05:00
ebd22ce156
Gracefully handle SRTM tile download failures on page load 2026-04-02 08:03:05 -05:00
20eb559bb0
Default contacts list sort to inserted_at desc 2026-04-01 16:00:21 -05:00
fd33eff8bc
Add required attribute to all single contact form fields 2026-04-01 15:56:25 -05:00
05654289af
Accept various date/time formats in CSV import
Parse US dates (6/15/2024 2:30 PM), 24h (06/15/2024 14:30),
dashes (06-15-2024 14:30), compact (20240615 1430), date-only
(2024-06-15), and ISO 8601 — all normalized to UTC before insert.
2026-04-01 15:35:35 -05:00
bc70b38e93
Add info box about contact availability after submission 2026-04-01 15:32:44 -05:00
4e092a0cf5
Use bulleted list for CSV upload instructions 2026-04-01 15:31:38 -05:00
2b11b872d9
Restyle submit page tabs as segmented control 2026-04-01 15:31:09 -05:00
bf90249a06
Format contact detail page via Styler 2026-04-01 15:06:02 -05:00
18a291555c
Add CSV bulk upload to /submit page
Adds a tabbed UI with single-contact form and CSV upload. Users can
download a sample CSV, upload their file, and get partial import with
per-row error reporting. Enrichment jobs enqueue for each imported contact.
2026-04-01 15:05:51 -05:00
86b6c0db31
Fix BadBooleanError when HRRR data is nil in propagation analysis 2026-04-01 14:06:49 -05:00
f02b111c1f
Disable auto HRRR backfill on contact detail page view 2026-04-01 13:57:23 -05:00
71091219f4
Add tidewave dev tool, taller elevation profile chart
- Install tidewave ~> 0.5 for dev MCP integration
- Increase elevation profile chart height from h-64 to h-96
2026-04-01 13:41:39 -05:00
dbc00989cd
Add propagation analysis with summary and factor breakdown
- Text summary explaining likely propagation mechanism (ducting,
  enhanced refraction, diffraction, troposcatter) based on terrain,
  HRRR, and sounding data
- 10-factor scoring table with per-factor notes, weights, contributions
- Composite score with tier badge
- Grid squares added back to path info line
- Analysis positioned between elevation chart and terrain profile
2026-04-01 13:38:10 -05:00
4aaa028ed1
Highlight likely propagation duct, improve contact detail layout
- Mark duct closest to endpoint elevations as likely signal path
- Likely duct drawn with solid borders, bold label, stronger fill
- Other ducts shown dimmed with dashed borders
- Fix path info layout: table columns prevent El wrapping
- Surface observations as compact table, limited to 5 nearest stations
2026-04-01 13:31:56 -05:00
5f8c689035
Improve duct detection and compact contact detail layout
- Use all HRRR profiles along path (start, mid, end) for duct detection
- Fall back to sounding ducts when HRRR too coarse to detect layers
- Merge overlapping ducts, show top 3 strongest with source labels
- Limit surface observations to 5 closest stations along path
- Compact solar conditions as inline row
- Compact band/mode/distance info line
- Fix button text to "Flag as Invalid"
2026-04-01 13:28:42 -05:00
7ba82f4601
Duct visualization, flag invalid contacts, fix HRRR pruning
- Overlay HRRR duct layers on elevation profile chart (green bands)
- Add flagged_invalid field to contacts with toggle button on detail page
- Fix prune_old_grid_profiles to only delete 0.125° grid profiles,
  preserving QSO-linked HRRR data at arbitrary positions
- Auto-enqueue HRRR fetch when viewing contact without HRRR data (<48h)
- Compact solar conditions and contact details layout
- Add band/mode/distance info line to elevation profile section
2026-04-01 13:09:09 -05:00
f7a753626d
Add interactive popups to contact map, dedup reciprocals, darken line colors 2026-04-01 12:39:02 -05:00
cd8d7a0f52
Add contact detail map, elevation profile chart, and UI improvements
- Leaflet map on contact detail pages showing both station locations
- SRTM elevation profile with Chart.js: terrain, LOS, Fresnel zone
- Path info: distance, azimuth, elevation angle in both directions
- Compact 4-column grid layout for contact details
- Contact map page with all contacts plotted
- Rename QSOs to Contacts in page header
- Wrap algo page in standard layout
2026-04-01 12:34:03 -05:00