Commit graph

26 commits

Author SHA1 Message Date
1d86e287b2 Add password auth with callsign + email confirmation
Generated Accounts context, User schema, and controllers via
phx.gen.auth. Adapted it for password-only login with required
email confirmation:

- Users have callsign (unique, uppercased), name, email, password
- Registration form fields: callsign, name, email, password, confirm
- Magic-link login path removed; login is email + password only
- After register, a confirmation email is sent and login is blocked
  until the account is confirmed via the token URL
- Confirmation link logs the user in on first use
- SMTP2GO configured as the outgoing mailer in k8s prod
2026-04-08 10:21:40 -05:00
55a51f69ab
Replace boolean enrichment flags with enum status fields
- New fields: hrrr_status, weather_status, terrain_status, iemre_status
  with values: pending, queued, processing, complete, failed, unavailable
- EnrichmentStatus module defines valid state transitions
- Migration backfills: queued=true → complete, false → pending
- Workers set status on transitions (queued on enqueue, complete on finish)
- Show page reads status from contact struct, not computed dynamically
- Backfill dashboard queries status fields for accurate progress counts
- Remove cron-scheduled ContactWeatherEnqueueWorker (enrichment only on
  submission, page view, or manual backfill)
- Partial indexes on status != 'complete' for fast unprocessed lookups
2026-04-02 12:07:51 -05:00
254e64dedc
Rename qsos to contacts throughout codebase, keep DB table name
Rename all modules, functions, variables, routes, and UI text from
qso/qsos to contact/contacts. Database table stays as "qsos" to avoid
migration. Add /qsos -> /contacts redirects for old URLs.
2026-04-01 11:25:04 -05:00
8949920b7f
Add Nx/Axon/EXLA ML model skeleton for propagation prediction
13-feature feed-forward network (atmospheric + temporal + frequency).
Includes build, init, predict, encode_features, save/load to disk.
Model weights saved to priv/models/propagation_v1.nx (gitignored).
Not yet trained — scaffolding only.
2026-03-31 16:26:34 -05:00
2da061201d
Disable ASOS adjustment cron, rely solely on HRRR hourly scoring
ASOS re-scoring was upserting 458K records every 10 min while losing
the refractivity gradient (the key ducting indicator). HRRR hourly
provides all surface variables plus vertical profile data.
2026-03-31 16:00:04 -05:00
ffae4b52e7
Fix HRRR fetch timeouts: 60s receive_timeout, reduce queue to 5
Req default 15s receive_timeout too short for GRIB2 byte-range
downloads on prod. Also reduce HRRR queue concurrency from 20 to 5
to avoid NOAA rate limiting (was causing burst-then-stall pattern).
2026-03-31 15:37:21 -05:00
6e5409bcd9
Update esbuild 0.27.4 and tailwind 4.2.2, add viewshed logging 2026-03-31 13:15:56 -05:00
45fb9736fa
Log client IP from X-Forwarded-For in request logs
Add RemoteIp plug that extracts real client IP from X-Forwarded-For
header (set by nginx/dokku proxy) and adds it to Logger metadata.
2026-03-31 12:05:05 -05:00
a6b4ac1965
Increase weather queue to 20 and iemre to 10 for faster backfill 2026-03-31 10:41:49 -05:00
e0d9900608
Add real-time ASOS adjustments between hourly HRRR updates
New AsosAdjustmentWorker runs every 10 minutes:
- Fetches latest ASOS observations from all ~2900 US stations via
  IEM bulk currents API (parallel fetch across 51 state networks)
- For each grid point within 75km of a reporting station, re-scores
  using fresh ASOS data (temp, dewpoint, wind, sky, pressure, precip)
  with HRRR refractivity gradient from the last hourly computation
- Pushes updated scores to the map via PubSub

Also stores HRRR profiles in the database during grid computation
so the data persists for reference and ASOS blending.
2026-03-31 09:10:12 -05:00
1e205cb471
Add propagation context, grid worker, and fix merge issues
- Replace stub propagation.ex with real implementation (score_grid_point,
  upsert_scores, latest_scores, latest_valid_time)
- Add PropagationGridWorker (hourly Oban cron) for CONUS grid scoring
- Add mix propagation_grid task for manual triggering
- Fix duplicate extract_grid/2 from parallel merges
- Fix extract_grid to skip outside-grid points instead of erroring
- Add propagation queue to Oban config
2026-03-30 17:18:05 -05:00
7f97c72f98
Increase weather enqueue frequency to every 30 minutes 2026-03-30 13:28:21 -05:00
3228835636
Add IEM precipitation data to QSO weather pipeline
- Add precip_1h_in and wx_codes fields to ASOS surface observations
- Add IEMRE reanalysis schema for radar-derived hourly precipitation
- Add IemreFetchWorker with exponential backoff and idempotency
- Integrate IEMRE enqueue into cron weather backfill pipeline
- All existing QSOs marked iemre_queued=false for automatic backfill
2026-03-30 13:18:11 -05:00
10e8feb486
Add commercial link monitoring via SNMP polling
Poll UBNT AirFiber radios (AF11X + AF60-LR) every 5 minutes via
net-snmp CLI, storing signal metrics in commercial_samples. Fetches
ASOS weather alongside each cycle for propagation correlation.

Includes 7 seeded link definitions, Oban cron worker, and net-snmp
in the Docker image.
2026-03-30 13:02:59 -05:00
8ccd778623
bump hrr downloads 2026-03-30 12:11:45 -05:00
9c25cff935
bump hrrr downloads 2026-03-30 10:37:55 -05:00
25bbaeefca
increase terrain workers 2026-03-30 10:28:33 -05:00
ead34990b0
Fix HRRR pipeline: single concurrency, cancel bad GRIB, rescue stuck jobs
- Reduce HRRR queue to 1 worker to avoid overwhelming upstream
- Add Oban Lifeline plugin to rescue orphaned executing jobs after 30m
- Treat GRIB2 decode errors (malformed section, missing sections, etc.)
  as permanent failures instead of retrying indefinitely
2026-03-30 09:16:07 -05:00
653efa450a
Reduce terrain queue concurrency to 1 to avoid 429 rate limits
Open-Meteo elevation API rate-limits when two terrain workers
fire concurrent requests.
2026-03-30 08:08:33 -05:00
45e2e69361
Add SRTM terrain path profiles for QSOs
Fetch elevation data along the path between two stations via the
Open-Meteo Elevation API (with Open-Topo-Data fallback), compute
Fresnel zone clearance, earth bulge, and knife-edge diffraction loss,
and store the results per QSO.

- terrain_profiles table with migration and TerrainProfile schema
- ElevationClient with batched API calls and fallback
- TerrainAnalysis with Fresnel/diffraction physics (ITU-R P.526-15)
- TerrainProfileWorker on Oban :terrain queue
- QsoWeatherEnqueueWorker enqueues terrain jobs automatically
- QSO show page displays verdict badge and collapsible elevation table
- Reorder show page: terrain, soundings, solar, HRRR, surface obs
- Fix Dockerfile wgrib2 build (add cmake dependency)
2026-03-29 16:22:29 -05:00
8bd0989fbe
Add HRRR model profile fetching for QSOs
HRRR provides hourly 3km-resolution atmospheric profiles, filling the
temporal gaps (12-hourly soundings) and spatial gaps (only 9 sounding
stations) in our current weather data.

- Add hrrr_profiles table and hrrr_queued flag on QSOs
- HrrrClient fetches GRIB2 data via HTTP Range requests + wgrib2
- HrrrFetchWorker derives refractivity/ducting params via SoundingParams
- QsoWeatherEnqueueWorker now also enqueues HRRR jobs
- QSO show page displays HRRR section with collapsible profile
- Dockerfile builds wgrib2 from source for production
2026-03-29 15:54:22 -05:00
c38596cc36
Add QsoWeatherEnqueueWorker cron job to auto-fetch weather for new QSOs
Oban cron worker runs every 4 hours, finds QSOs without weather data,
discovers nearby ASOS/sounding stations, and enqueues WeatherFetchWorker
jobs. Migrations run automatically on app start in production.
2026-03-29 14:11:03 -05:00
1008ce7fd2
Add Oban Pruner plugin to clean up completed jobs after 24 hours 2026-03-29 13:35:04 -05:00
5124cc09c1
Add IEM rate limiting, Req retry, and Oban retry for failed weather fetches
IEM was returning 503s under 10 concurrent workers. Three fixes:
- Req transient retry with exponential backoff on IEM requests
- WeatherFetchWorker Oban job retries failed ASOS/RAOB fetches 2-3h later
  with random jitter to avoid thundering herd
- Import script concurrency reduced 10→5, weather queue capped at 3
2026-03-29 13:24:52 -05:00
6f16395f44
Add QSO import, solar indices, Oban workers, LiveView UI, and parallel weather import
- Radio context with QSO schema and CSV import script (58K contacts)
- Solar index schema, GFZ client, and daily Oban cron worker
- LiveView dashboard with QSO table and weather correlation views
- Parallelize weather import script using Task.async_stream (10 concurrent workers)
- Replace sequential rate_limit sleeps with concurrency-based backpressure
- Atomic progress counter for interleave-safe reporting
2026-03-29 13:04:55 -05:00
bc674f0d54
initial 2026-03-28 11:28:47 -05:00