prop/priv/repo/migrations
Graham McIntire 99e7560601
Drop gridmap.org dependency, resolve callsigns locally
Instead of shelling out to https://gridmap.org/locate/:callsign for
callsign → lat/lon lookups, ports the resolver pipeline from gridmap-web
into this project so the whole flow runs in-process.

New modules:

- Microwaveprop.Qrz — cache facade over the QRZ.com XML callsign API.
  Looks up from qrz_callsigns first, falls back to a live fetch, and
  upserts the result with a configurable cache_ttl_hours (default
  168h / 7 days).
- Microwaveprop.Qrz.Client — HTTP/XML client against
  https://xmldata.qrz.com/xml/current/. Holds the session key in an
  Agent, transparently re-logs-in on :session_expired, and parses
  responses via xmerl.
- Microwaveprop.Qrz.Callsign — Ecto schema for the qrz_callsigns
  cache table, binary_id primary key per project convention.
- Microwaveprop.Qrz.Record — slim struct with only the 11 fields
  we actually consume (identity, name, grid, address, lat/lon).
  The full XML payload stays in the raw :data jsonb column for
  anyone who wants the other ~40 QRZ fields.
- Microwaveprop.Geocoder — Req-based client against the Google Maps
  Geocoding API. Only called as a fallback when QRZ has no explicit
  <lat>/<lon> for the callsign.
- Microwaveprop.CallsignLocation — orchestrator. Reads the
  callsign_locations cache, on miss calls Qrz then either uses QRZ's
  coords directly or geocodes the formatted address, snaps to an
  8-char Maidenhead grid via Microwaveprop.Radio.Maidenhead, and
  upserts the result.

Microwaveprop.Radio.CallsignClient.locate/1 is rewritten to delegate
to CallsignLocation.lookup/1 and shape the response back to the
existing {:ok, %{callsign, gridsquare, lat, lon}} contract so the
callers in PathLive and RoverLive don't change.

Wiring:

- priv/repo/migrations/20260413000000_create_qrz_callsigns_and_callsign_locations.exs
  creates qrz_callsigns and callsign_locations with unique indexes
  on :callsign.
- Microwaveprop.Qrz.Client added to the application supervision tree
  so the session Agent is started.
- :xmerl added to extra_applications so the release bundles it.
- config/test.exs wires Req.Test plugs for Microwaveprop.Qrz.Client
  and Microwaveprop.Geocoder, forces cache_ttl_hours: 0 so the cache
  never short-circuits test-level stubs, and supplies dummy QRZ
  credentials.
- config/runtime.exs pulls QRZ_USERNAME / QRZ_PASSWORD / QRZ_AGENT
  and GOOGLE_API_KEY from the environment so prod and dev can
  configure both upstream keys out of band.

Tests (ported verbatim from gridmap-web):

- test/microwaveprop/qrz/callsign_test.exs — schema/changeset
- test/microwaveprop/qrz_test.exs — cache hit, cache miss, TTL
  behavior, upsert on stale, error passthrough, case-insensitive
  input
- test/microwaveprop/geocoder_test.exs — success, zero results,
  request denied, transport error
- test/microwaveprop/callsign_location_test.exs — end-to-end flow
  including the QRZ-lat/lon shortcut and the missing-address error
  path

All 1294 tests still pass. Credo strict clean.
2026-04-12 17:43:29 -05:00
..
.formatter.exs initial 2026-03-28 11:28:47 -05:00
20260328172746_create_qsos.exs Add QSO import, solar indices, Oban workers, LiveView UI, and parallel weather import 2026-03-29 13:04:55 -05:00
20260328173025_add_distance_km_to_qsos.exs Add QSO import, solar indices, Oban workers, LiveView UI, and parallel weather import 2026-03-29 13:04:55 -05:00
20260328204648_create_weather_tables.exs Add weather data schema, context, sounding params, and IEM ingestion 2026-03-28 15:57:19 -05:00
20260328211118_create_solar_indices.exs Add QSO import, solar indices, Oban workers, LiveView UI, and parallel weather import 2026-03-29 13:04:55 -05:00
20260329142459_add_oban.exs Add QSO import, solar indices, Oban workers, LiveView UI, and parallel weather import 2026-03-29 13:04:55 -05:00
20260329190247_add_weather_queued_to_qsos.exs Add QsoWeatherEnqueueWorker cron job to auto-fetch weather for new QSOs 2026-03-29 14:11:03 -05:00
20260329204441_create_hrrr_profiles_and_add_hrrr_queued.exs Add HRRR model profile fetching for QSOs 2026-03-29 15:54:22 -05:00
20260329210536_create_terrain_profiles_and_add_terrain_queued.exs Add SRTM terrain path profiles for QSOs 2026-03-29 16:22:29 -05:00
20260329213831_add_submission_fields_to_qsos.exs Add /submit route for user QSO submission 2026-03-29 16:46:35 -05:00
20260330174211_create_commercial_links.exs Add commercial link monitoring via SNMP polling 2026-03-30 13:02:59 -05:00
20260330174511_seed_commercial_links.exs Add commercial link monitoring via SNMP polling 2026-03-30 13:02:59 -05:00
20260330174723_create_commercial_samples.exs Add commercial link monitoring via SNMP polling 2026-03-30 13:02:59 -05:00
20260330180421_add_precipitation_to_surface_observations.exs Add IEM precipitation data to QSO weather pipeline 2026-03-30 13:18:11 -05:00
20260330180721_create_iemre_observations_and_add_iemre_queued.exs Add IEM precipitation data to QSO weather pipeline 2026-03-30 13:18:11 -05:00
20260330182625_ensure_precipitation_columns.exs Fix missing precipitation columns and backfill weather data 2026-03-30 13:27:58 -05:00
20260330182652_backfill_surface_observations_with_precipitation.exs Fix missing precipitation columns and backfill weather data 2026-03-30 13:27:58 -05:00
20260330214829_create_propagation_scores.exs Add CONUS grid definition and propagation score schema 2026-03-30 16:50:06 -05:00
20260331143209_add_propagation_scores_viewport_index.exs Add covering index for map viewport query (6s -> 25ms) 2026-03-31 09:36:36 -05:00
20260331214714_add_point_forecast_index.exs Add forecast sparkline graph to point detail panel 2026-03-31 16:47:52 -05:00
20260401151012_add_training_performance_indexes.exs Integrate ML model into grid worker, QSO search, and UI improvements 2026-04-01 10:14:22 -05:00
20260401154846_partition_hrrr_profiles.exs Rename QSO to Contact in UI, add higher bands, improve submit page 2026-04-01 12:28:24 -05:00
20260401180056_add_flagged_invalid_to_contacts.exs Duct visualization, flag invalid contacts, fix HRRR pruning 2026-04-01 13:09:09 -05:00
20260402170033_replace_enrichment_booleans_with_enums.exs fix enrichment 2026-04-02 15:30:41 -05:00
20260402172332_fix_enrichment_status_from_actual_data.exs Remove unused alias in migration 2026-04-02 12:40:14 -05:00
20260402204809_rename_qsos_to_contacts.exs Rename qsos table to contacts, qso_id to contact_id 2026-04-02 15:52:19 -05:00
20260402212323_add_enrichment_notify_triggers.exs Replace polling with Postgres LISTEN/NOTIFY for live updates 2026-04-02 16:26:22 -05:00
20260404145255_tune_propagation_scores_autovacuum.exs Skip updating unchanged scores to reduce dead tuples, tune autovacuum 2026-04-04 09:53:45 -05:00
20260405001416_add_is_grid_point_to_hrrr_profiles.exs Batch is_grid_point backfill and reduce HRRR workers to 5 2026-04-05 08:06:20 -05:00
20260405001517_add_contacts_pos1_partial_index.exs Database performance fixes and async backfill enqueue 2026-04-04 19:19:18 -05:00
20260407165919_add_era5_profiles_and_rtma_observations.exs Add ERA5 reanalysis and RTMA data sources 2026-04-07 12:04:16 -05:00
20260408145951_create_users_auth_tables.exs Add password auth with callsign + email confirmation 2026-04-08 10:21:40 -05:00
20260408154735_create_beacon_monitors.exs Add beacon monitor registration and fix prod SMTP 2026-04-08 11:30:28 -05:00
20260408163732_add_is_admin_to_users.exs Add is_admin flag, auto-grant to graham@mcintire.me 2026-04-08 11:43:52 -05:00
20260408165341_create_beacons.exs Add /beacons CRUD and /users admin page 2026-04-08 12:01:45 -05:00
20260408174251_rename_beacon_power_to_mw.exs Beacon power in mW, settings tweaks 2026-04-08 12:45:35 -05:00
20260408174719_rename_beacon_height_to_ft.exs Beacon height in feet, auto-fill lat/lon from grid 2026-04-08 12:55:10 -05:00
20260408180611_add_on_the_air_to_beacons.exs Beacon detail map with range estimate and on_the_air flag 2026-04-08 13:29:58 -05:00
20260408192634_add_approved_to_beacons.exs Beacon submission approval workflow 2026-04-08 15:42:32 -05:00
20260408211056_add_keying_to_beacons.exs Add beacon keying field (on/off or FSK) 2026-04-08 16:13:35 -05:00
20260408212103_add_notes_to_beacons.exs Beacon improvements: anon submit, notes, /map nav, admin backfill 2026-04-08 16:23:59 -05:00
20260409135526_add_bearing_and_beamwidth_to_beacons.exs Add beacon bearing, beamwidth, and weak-signal keyings 2026-04-09 08:59:16 -05:00
20260409175343_add_user_id_to_contacts.exs Link submissions to the logged-in user on /submit 2026-04-09 13:01:49 -05:00
20260409191733_add_oban_pro.exs Switch Oban to the Pro Smart engine, rate-limit ERA5 CDS 2026-04-09 14:19:04 -05:00
20260409211647_create_hrrr_native_profiles.exs Phase 1 tasks 1.1-1.5: HRRR native hybrid-sigma ingestion 2026-04-09 16:23:51 -05:00
20260410134532_create_hrrr_climatology.exs Phase 3 spike (parked) + Phase 6: temperature anomaly 2026-04-10 08:47:11 -05:00
20260410134951_create_metar_5min_observations.exs Phase 8: 5-minute METAR ingestion pipeline 2026-04-10 08:53:54 -05:00
20260410200000_create_nexrad_observations.exs Phase 9.3, 9.4, and Phase 3 NEXRAD pipeline 2026-04-10 12:48:36 -05:00
20260411144239_change_beacon_height_ft_to_integer.exs Change beacon height_ft from float to integer 2026-04-11 09:44:42 -05:00
20260411144719_round_beacon_lat_lon_to_6_decimals.exs Round lat/lon to 6 decimals everywhere, add commas to EIRP display 2026-04-11 09:49:12 -05:00
20260411210308_create_contact_edits.exs Add contact edit approval system with admin review queue 2026-04-11 16:15:49 -05:00
20260412173459_add_contacts_inserted_at_desc_index.exs Cache /contacts, fix map blink, make mode optional 2026-04-12 12:47:25 -05:00
20260412174304_make_contacts_mode_nullable.exs Cache /contacts, fix map blink, make mode optional 2026-04-12 12:47:25 -05:00
20260413000000_create_qrz_callsigns_and_callsign_locations.exs Drop gridmap.org dependency, resolve callsigns locally 2026-04-12 17:43:29 -05:00