StatusLive had a hand-rolled thousands-separator format_number/1 and a
binary-prefix format_bytes/1 — duplication that was already part of the
review punch-list. Move both to Microwaveprop.Format (which previously
held only distance_km) and delete the local defps. Fourteen call sites
switched to Format.number / Format.bytes.
Also extend the IEM error classifier to snooze on Req.TransportError
{:closed, :timeout, :econnrefused, :nxdomain}. Same rationale as the
429 snooze in batch 1: upstream TCP resets mid-fetch aren't a job
failure, they're transient — returning {:error, _} blows them up into
Oban.PerformError logspam with full stacktraces.
Batched from a system-wide review pass.
**Rate-limit + transient-error log hygiene**
- WeatherFetchWorker: classify IEM HTTP 429 as {:snooze, 300} instead
of {:error}. Stops Oban.PerformError stack-trace spam on every
routine rate-limit during backfill, keeps the retry semantics.
- IonosphereFetchWorker: compress GIRO TLS :unknown_ca error from a
~400-char inspect blob to 'TLS unknown_ca (CA bundle missing)'.
- FreshnessMonitor: log only when an enqueue actually lands (the
Oban unique conflict case was silently dropping jobs but still
producing 'enqueuing grid worker' info lines every 5 minutes).
**Perf**
- Rust grid_level_keys(): pre-format 'TMP:{p} mb' / DPT / HGT keys
once via OnceLock instead of format!()'ing per-cell. Removes ~3.6M
String allocations per f01..f18 chain step across pipeline.rs,
hrrr_points.rs. Same for the wgrib2 :(TMP|DPT|HGT): pattern in
hrrr_points.process_batch (sfc_pattern / prs_pattern OnceLock).
- MapLive preload_forecast: Task.async_stream with max_concurrency=4
replaces the 18-wide Enum.map serial walk. Forecast cache warms
~4× faster after a band change, with ordering preserved.
- grid_tasks: partial composite index on (run_time DESC,
forecast_hour ASC) WHERE status='queued' AND kind='forecast',
matching the Rust claim query's ORDER BY. Drops the old
status-only partial that forced a sort per claim.
**Correctness**
- ContactImportWorker: add Oban unique:[keys: [:import_run_id,
:offset]]. Was missing on a worker whose perform() does a
non-idempotent atomic counter increment — a retry would double-
count imported rows.
- CommonVolumeRadarWorker: x_min..x_max default step is -1 when
x_min > x_max, triggering a runtime deprecation warning. Force
Range.new(_, _, 1) explicitly.
**Cleanup**
- Drop unused tmp_dir parameter from hrrr_point_worker + its
process_batch signature.
Score-grid files now land at `<band>/<iso>.prop` with a 4-byte "PROP"
magic header. Readers still accept the legacy `.ntms` extension and
"NTMS" magic so a rolling deploy doesn't invalidate any file already
on the shared NFS mount — legacy files age out through normal
retention.
Applied both sides of the seam: Elixir ScoresFile + regex parsers,
Rust scores_file writer + decoder. Updated the comments in all
modules that mention the extension (NotifyListener, Reconciler,
gefs_fetch_worker, map_live) and the runbook diagram. talos5 is a
DB host now, not a Rust worker — corrected the runbook accordingly.
Two-stage mount splits the propagation map's first paint: the static
HTTP render still fetches scores synchronously so SEO/noscript shipping
gets real data baked into `data-scores`, but the websocket-connected
mount defers the potentially-slow fetch to `start_async/3`. Chrome
paints immediately; scores stream in via an `update_scores` push_event
once `handle_async(:initial_scores, ...)` resolves.
A small overlay (spinner on `:loading`, retry button on `:failed`)
uses `<.async_result>` so users can see state and recover from a
transient fetch error.
Rework of d61fbd3's dialyzer suppression: the 4 warnings from
LiveTable.LiveResource's macro expansion are now fixed at the
root rather than hidden via .dialyzer_ignore.exs.
Changes:
- Delete .dialyzer_ignore.exs and remove ignore_warnings from
mix.exs. No more hidden suppressions.
- New MicrowavepropWeb.LiveTableResource shim that wraps
use LiveTable.LiveResource and overrides:
* maybe_subscribe/1 — explicit :ok = on Phoenix.PubSub.subscribe/2
(previously discarded, tripping :unmatched_return).
* handle_info/2 — explicit _ = on File.rm, Process.send_after.
Switch all 4 LiveTable-using LiveViews (contact_live/index,
beacon_live/index, admin/contact_edit_live, user_management_live/index)
to use MicrowavepropWeb.LiveTableResource instead.
- New priv/dep_patches/live_table-0.4.1.patch adds _ = in front of
LiveTable.LiveSelectHelpers.restore_live_select_from_params/2 in
the dep's macro-expanded handle_params/3. The call's return was
silently discarded, tripping the fourth warning per LiveView.
- New lib/mix/tasks/deps.patch.ex applies every
priv/dep_patches/*.patch to its target dep idempotently after
mix deps.get. Wired into the :setup alias + overridden
mix deps.get so the patch survives cache regeneration in CI.
- Other modified files in this commit are the spec-coverage
additions from the parallel agent pass (beacons.ex,
commercial.ex, propagation.ex, weather.ex, narr_client.ex,
run_timing.ex, 3 workers) — tighter specs for bounds, tuple
shapes, schema-typed params. Dialyzer remains at 0 after
the changes.
Upstream TODO: send a PR for the live_table `_ =` fix. Once
merged and released, drop the patch + mix task + bump the dep.
mix dialyzer --format short | grep ^lib/ | wc -l -> 0
mix test: 2165 tests, 2 pre-existing flakes (MapLive timestamp +
PropagationPrune ScoresFile), 0 regressions.
count_unprocessed_raw fired seven separate COUNT(*) scans on the
contacts table every 2 s refresh (total, terrain, hrrr, weather, iemre,
radar, all-done). Each one went through the 58k+ row table
independently, so the status page was hitting Postgres with ~3.5 full
scans per second of connected viewer time.
A single query with COUNT(*) FILTER clauses pulls every count in one
sequential scan. Same result, roughly 1/7 the CPU on Postgres.
Two wins in the /map hot path.
1. preload_forecast fired on every Leaflet moveend — which arrives in
bursts during pan/zoom — and each firing read + filtered + shipped
18 forecast-hour score lists (up to 90k cells per hour at full
CONUS view) over the LiveView websocket. Now debounced to the
trailing edge: the user must stop moving for 750 ms before we pay
the preload cost. select_band and propagation_updated go through
the same scheduler so a storm of PubSub updates during an hourly
chain also coalesces.
2. ScoreCache.grid_to_filtered_list walked the 92k-cell grid twice
(Enum.filter then Enum.map) and allocated an intermediate tuple
list between them. Enum.reduce emits only in-bounds result maps
directly — halves map traversal + drops the tuple intermediate.
The 'Fetching HRRR atmospheric data' spinner stayed up forever for
contacts stuck in :queued without a matching hrrr_fetch_tasks row.
Ways a contact lands in that state:
- HrrrPointEnqueuer.enqueue/1 raises, the rescue swallows the error
and returns {:ok, 0}, but the caller still marked the contact
:queued before it noticed the failure.
- Rust worker marks a (valid_time) task :done with a points array
that never included this contact's point, so no profile is ever
written for it.
- hrrr_fetch_tasks row is cleaned up by retention before the contact
page loads.
Drop the hrrr_status != :queued short-circuit and always call the
enqueuer when the profile is nil. The ON CONFLICT clause in
HrrrPointEnqueuer.enqueue/1 unions points into the existing row,
resets :done/:failed rows back to :queued with attempt=0, and no-ops
when the point is already present — so re-entering on every page view
is cheap and self-healing.
Second row in the Rust worker table pulls from hrrr_fetch_tasks so
per-QSO HRRR backfill progress is visible alongside prop-grid-rs.
Same columns (Running / Queued / Retrying / Done 1h), 2s cache, same
refresh debounce as the grid_tasks panel.
Phase 3 Stream C Elixir-side: HrrrFetchWorker is deleted; per-QSO HRRR
enrichment now writes to the new hrrr_fetch_tasks table for the Rust
hrrr-point-worker to drain.
Table shape:
- one row per valid_time (primary key) with a JSONB array of
{lat, lon} points
- UPSERT on conflict: array-union of points, status flips back to
queued if previously done/failed so a backfill re-scan naturally
refills the queue for Rust
Elixir changes:
- new migration 20260419231502_create_hrrr_fetch_tasks
- new Microwaveprop.Weather.HrrrPointEnqueuer with enqueue/1 and
enqueue_for_contacts/1. Pre-2014 contacts (NARR's territory)
are skipped here so hrrr_status can pin them to :unavailable.
- ContactWeatherEnqueueWorker: build_hrrr_jobs/1 removed; single-
contact path and batch perform/1 both route through
HrrrPointEnqueuer.enqueue_for_contacts/1. A placeholder jobs-list
is kept just to feed mark_hrrr_status!.
- contact_live/show.ex retry button enqueues via the same path.
- :hrrr queue removed from dev/config/runtime.exs
- HrrrFetchWorker module + test deleted
BackfillEnqueueWorker scans continue to flow through
ContactWeatherEnqueueWorker.enqueue_for_contact (unchanged), so the
30-min reconcile refills hrrr_fetch_tasks automatically.
4 new tests cover the routing, pre-2014 skip, UPSERT-union, and
status-reset-on-reschedule behaviour. Rust-side hrrr-point-worker
binary + k8s deployment land in the next commits.
Mirrors the Oban Job Queue Status table with a new section sourced
from the grid_tasks hand-off table. Columns: Running, Queued,
Retrying, Done (1h).
Active rows render an f## badge per in-flight forecast hour and a
detail line below with run_time / fh / attempt so it's obvious which
HRRR cycle the worker is on.
Pipes through the existing 2s debounced refresh plus a new PubSub
subscription to "propagation:updated" — PropagationNotifyListener
fans out the Rust worker's NOTIFY propagation_ready, so the panel
transitions running → done the moment Rust writes the score file.
Flagged contacts were only reachable from the contact detail page;
once the Flag column came off the /contacts index there was no
at-a-glance view of what's been flagged. Add a second table on the
admin contact-edits page (below the pending-edits table) that lists
every flagged contact newest-first with a direct View link. Hidden
entirely when nothing is flagged.
The `!` flag column was a tiny sliver that rarely fired in practice
and was pushing more useful columns offscreen on narrow layouts. The
underlying `flagged_invalid` field stays on the schema for the detail
page; only the index column goes.
Row-level privacy was already correct: private rows only render for
the owner and admins. The header column was still showing for every
viewer, leaving an always-empty slot for anyone without private
contacts of their own. Drop `:private` from the fields list unless
the scope can actually see at least one private row.
Three UI fixes:
1. MapLive now ticks every 60s and, when the selected_time is still
the "Now" slot, advances to the newer latest-past-or-now hour as
the clock crosses into it. The URL is patched in place so shared
links stay accurate without polluting history. Manually picking a
specific past/future hour turns tracking off.
2. point_forecast/3 now reads its valid_times list straight from the
on-disk .ntms files (same source the main-map timeline uses), then
consults the cache per-hour for a fast score lookup. The old cache-
or-store branch could leave the click-to-detail sparkline at 13
hours while the bottom timeline showed 14+; both now line up.
3. Point-detail panel moves from bottom-14 → bottom-24 on desktop so
its lower edge clears the forecast-timeline bar (+ max-height tuned
to match), fixing the overlap visible in screenshots.
Every parser already returned nil on malformed input, so the ||
chains fell through to defaults — except parse_time_param, which
happily accepted any well-formed ISO8601 even if it pointed
outside the current HRRR forecast window (e.g. a stale shared
link from last week). Gate it behind the valid-times window and
fall back to the normal 'now' cursor when it's out of range.
Adds explicit coverage: non-numeric band, unknown band MHz,
non-numeric lat/lon/zoom, out-of-range values, malformed time,
ancient time, and all-params-garbage-at-once. Each case mounts
cleanly and lands on the defaults.
The main map page now reads band, selected time, and map
center/zoom from URL params and updates the URL as the user pans,
zooms, picks a band, or scrubs the timeline. Pan/zoom use
push_patch replace-mode so browser history isn't spammed, while
band and time changes push fresh entries.
Also fix the 'Up to date · just now ago' pipeline chip — 'just
now' already reads as an instant so the trailing 'ago' is
redundant. Any older duration keeps it.
- Remove the 7-day outlook strip and daily_outlook_at/3. It was
derived from point_forecast, which is capped at HRRR's 18-hour
horizon, so it never had 7 days of data to show.
- factors_for: fall back to the nearest persisted analysis profile
at or before the requested time. Only f00 hours persist profiles,
so clicking any other hour used to yield an empty Analysis and
factor table. After the recent cursor fix lands 'Now' on a
forecast hour more often, this regressed further.
- available_valid_times/1: cap the forward horizon to 18h from now.
Leftover valid_times from prior cycles used to pile on the
timeline without adding information.
Both the Elixir initial-cursor pick and the JS timeline renderer
used nearest-by-absolute-distance, so any wall-clock time past
:30 snapped the 'Now' label onto the next top-of-hour forecast —
up to ~30 min in the future. Switch to the latest valid_time at
or before wall-clock, with a fallback to the earliest time when
every slot is future (shouldn't happen for HRRR, safe default).
Contacts can be marked private at submit time (single form, CSV
import, ADIF import) and edit time. Private contacts are hidden
from the public map, the public /u/callsign profile, and the
browse table for anonymous and non-owning viewers. The original
submitter and admins see them inline on the browse table with a
"Yes" badge, and the detail page shows a lock icon. Non-authorized
viewers get a 404 on the detail page.
Contact detail now searches for soundings in progressively wider radii
(150 → 300 → 600 → 1000 km) before declaring "No soundings found".
When even 1000 km turns up nothing, enqueue WeatherFetchWorker RAOB
jobs for every station at the widest non-empty radius. The worker
passes the contact_id through so it can PubSub back to
contact_enrichment:<id> as each sounding lands, and the LiveView
rehydrates without a refresh.
Also adds Weather.soundings_with_widening_radius/1 and
ContactWeatherEnqueueWorker.enqueue_raob_fetch_with_widening/1 so the
same logic can be reused from other callers.
Add Microwaveprop.Format.distance_km/1 and a matching formatDistanceKm
helper for the JS side. Both emit "X mi (Y km)" with one decimal under
10 mi and whole numbers above.
Replace the ad-hoc format_dist/format_km_mi helpers with the shared
formatter and update every user-facing distance render: contact
detail, contacts index column, user profile contact/involving tables,
path finder summary + sounding list, contacts map line popups, beacon
coverage tooltip, and propagation map range estimate + rain scatter
cell popup.
Add optional height1_ft/height2_ft to contact schema, submit form, and
edit form (direct + suggest-edit paths). Heights flow through to the
elevation-profile terrain analysis so clearance and diffraction are
computed from the actual antenna height instead of a 10-ft default.
Editing heights resets terrain_status and re-enqueues
TerrainProfileWorker so the stored path analysis picks up the new
geometry.
Also fix mark_likely_duct/3: the pattern was grabbing the element
instead of the index, so no duct ever got flagged as the likely
propagation path even when extract_ducts returned a valid layer.
Adds a compact seven-card horizontal strip to both mobile and desktop
map sidebars showing the per-day peak score at the viewport center
for the selected band. Cards are color-coded (emerald for 80+,
rose for bad) so the weekend verdict reads at a glance.
- Propagation.daily_outlook_at/3: groups point_forecast entries by
UTC date, picks each day's peak, returns ascending order
- MapLive mounts with today's outlook for the initial center; the
select_band handler refreshes it from the viewport midpoint so the
strip tracks the user's current band and rough location
- Empty-state message covers the case where no extended-horizon
scores have landed yet (fresh deploy, or before the first GEFS
run completes)
This is the consumer of the GEFS pipeline — once the cron starts
running and f024-f168 scores accumulate, days 2-7 of the strip
populate automatically.
Two related fixes:
- Drop the global contact_enrichment:{hrrr,weather,solar} subscriptions.
These topics fire for every QSO's enrichment completion, and the
handlers blindly re-queried data for the currently-displayed contact,
causing spurious re-renders whenever any enrichment worker finished
anywhere in the system. The per-contact terrain subscription stays —
that one is correctly scoped.
- Only add a slot to `hydration_pending` when the contact's status
column says the work is genuinely in flight (:queued or :processing).
Previously every connected mount flashed spinners over all eight
enrichment sections, even for contacts whose data was already cached
and about to return instantly from the async task.
Also drop the Donate-to-NTMS footer link.
The antenna-height input only scaled the displayed range labels and
the viewshed max-range — it never fed into the propagation score
itself. Showing the control implied the colored overlay responded to
it, which was misleading. Viewshed now uses a fixed 10 m baseline,
matching the previous 33 ft default.
Three improvements to the contact detail page:
* Track async hydration per source in `hydration_pending` so the
Terrain / Soundings / Atmospheric Profile sections render a loading
spinner while their task is in-flight instead of briefly flashing
"no data available" before the payload arrives.
* Propagation analysis summary now names the origin station by
callsign ("Path is terrain-obstructed at N mi from W5XD") instead
of the generic "station 1" placeholder.
* Elevation profile chart labels the 0-mi and max-mi ticks with the
matching callsign so it's obvious which end of the profile is
which station.
Propagation analysis panel now adds a "Tropo duct detected at N/M HRRR
samples along the path" line whenever any HRRR sample flags ducting,
even if the overall mechanism classification doesn't end up pointing at
the duct (e.g. short paths, clear-LoS contacts). Also rewords the map
caveat to invite users with better coordinates to file a suggested
edit — clickable when logged in, otherwise a login link.
Antenna height now drives typical/extended/exceptional range estimates
and viewshed max range via a sqrt(h_m/10) factor clamped to [0.6, 2.0].
Baseline 33 ft matches the existing defaults. Changing the height input
now pushes update_band_info and refreshes the clicked point's detail
panel plus viewshed so the range-estimate line and coverage shape track
the user's actual station height.
Deploy stamp was in Layouts.app's nav but the map page uses a custom
full-screen layout that skips the standard nav, so the info wasn't
visible on the page most users land on. Add a compact
"Deployed 2h ago" under the sidebar's data-timestamp/pipeline-status
block with the full UTC time as a tooltip, matching the shape of
Layouts.deploy_stamp.
Added 5-arity Scorer.score_refractivity/5 that takes bulk_richardson
alongside best_duct_band_ghz. The existing 1.15× boost now only
applies when Richardson is in the stable regime (< 25) — native-
profile duct cells average 8.7-18.4 for ducting vs 38.3 for
non-ducting, so a duct-band reading under turbulent conditions is
more likely to be torn up by mechanical mixing than to carry a
beyond-LOS signal.
Wiring:
- Weather.nearest_native_duct_info/3 returns {ghz, richardson}; the
bare nearest_native_duct_ghz/3 now delegates.
- PathLive and Propagation.score_with_algorithm/7 both pull the
Richardson value into the conditions map alongside best_duct_band.
- Scorer.composite_score/2 reads conditions[:bulk_richardson] and
passes it into score_refractivity/5.
- Backward compat: 4-arity variant and nil Richardson keep the old
unconditional-boost behaviour so existing callers don't regress.
Path calculator conditions map was missing :latitude and
:best_duct_band_ghz. Scorer read them with bracket-notation so the
behaviour was defensively correct but we were leaving signal on the
floor:
- latitude: midpoint of src/dst so `score_season` picks up the right
latitude-based seasonal shift for northern vs southern paths
- best_duct_band_ghz: queried via new Weather.nearest_native_duct_ghz/3
at the midpoint so score_refractivity/4 applies the 1.15× boost
when a native-resolution duct supports the target band
Recalibrator.fit/1 was serialising ~10k per-contact HRRR lookups
(5000 positives + 5000 negatives, one Weather.find_nearest_hrrr each).
Parallelised both sides with Task.async_stream at 20 concurrency —
well under the prod db pool of 30. Cuts a band recalibration from
~minutes of query round-trips to seconds.
Audit pass found two real issues (most agent-flagged items were false
positives — the hrrr_native_profiles composite index exists,
pwat_mm/bl_depth_m are already in the path conditions map, and the
/1.0 in MechanismClassifier is the standard Elixir int→float coerce).
1. Path calculator's 9-point HRRR sampling was running sequentially,
~9× the latency of a single query. Task.async_stream with
max_concurrency: 9 makes them concurrent so path calc pays roughly
one query's worth of time instead of nine.
2. MechanismClassifyWorker's meteor-shower window used hardcoded 2024
peak dates with `%{date | year: peak.year}` projection. That broke
across year boundaries — a Dec 30 contact couldn't match a Jan 4
Quadrantids peak even in-range of the ±3 day window. Switched to
{month, day} tuples and check the peak projected onto year N-1, N,
N+1 so the window works correctly near Dec 31 / Jan 1.
The "Plot path to beacon" action now includes `tx_power_dbm` and
`src_gain_dbi` in the /path query string so the path calculator's TX
side emits the same EIRP as the beacon out of the box. The beacon
schema stores `power_mw` as EIRP already (per Beacons.RangeEstimate),
so we take 10·log10(power_mw), split against a default 30 dBi dish,
and clamp the TX power to a 0 dBm floor so very-low-EIRP beacons
produce a sane gain/power split instead of -10 dBm outputs.
Examples:
10 W EIRP (40 dBm) → 30 dBi dish + 10 dBm TX
100 W EIRP (50 dBm) → 30 dBi dish + 20 dBm TX
100 mW EIRP (20 dBm) → 20 dBi + 0 dBm (clamped)
Path calculator now samples nine HRRR grid cells evenly along the
source→destination line (was three: Source/Midpoint/Destination) so
mid-path ducts that endpoint-only queries missed show up in the duct
count. The UI reports "Duct at N / 9 HRRR points" instead of a binary
flag, and the table lists every sample with its percent-of-path label.
Independent RAOB signal: uses the new `Weather.nearest_sounding_to/3`
to find the closest sounding within 300 km / ±3 h of the midpoint. If
the sounding's `ducting_detected` is true we surface a "RAOB <code>
confirms duct" badge — HRRR under-reads thin surface ducts that
soundings resolve, so the two signals together are the right
cross-check for live duct prediction.
Distance formatting: all km values on the path results (top Distance
summary, sounding "km from midpoint") now render as
`123 km (76 mi)`; under 10 km we keep one decimal so near-LOS paths
don't round to zero.
Ran recalibrate_algo.py against the full local prop_dev (81,994 contacts,
18.6M HRRR rows) and derived per-band composite weights for the nine
bands with >=200 matched contacts. Moisture (dewpoint/PWAT/surface N)
is consistently beneficial through 5.76 GHz, reverses at 24 GHz; rain
scales sqrt(rain_k) not linearly so 24 GHz gets 0.215 rain weight
instead of the linear-ratio 0.95. 10 GHz stays as the reference band
(defaults); 47+ GHz inherits defaults (n<200).
- BandConfig.weights/1 returns per-band override or default fallback
- @band_configs carries :weights on 222/432/902/1296/2304/3400/5760/24G
- Recalibrator.compute_factors/3 + fit(band_mhz:) band-aware fitting
- Scorer + ContactLive.Show pass band_config to weights/1
- algo.md Part 2d documents the 2026-04-18 analysis + derivation rule
- scripts/derive_band_weights.py turns correlations into weight maps
- report preserved at docs/algo-reports/2026-04-18-recalibration.md
Previously the map loaded the latest cached valid_time from GridCache,
which can be any +Nh forecast hour the worker has written while walking
f00-f18. Users expect to see current conditions on load, not a forecast
that's ~11 hours out.
Band toggles previously roundtripped through LiveView and rebuilt every
polyline + popup + hover handler on each click. Now the hook builds one
L.LayerGroup per band once after the initial fetch; toggling is just
addLayer/removeLayer on the map (O(bands), not O(contacts)). Checkbox
and All/None events are handled directly in the hook via delegated
document listeners — no LV roundtrip. Callsign filter still roundtrips
(debounced) since it's cross-cutting, and now iterates pre-built
polylines without recreating them.
The detail page was hard-coding GHz formatting, so 50/144/222/432/902
rendered as 0.1/0.4/0.9 GHz instead of the MHz labels every amateur
operator actually uses. Switch to MHz when band < 1000, keep GHz for
microwave bands.
Also surface band and mode in the header subtitle so the key QSO
metadata is visible before SRTM/terrain enrichment completes (the
elevation-profile block that previously rendered them is gated on
that enrichment).
Vendors canvas-confetti 1.9.3 and wires an ImportConfetti hook on the
alert-success banner that /imports/:id already renders when
run.status == "complete". The hook uses the "realistic look" preset
from confetti.js.org and runs on mount, so it fires exactly once —
when the banner first appears.
SubmitLive now hands off to the async pipeline:
- confirm_csv/confirm_adif → CsvImport.enqueue/2 → push_navigate to
/imports/:id. ADIF preview shape matches CSV, so one enqueue call
serves both.
- allow_upload(auto_upload: false) so bytes only stream when the user
clicks submit. Progress bar + percentage now gated on
entry.progress > 0 and < 100 — invisible during file selection,
only rendered while the actual upload is in-flight.
- Old in-page csv_results / @csv_result / reset_csv removed; the
/imports/:id page replaces them.
New ImportLive at /imports/🆔 subscribes to 'csv_import:<id>' PubSub,
shows total/processed/imported/refined/error counters in a daisyUI
stats grid, progress bar, status badge, and a collapsed errors table
when error_count > 0. Missing or malformed run_id redirects to /submit
with a flash.
7 new frontend tests (5 ImportLive, 2 updated SubmitLive), 1902 total,
credo clean.
For large CSVs (e.g. the 34k-row ARRL dump), the synchronous commit
path blocked for minutes and a crash mid-commit left no audit trail.
Refactor to run inserts + enrichment enqueues via Oban.
- New import_runs table: stores preview rows + running counters (total,
processed, imported, refined, error) + status + errors map + timing.
- New :contact_import Oban queue (limit 4) + ContactImportWorker.
Args: {import_run_id, offset, limit}. Each chunk (100 rows) inserts
its slice via CsvImport.commit_rows/1, atomically increments counters
in one update_all, flips status, and broadcasts progress on
'csv_import:<id>' PubSub topic.
- CsvImport.commit_rows/1 extracted from commit/1 (back-compat preserved).
New CsvImport.enqueue/2 persists the run + dispatches N chunk jobs.
Also: submit page copy updated from '902 MHz and up' to '50 MHz and up'
matching the band allowlist expansion done earlier.
LiveView UI for /imports/:id is a separate commit.
57,186 prod contacts stored pos1/pos2 with 'lng'; 1,133 used 'lon'.
Every Elixir caller carried a `pos["lon"] || pos["lng"]` fallback
— which just caused a SQL widget to silently miscount 98% of contacts
(count_narr_done used `pos1->>'lon'` directly, no fallback, so every
lng-keyed row returned NULL and failed the coverage check).
- Migration rewrites every pos1/pos2 JSONB in place, renaming 'lng' to
'lon' and dropping 'lng'.
- Removes all 20+ `|| pos["lng"]` fallbacks across lib/, workers,
scorer, weather, radio.ex, contact show view, and recalibrator.
- lib_ml/propagation_analyze.ex SQL now reads pos1->>'lon' directly
(was reading 'lng' only, which would have broken after migration).
- priv/repo/import_contacts.exs one-time seed script now emits 'lon'
with string keys, matching production shape.
- Test fixtures in 4 test files normalized to 'lon'.
- Two lng-characterization tests deleted — nonsensical post-normalize.
- Updated notebook + old import_weather script to match.
- JS hook contact_map_hook.ts TypeScript type narrowed to 'lon'.
Visitor geolocation already flows via the Cloudflare CF-IPLatitude /
CF-IPLongitude headers into the session; MapLive centers the map on
the visitor's location (or DFW as fallback). Add a banner that appears
above the map when those coords are outside the CONUS bbox
(lat 24.5-49.5, lon -125 to -66.5), letting overseas visitors know
propagation data won't be available at their location yet.
Map still centers on the visitor's actual location — this is a note,
not a re-center. Banner is fully absent from the DOM when not needed
(:if conditional).
