- grid.ex: memoize conus_points/0 with module attribute (was ~94k recompute per call)
- path_compute.ex: reduce 6 list traversals to single Enum.reduce pass
- radio.ex + application.ex: fix unused variable e and redundant case warnings
- priv: add 5 partial indexes on (qso_timestamp) for enrichment queries (weather, hrrr,
terrain, iemre, radar) matching the WHERE status IN ('pending','failed') AND pos1 IS NOT NULL
pattern used by contacts_needing_enrichment
- findings.md: mark all performance hotspots and corresponding items as fixed
- /u/:callsign profile no longer leaks private contacts or pending beacons;
Radio.list_contacts_for_user/2 and Beacons.list_beacons_for_user/2 now
filter by viewer (owner/admin see everything, others see only public).
- Radio.create_contact/2 places user_id on the struct before
submission_changeset so validate_user_or_email/1 accepts authenticated
submissions that omit submitter_email.
- valkey_test.exs runs async: false; its setup mutates global Application
env and Process registry, which otherwise crashed concurrent ConnCase
GridCache.clear/0 calls (Mox.UnexpectedCallError on SCAN).
- Radio.normalize_proposed/1 now Map.take/2 the allowed-edit keys
before normalization. Owner / admin / pending-edit submit paths all
funnel through this boundary, so a crafted form can no longer
mass-assign user_id, flagged_invalid, inserted_at, etc. on a
contact via String.to_existing_atom -> Ecto.Changeset.change.
- Accounts.revoke_api_token/2 rescues Ecto.Query.CastError and
returns {:error, :not_found} so a malformed UUID in
DELETE /api/v1/me/api-tokens/:id renders the API's clean 404
problem+json instead of a 500.
- ProfilesFile.read_etf decodes with :erlang.binary_to_term(bin, [:safe]).
Defense-in-depth against tampered on-disk profiles (atom-table
exhaustion via untrusted ETF).
- Regression tests: contact_edit_test.exs covers the rejected
mass-assignment fields; accounts_api_token_test.exs covers the
malformed-UUID path.
- Beacon detail endpoints (LiveView + REST API) now hide unapproved
beacons from anonymous and unauthorized viewers; only the submitter
and admins can see pending records before approval. Adds
Beacons.get_visible_beacon/2 with scope-aware checks.
- API contact pagination now honors per_page end-to-end.
Radio.list_contacts/1 accepts :per_page and clamps to 200.
- API rate limiter: ETS table is now owned by a long-lived Sweeper
GenServer (won't die with a request task); Sweeper periodically
prunes expired-window rows to bound memory; init_table/0 race is
rescued.
- /scores/cells and /weather/cells: add per-IP rate limiting and a
shared GridBounds clamp/413 guard so global / oversized viewports
no longer drive unbounded binary responses.
- NEXRAD PNG unfilter (sub/up/average/paeth): replace acc++[byte]
+ Enum.at(acc, idx-bpp) with O(n) binary recursion. Decode time
for the 12200x5400 n0q frame goes from quadratic to linear.
- LiveTableFooter.parse_page and ScoresFile.fetch_bound: switch
String.to_integer/String.to_float to Integer.parse/Float.parse,
fall back to defaults instead of raising.
- PathLive and MapLive band-event handlers: replace
String.to_integer(params["band"]) with parse_int / parse_band_param
so a non-numeric band parameter no longer crashes the LiveView.
Two related changes to the contact detail page:
1. The "Flagged invalid" badge was wrapping to two lines on
narrow widths because the daisyUI badge has no explicit
no-wrap class. Add `whitespace-nowrap`, drop the redundant
capitalization on "Invalid", and append the flagger's
callsign so the badge reads "Flagged invalid by W5XYZ".
Hover tooltip carries the timestamp.
2. New `flagged_by_user_id` + `flagged_at` columns on contacts.
`Radio.toggle_flagged_invalid!/2` now takes the admin User
doing the flagging; on unflag it clears both fields so a
subsequent re-flag gets a fresh attribution and the badge
never shows a stale flagger.
3. `Radio.delete_contact!/1` + a Delete button next to Flag.
Admin-only, gated in both the event handler and the template.
Uses LV's `data-confirm` for the destructive-action prompt.
All FK references to `contacts` already declare
`on_delete: :delete_all` (terrain_profiles, contact_edits,
contact_common_volume_radar) so a single Repo.delete!
cascades cleanly.
- Unify haversine_km / deg_to_rad: Geo is the canonical home (atan2
form for antipodal stability); Radio, common_volume, rain_scatter,
ionosphere, terrain/srtm, terrain/elevation_client now delegate.
- Drop safe_min/safe_max wrappers in favor of Enum.min/max defaults.
- Move parse_float / parse_int to MicrowavepropWeb.LiveHelpers; eme,
path, and rover LiveViews import from there.
- Extract MicrowavepropWeb.LocationResolver from the eme/path
resolve_source / resolve_location duplicate. Standardize the kind
key and "Invalid grid square" error message.
- Convert Scorer cond chains (humidity, td_depression, sky, wind,
rain, pwat, pressure) to multi-clause guarded defs, matching the
existing classify_time_period style.
- extract_profile_fields uses a named map accumulator instead of a
6-tuple; build_path_conditions guards on %{temps: []} / %{dewpoints: []}.
- Collapse scores_file clamp/3 to max |> min.
- humidity_effect_label lives in BandConfig; map_live and path_live
both use it.
Each fix is covered by a regression test that fails on `main` and
passes on this commit.
Round 1 (initial review):
* propagation: thread `latitude` into the conditions map so
`score_season/4` actually picks up regional multipliers
* hrrr_client / fetcher.rs: `nearest_hrrr_hour` rounds DOWN, never at
a future cycle that NOAA hasn't published yet
* radio: spherical-vector great-circle midpoint replaces the
arithmetic mean — anti-meridian paths no longer fold to Greenwich
* weather: `reconcile_weather_statuses` scales the longitude band by
`1 / cos(lat)` so the bbox stays ~150 km wide at every latitude
* radio/maidenhead: clamp 90°/180° below the field-bucket overflow so
`from_latlon` never emits invalid characters like 'S'
* prop_grid_rs/pipeline: merge HRRR + NEXRAD-derived rain rates and
read `best_duct_freq_ghz` into `best_duct_band_ghz` so the Native
Duct Boost actually fires
* propagation/region (Elixir + Rust): inclusive upper bounds so points
exactly at lat_max get the regional multiplier
* weather/sounding_params (Elixir + Rust): drop the 10 m gradient
floor so HRRR's thin near-surface layers stop hiding sharp ducts
* weather/sounding_params: when the profile ends inside a duct,
finalize it with the highest sample as the top instead of throwing
it away (Rust port already correct)
Round 2 (post-fix sweep):
* radio + commercial: single canonical haversine in Radio (atan2
form); Commercial delegates instead of carrying a second copy that
could disagree at threshold distances
* prop_grid_rs/profiles_file: `snap_coords` matches Elixir's
step-aware snap (`round(coord/0.125) * 0.125`, then 3-dp round) so
Rust-keyed and Elixir-keyed profile maps land on the same cell
* weather/grib2/wgrib2: `parse_lon_val_segment` uses `Float.parse`
uniformly — wgrib2 dropping the trailing `.0` from a longitude no
longer crashes the whole chain step
`apply_edit_to_contact/2` is the path for both owner direct-edits and
admin reviewed-edits. The public contact-map + total-count + gzipped
controller payload caches are built from `private == false` rows, but
only the insert path was clearing them — every other update left
/api/contacts/map and /contacts/map serving the previous snapshot for
up to 10 minutes. A `private: false -> true` flip on a contact already
present in the cached payload was a temporary privacy leak; grid
corrections, callsign changes, and flagged_invalid flips silently
diverged from the live row.
Extracts the three `Cache.invalidate` calls into
`invalidate_contact_map_caches/0` and runs it from both
`apply_edit_to_contact/2` and the existing insert path so the public
view stays in sync.
The /contacts/:id edit form always submits the `private` checkbox, but
`current_value/2` had no `"private"` clause and fell through to the
catch-all `nil`. The Contact schema defaults `private: false`, so an
unchanged box compared `false != nil` and was always tagged as a
change — for non-owners that meant a spurious pending review queued
on every save, and for owners/admins it triggered a no-op direct
update.
Adds the missing clause so the diff sees boolean-vs-boolean and
correctly returns `:no_changes` when the box is untouched.
The /contacts/:id edit form pre-fills qso_timestamp via
`Calendar.strftime(ts, "%Y-%m-%d %H:%M")` and submits whatever the
user typed back unchanged. Two failure modes resulted:
1. `diff_against_contact/2` compared that string against the stored
`%DateTime{}`, never matched, and treated every untouched submit
as a modification.
2. `build_contact_changes/2` then ran `DateTime.from_iso8601/1` on
the space-separated, no-seconds, no-Z form and crashed with
`MatchError` on `{:error, :invalid_format}` — direct owner/admin
submits raised instead of saving.
Adds `normalize_timestamp_field/2` to `normalize_proposed/1` that
parses both the form's `"YYYY-MM-DD HH:MM[:SS]"` and the strict
`"YYYY-MM-DDTHH:MM:SSZ"` shapes into a UTC `%DateTime{}` (with
`NaiveDateTime.from_iso8601` doing the heavy lifting after a small
seconds-padding pass). Empty strings drop the field; unparseable
input drops it too so a typo becomes "no changes" rather than a
500.
Flagged contacts were only reachable from the contact detail page;
once the Flag column came off the /contacts index there was no
at-a-glance view of what's been flagged. Add a second table on the
admin contact-edits page (below the pending-edits table) that lists
every flagged contact newest-first with a direct View link. Hidden
entirely when nothing is flagged.
Contacts can be marked private at submit time (single form, CSV
import, ADIF import) and edit time. Private contacts are hidden
from the public map, the public /u/callsign profile, and the
browse table for anonymous and non-owning viewers. The original
submitter and admins see them inline on the browse table with a
"Yes" badge, and the detail page shows a lock icon. Non-authorized
viewers get a 404 on the detail page.
Add optional height1_ft/height2_ft to contact schema, submit form, and
edit form (direct + suggest-edit paths). Heights flow through to the
elevation-profile terrain analysis so clearance and diffraction are
computed from the actual antenna height instead of a 10-ft default.
Editing heights resets terrain_status and re-enqueues
TerrainProfileWorker so the stored path analysis picks up the new
geometry.
Also fix mark_likely_duct/3: the pattern was grabbing the element
instead of the index, so no duct ever got flagged as the likely
propagation path even when extract_ducts returned a valid layer.
57,186 prod contacts stored pos1/pos2 with 'lng'; 1,133 used 'lon'.
Every Elixir caller carried a `pos["lon"] || pos["lng"]` fallback
— which just caused a SQL widget to silently miscount 98% of contacts
(count_narr_done used `pos1->>'lon'` directly, no fallback, so every
lng-keyed row returned NULL and failed the coverage check).
- Migration rewrites every pos1/pos2 JSONB in place, renaming 'lng' to
'lon' and dropping 'lng'.
- Removes all 20+ `|| pos["lng"]` fallbacks across lib/, workers,
scorer, weather, radio.ex, contact show view, and recalibrator.
- lib_ml/propagation_analyze.ex SQL now reads pos1->>'lon' directly
(was reading 'lng' only, which would have broken after migration).
- priv/repo/import_contacts.exs one-time seed script now emits 'lon'
with string keys, matching production shape.
- Test fixtures in 4 test files normalized to 'lon'.
- Two lng-characterization tests deleted — nonsensical post-normalize.
- Updated notebook + old import_weather script to match.
- JS hook contact_map_hook.ts TypeScript type narrowed to 'lon'.
Widen dedup match to 4-char grid prefix so an upload with a longer grid
(e.g. FN42aa25) finds the existing FN42 contact. Within a match, classify
as a refinement when the upload strictly extends the existing grid or
fills a missing mode; as a contradiction (still skipped) when grids or
modes genuinely disagree. Refinements flow through a new preview bucket,
update the existing row in place, and re-enqueue enrichment when the
position changed.
The pending-edits list on the admin review page now sorts through
live_table. Columns for contact, submitted-by user, changed-fields
summary, and submitted timestamp are rendered via 2-arity custom
renderers so the Ecto-preloaded :contact and :user associations stay
accessible — live_table's data_provider path skips select_columns so
the full struct (with preloads) reaches the renderer.
The review drawer, diff table, approve/reject flow, and note textarea
all stay as-is; after approve/reject the page push_patches back to
its current path so live_table re-runs handle_params and the list
refreshes.
Pending count in the header now reads Radio.pending_edit_count/0
instead of length(@edits) since we no longer hold the full list in
socket state.
The profile page's involving-contacts query now limits to the 100
most recent rows (ordered by qso_timestamp desc). A small note is
shown under the heading when the cap is hit so users know there's
more data they're not seeing.
Add a new card on the profile page listing every contact where the
callsign appears as either station1 or station2, case-insensitive,
newest first. Backed by a new `Radio.list_contacts_involving_callsign/1`
query with unit coverage for station1/station2 matching, case handling,
ordering, and blank input.
Profile page:
* New MicrowavepropWeb.UserProfileLive at /u/:callsign is a public
page showing a user's contacts and beacons. Resolves case-
insensitively so /u/w5isp and /u/W5ISP are the same thing; unknown
callsigns redirect to /. Uses daisyUI card / stats / table
components with an avatar-placeholder initial and hero icons.
* Accounts.get_user_by_callsign/1 (case-insensitive) plus
Radio.list_contacts_for_user/1 and Beacons.list_beacons_for_user/1
back the page. Beacons list includes both approved and pending so
owners see their drafts.
* The top nav bar and the three LiveView sidebars (MapLive,
WeatherMapLive, ContactMapLive) now render the logged-in callsign
as a navigate link to /u/:callsign instead of a static label.
* Nine new tests cover the lookup, the LiveView render, and the
ownership-scoped queries.
Flexible band input:
* New Microwaveprop.Radio.BandResolver module converts any of:
ADIF wavelength labels ("33cm", "1.25cm", "6mm", case/whitespace
insensitive), numeric frequency strings ("903.100", "10368.000"),
and canonical MHz integers into the one of the site's known bands.
Returns the nearest allowed band for numeric inputs >= 900 MHz,
nil otherwise.
* 902 MHz is added to Contact.@allowed_bands, ContactEdit.@allowed_bands,
AdifImport.@allowed_bands, and the BandResolver list so "33cm"
round-trips end-to-end.
* AdifImport and CsvImport now delegate band resolution to
BandResolver, and Radio.create_contact/2 normalizes the :band attr
on the way in so the manual form and any API callers benefit too.
CsvImport's "invalid band" tests previously used 99999 MHz which
the new resolver snaps to the nearest allowed band; swapped to
"notaband" which is truly unresolvable.
Contacts and beacons list UX:
* Remove the "Submitted" column from /contacts — it duplicated info
already visible on the detail page and was pushing the real
columns off narrow viewports. submitted_cell/1 and its three
column-specific tests go with it.
* Hide the Lat / Lon columns from /beacons — six decimal places of
coordinates weren't useful next to the grid square and took a
disproportionate amount of row width.
Submit and contact-detail forms now use a plain text input for the QSO
timestamp with a "YYYY-MM-DD HH:MM" placeholder and regex pattern. The
native datetime-local input ignores lang="en-GB" on macOS/iOS when the
system clock is 12-hour, so it was rendering AM/PM against the user's
wishes. Ecto's :utc_datetime cast already accepts this format (verified
space+seconds, T+seconds, and both with and without seconds/Z).
Contact edit workflow grows a third branch: admins still apply directly,
non-owners still go through the admin review queue, and now logged-in
owners of a contact (user_id == current_user.id) can apply edits
directly too. New helpers Radio.owner?/2 and Radio.apply_owner_edit/3
reuse the existing normalize + diff + apply_edit_to_contact path so the
grid-change → enrichment re-enqueue pipeline kicks in automatically.
Anonymous contacts (user_id nil) and other users' contacts both return
{:error, :not_owner}.
Nine new tests cover owner?/2 and apply_owner_edit/3 including the
no-changes, wrong-user, and anonymous-contact paths.
- Add ContactMapController serving /api/contacts/map with pre-gzipped
JSON from Microwaveprop.Cache. Browsers handle Content-Encoding: gzip
natively; payload drops from 5.7 MB raw to 1.5 MB on the wire (73%).
Cache TTL 10 min, invalidated on contact insert alongside the other
contact-map cache keys.
- Move contact map payload out of the LiveView's initial HTML entirely.
Shell renders with the filter chrome (count + band list); the hook
fetches contacts from the HTTP endpoint in parallel with map init.
This also avoids Phoenix.Socket's 64 KB frame limit (push_event would
have required chunked transfer or a global frame size bump).
- Parallelize ContactLive.Show hydrate: five independent DB loads
(weather, solar, hrrr_path, terrain, iemre) now run concurrently via
Task.async + Task.await_many. Total latency drops from sum(each) to
max(each), cutting contact show hydrate time roughly 5x.
- Add contacts_inserted_at_desc_id_desc_idx so /contacts list runs as an
Index Scan (~0.4ms) instead of a Seq Scan + top-N heapsort (~77ms on
58k rows)
- Add Microwaveprop.Cache: tiny ETS-backed TTL cache for memoizing
expensive-but-stale-tolerant values
- Cache total_entries for unsearched /contacts page loads (30s TTL,
invalidated on insert)
- Cache the entire /contacts/map payload (pre-encoded JSON + band list),
moving load_contacts into Radio.contact_map_payload; invalidated on
new contact insert. Mount goes from ~150-300ms to ~5ms.
- Fix /map overlay blinking on load: reuse the Leaflet GridLayer across
renderScores calls and just redraw() against the updated ScoreGrid,
instead of removeLayer + new layer which flashed between tile regens
- Make mode optional on submission: schema change (null: true),
submission_changeset no longer requires it, blank strings normalise to
nil, CSV import accepts 6-column (no mode) or 7-column layouts
- core_components .input adds a red asterisk to labels when required,
giving users a visual cue for which fields must be filled on /submit
1. Prevent user_id impersonation in contact submissions
- Remove user_id from submission_changeset cast fields
- Pass user_id as a separate server-side argument to create_contact/2
- Client form params can no longer forge account-linked contacts
2. Restrict flag toggle to admins only
- Add admin? guard to toggle_flag event handler
- Hide flag button from non-admin users in template
3. Reset enrichment on timestamp edits
- Add qso_timestamp to the fields that trigger enrichment reset
- Prevents stale weather/HRRR/IEMRE data after time changes
Aliases: add module aliases for 9 nested module references
Apply: replace apply/3 with direct module attribute calls
Line length: break 1 long spec line
Refactoring: extract helpers to reduce complexity and nesting
in show.ex, radio.ex, weather workers, terrain, duct detection,
backfill dashboard, contact map, and mix tasks
- Replace %Struct{} with Struct.t() in all @spec annotations
- Replace length(x) > 0 with x != [] in test assertions
- Fix multi-line spec struct references in weather.ex
- Add belongs_to :user on Beacon schema, preload in get_beacon!/1
- Show "Submitted by {callsign}" on beacon detail page
- Add NTMS donation link on the about page
- Change grid preference text to 8 characters (e.g. EM12kp37)
Registered users can suggest edits to any contact's core fields
(callsigns, grids, band, mode, timestamp). Edits enter an admin
approval queue with field-by-field diff view. On approve, changes
are applied and enrichment re-enqueued if grids/band changed.
Users receive email notification on approve or reject.
Also updates dependabot.yml for mix ecosystem.
When a contact is linked to a logged-in user account, display that
user's callsign in the Submitted column instead of a generic "Yes".
Anonymous submissions still show "Yes"; scraped contacts show an em
dash.
- Fix score_pressure crash on nil pressure_mb (coastal HRRR points)
- Set 10-min timeout on grid score upsert transaction (was :infinity)
- Single DELETE for prune_old_scores instead of N queries in a loop
- Remove dead load_hrrr_refractivity that loaded 95k rows into nil map
- Pass selected_time to point_detail to skip latest_valid_time sub-query
- Batch station existence checks (1 query per path point, not per station)
- Batch solar index upserts via insert_all in chunks of 500
- Batch backfill_distances via single UPDATE FROM VALUES statement
- Add is_grid_point boolean + partial index to hrrr_profiles (replaces
non-sargable modular arithmetic filter on every weather map query)
- Add partial index on contacts(qso_timestamp) WHERE pos1 IS NOT NULL
- Move backfill enqueue to Oban worker so UI returns immediately
- Radio.ensure_positions!/1 computes pos1/pos2/distance from grids
- Called in contact show page mount and enqueue_for_contact
- Migration backfills positions for existing contacts missing them
- Migration corrects enrichment statuses based on actual data presence
- Backfill dashboard includes contacts with grids but no positions
- New fields: hrrr_status, weather_status, terrain_status, iemre_status
with values: pending, queued, processing, complete, failed, unavailable
- EnrichmentStatus module defines valid state transitions
- Migration backfills: queued=true → complete, false → pending
- Workers set status on transitions (queued on enqueue, complete on finish)
- Show page reads status from contact struct, not computed dynamically
- Backfill dashboard queries status fields for accurate progress counts
- Remove cron-scheduled ContactWeatherEnqueueWorker (enrichment only on
submission, page view, or manual backfill)
- Partial indexes on status != 'complete' for fast unprocessed lookups
- Overlay HRRR duct layers on elevation profile chart (green bands)
- Add flagged_invalid field to contacts with toggle button on detail page
- Fix prune_old_grid_profiles to only delete 0.125° grid profiles,
preserving QSO-linked HRRR data at arbitrary positions
- Auto-enqueue HRRR fetch when viewing contact without HRRR data (<48h)
- Compact solar conditions and contact details layout
- Add band/mode/distance info line to elevation profile section
Rename all modules, functions, variables, routes, and UI text from
qso/qsos to contact/contacts. Database table stays as "qsos" to avoid
migration. Add /qsos -> /contacts redirects for old URLs.
