The 'show estimated current coverage' toggle pushes 1-8k grid cells
to the browser at 5.76 GHz+ and previously rendered each as its own
SVG <path> via Leaflet's default vector renderer. That meant 1-8k
DOM nodes plus per-element layout/paint on every pan/zoom — enough
to lock up tabs on mid-range hardware.
Two changes:
1. All coverage rects share a single L.canvas({padding: 0.2})
renderer. Browser sees one <canvas> element drawn in one pass
per redraw instead of thousands of <path> nodes. Hit-testing for
tooltips still works through Leaflet's canvas-renderer mouse
events.
2. Drop `sticky: true` from the tooltip binding. Sticky tooltips
reposition to follow the cursor on every mousemove, which scales
O(N) with cell count — across thousands of rects that becomes the
dominant cost as the cursor moves. Without sticky, the tooltip
anchors to the cell's centre on hover (same info, cheaper).
Owner / admin click "Edit" on /rover-locations/:id and the marker
becomes draggable. As they drag, the JS hook pushes `location_dragged`
events with the new lat/lon; the LiveView updates `working_lat` /
`working_lon` / `grid` so the page shows the live preview without
writing to the DB. "Save" persists via Rover.update_location, "Cancel"
reverts both the assigns and the marker position.
Implementation notes:
- Switched the map marker from L.circleMarker to a draggable L.marker
with a divIcon (CircleMarker has no `dragging` handler).
- A second `draggingDotIcon` (amber, larger, grab cursor) makes the
edit affordance obvious.
- Server <-> hook coordination uses push_event:
set_marker_draggable / reset_marker. Drag results come back on
pushEvent("location_dragged", { lat, lon }).
- Coordinates row binds to working_lat/working_lon so the displayed
decimals + derived grid update on every drag, not only on save.
Adds a full-screen map view rendering every Good rover-location as a
green circle marker (popup links to the detail page). Uses the standard
OSM/Satellite + Maidenhead grid overlay pattern. Bad locations are
excluded. Adds a Map button next to Add location on the index header.
Adds start/end date inputs (UTC) to the sidebar (mobile + desktop) with
Apply dates and Clear buttons. The hook keeps a sliced YYYY-MM-DD on
each line entry so the range check is a string compare during filter
re-application; combines with the callsign filter so e.g. AA5C between
2025-01-01 and 2026-01-01 is one query.
LocationMap hook now adds a togglable grid layer (matching the contact
map pattern) so the rover-location detail page shows the standard
Maidenhead overlay alongside the marker.
/weather now fetches both grids and paints each as its own canvas
overlay at its native cell size (HRRR 0.125°, HRDPS 0.5°). Server-side
chunk filtering means a US-only viewport gets an empty HRDPS pack and
vice versa — no wasted payload. /weather-ca's HRDPS-only mode is
unchanged.
Hook state moves from single (pack/overlay/abort) to per-source maps
keyed by "hrrr"|"hrdps". Render order is fixed (hrdps first, hrrr
second) so HRRR's higher-res canvas stacks on top along the border
where both sources cover the viewport.
The canvas overlay hard-coded a 0.0625° half-step when projecting each
cell to canvas pixels — correct for HRRR (0.125° native grid) but wrong
for HRDPS, which we downsample to 0.5° (0.25° half-step). At HRRR's
half-step every HRDPS cell got painted at 1/16th its actual area, so
the overlay over Canada looked like a sparse pinstripe of vertical
streaks instead of contiguous coverage.
Plumb halfStep through the WeatherCanvasOverlay constructor and pick
0.25 vs 0.0625 in the hook based on data-source.
New LiveView at /weather-ca duplicates the /weather UI but defaults the
viewport to central Canada (55N, -100W, z=4) and renders the map div
with data-source="hrdps". The JS hook reads that attribute and appends
&source=hrdps to every cell fetch; the WeatherTileController routes
those reads through Weather.weather_grid_hrdps_at/2, which only reads
the <vt>.hrdps scalar dir and skips HRRR merging entirely. Timeline is
sourced from ScalarFile.list_valid_times_hrdps/0 so HRDPS-only hours
show up even when no HRRR file exists for the same valid_time.
Also drops "— Unified" from the algo.md H1.
Adds lat/lon/zoom URL params to /weather so reload / share-link
restores the exact map state. The existing ?layer= param keeps
working alongside.
LiveView side:
- mount() reads lat/lon/zoom from query params, clamps to valid
ranges (lat -90..90, lon -180..180, zoom 4..10), and falls back
to DFW@z7 when missing or invalid. Initial values flow to the
hook via three new data attributes.
- handle_event("viewport_changed", ...) accepts {lat, lon, zoom}
pushed by the hook on debounced moveend, clamps + rounds, and
push_patches the URL with replace: true so browser history
doesn't grow an entry per pan tick.
- build_path/2 helper centralises URL construction so select_layer
and viewport_changed both produce a deterministic param order
(layer, lat, lon, zoom). Tests assert exact URLs.
JS hook side:
- mounted() reads data-initial-lat/lon/zoom from the el dataset
with parseFloat/parseInt + Number.isFinite gates, then seeds
Leaflet's center/zoom from those instead of the previous
hardcoded (32.897, -97.038, 7).
- moveend handler now also schedules a 400 ms-debounced
pushEvent("viewport_changed", {lat, lon, zoom}) so the URL
catches up with whatever the user panned/zoomed to. Separate
debounce timer from the cells fetch since the URL push and the
network fetch have different cadence sweet spots.
Tests cover: initial seeding from URL params, clamping of bad
values, viewport_changed event producing the right patched URL,
and the original layer-shareability test (now expecting the full
viewport in the URL).
Apply the same overlay refactor we did for /weather to the propagation
map. Score data used to flow as JSON via LiveView push_event
(update_scores + the 18-hour preload_forecast batch); switching bands
or scrubbing time meant a synchronous server roundtrip + ~MB JSON push
per change.
Now:
1. New /scores/cells endpoint returns a "PSCR" binary cell-pack —
columnar f32 lats/lons + u8 scores, ~3-4× smaller than JSON and
parsed via DataView + typed-array views with no JSON.parse.
2. JS hook owns score fetching. mount/moveend/band/time changes
trigger HTTP fetches against /scores/cells. After the initial
paint, every other forecast hour for the current band is fetched
in parallel in the background — timeline scrubs after the prefetch
finishes are pure cache hits with no network at all.
3. LiveView no longer pushes scores. select_band emits update_band_info
with band_mhz so the client knows what to refetch; select_time and
set_selected_time only update server-side state (URL, point detail
bbox). map_bounds, propagation_updated, and advance_now_cursor all
skip the score push — pack_scores, schedule_bounds_update,
schedule_preload_forecast, the :flush_bounds and :preload_forecast
handle_info clauses, and the start_async :initial_scores lifecycle
(incl. retry_initial_scores event) are all deleted.
Net result: instant band toggles after the first paint, no LiveView
WebSocket payloads larger than the timeline metadata, and ~370 fewer
lines in MapLive.
Three changes that shrink first-paint and make layer toggles instant:
1. Server: /weather/cells now supports ?format=bin and ?layers=a,b,c. The
binary "WCEL" pack is ~3× smaller than JSON (Float32 columns, no key
overhead) and parses with DataView + typed-array views — essentially
memcpy on the client.
2. Client: replaced JSON parse + per-cell SVG <rect> with a custom
Leaflet canvas layer. The pack is columnar (lats/lons + Float32Array
per layer) and each draw is one fillRect per cell against a
precomputed 256-step color LUT. At low zoom (CONUS, ~80k cells) SVG
paint cost dominated; canvas keeps it under one frame.
3. Lazy prefetch: the initial fetch only requests the *current* layer
(~1/19 the bytes). Right after paint, a single background fetch
pulls every other layer into the same pack. Layer switches that land
after the prefetch finishes are pure recolor — no network, no
server CPU.
The /weather/tiles endpoint stays for backward compat. The JSON path of
/weather/cells stays too — the binary path is opt-in via ?format=bin.
Previously every layer toggle triggered 21 fresh tile requests with the
new layer in the URL — ~21 server-side SVG renders, plus the round-trip
overhead. Even with the GridCache fix the server still spent real CPU
generating thousands of <rect>s per tile, and the user felt it.
Add /weather/cells: a single JSON endpoint returning every layer field
for every cell in the requested viewport. The hook fetches once on
mount, time change, or pan/zoom, caches the cells in memory, and paints
them via L.svgOverlay. Layer switches now re-color the same cached
cells locally — zero network, zero server CPU.
The single SVG uses lat/lon coords with preserveAspectRatio="none";
Leaflet stretches it linearly to the bbox. There is mild equirect→
mercator distortion at low zoom over CONUS but it's imperceptible at
typical use (z6+) and the layer-toggle UX win is large.
Tile endpoint kept for back-compat. svgOverlay opacity matches the
prior tile renderer's 0.55 fill-opacity.
renderLayer was called during mounted() before selectedTime was seeded
from the dataset, so buildWeatherTileUrl returned null and no tile
layer was added. The overlay only appeared when the user clicked a
forecast hour and update_weather_overlay fired. Seed selectedTime
(and timelineData) from data attributes before the first renderLayer.
Move the /weather render path off raw HRRR profile decoding + per-cell
SoundingParams + WeatherLayers derivation. The dominant cost was
re-deriving the same scalar fields on every viewport pan and timeline
scrub.
Server side:
- New Microwaveprop.Weather.ScalarFile persists pre-derived scalar
rows on NFS, bucketed into 5°×5° chunk files under
<base>/weather_scalars/<iso>/<lat_band>_<lon_band>.etf.gz. Viewport
reads only decode the chunks that overlap the requested bounds;
point-detail clicks read exactly one chunk.
- weather_grid_at/2 and weather_point_detail/3 prefer ScalarFile;
ProfilesFile is now the cold-start fallback only.
- warm_grid_cache_and_broadcast/1 and warm_grid_cache_from_latest_profile/0
also persist the scalar artifact, and the cold-derive path kicks
off a per-valid_time-locked async materialization so the next
reader gets the cheap path.
- NotifyListener.handle_propagation_ready/1 fires
Weather.materialize_scalar_file/1 in a detached Task on the Rust
pipeline's NOTIFY propagation_ready, so steady-state forecast hours
arrive with their scalar artifact already on disk.
- available_weather_valid_times/0 unions ScalarFile + ProfilesFile so
the timeline survives an aggressive retention sweep on either side.
Browser side (finding #8):
- Mount the legend Leaflet control once and patch its inner content
on layer changes instead of remove + re-add on every renderLayer.
- renderTimeline now keys off the rendered button set: selection-only
updates take an applyTimelineSelection patch path that restyles
existing buttons in place. Full innerHTML rebuild only fires when
timelineData itself changes.
Also fixes a credo nesting-depth flag in tile_renderer.ex by extracting
interpolate_pair/2 from the inner anonymous function.
Click a row's info area to expand a Leaflet mini-map centered on that
pin, defaulting to ESRI World_Imagery satellite (zoom up to 21) with
OSM/Topo toggles in a top-right layer control.
Adds Microwaveprop.Canopy module that reads 1° × 1° uint8 per-degree
canopy-height tiles (mirrors SRTM .hgt naming/layout, 30 m resolution).
Returns 0 m for areas with no tile on disk so the lookup is safe to
thread through the existing path-clearance pipeline before any data
lands.
Wires canopy into:
- PathTerrain.obstacle_top: per-sample blocker = ground +
max(building_m, canopy_m), so the rover ranks paths through forests
as obstructed even when no buildings sit on the line
- CandidateDetail profile: each sample now carries canopy_m alongside
building_m
- Rover-detail SVG: green "trees" polygon stacked on terrain, under
the red building polygon
- Path calculator elevation chart: green Tree Canopy dataset between
terrain and buildings
Data prep (download Potapov 2019 / Lang 2022 GeoTIFF, slice to per-degree
uint8 tiles, drop in /data/canopy/) is a separate one-shot operation —
without tiles, lookups return 0 and the existing behavior is unchanged.
Path calculator now queries Microsoft Building Footprints for the
tallest structure within 80m of each profile sample and renders a red
"Buildings" dataset on top of the terrain layer in the elevation chart.
Also normalize on-disk grid HRRR cells to include the legacy
:min_refractivity_gradient / :surface_refractivity / :ducting_detected
keys (mapped from :native_min_gradient and :duct_count). Path-calculator
crashed in prod with KeyError when a path's HRRR points came from the
new on-disk grid format instead of the DB profile shape.
Phase 2/3/5 of the building-blockage support:
* Parser streams csv.gz tiles into compact records (centroid, radius,
height) — keeps RAM ~32 B per polygon, drops -1.0-height entries.
* Index buckets records into 0.01° (~1 km) grid cells in a public ETS
table for concurrent reads; max_height_near/3 and records_near/3
scan only the 9 nearest buckets.
* Loader lazily parses any cached quadkey before a Calculate so the
scorer is terrain-only when tiles aren't on disk yet.
* Rover.PathTerrain now treats each path-sample obstacle as
terrain_elev + max_local_building_height, so blocked LOS through
recent construction actually reduces clearance.
* Selecting a candidate pushes a candidate_buildings event with the
buildings near each rover→station path; the hook renders them as
height-coloured circles (yellow <10 m, orange 10-30 m, red >=30 m)
with a tooltip showing height in meters.
- Detail panel now reads as an aiming card: "Aim 220° (SW) · clearance +12 ft"
with the precise bearing in degrees, plus rover/obstacle elev on a second line
- Elevation profile labels swapped from min/max-of-profile to start (rover) and
end (station) elevations, since their positions in the chart now match
- Map draws a dashed indigo polyline from the candidate to each selected
station when a candidate is opened; cleared when the panel closes
For each (rover cell, fixed station) pair, sample SRTM along the
great-circle path and add a per-link dB bonus proportional to the
rover's elevation above the highest intermediate terrain. Rover spots
on hilltops with clear sight to multiple stations now rise to the top.
Also vendor Leaflet's layers control PNGs and copy them under
priv/static/assets/css/images/ on build so /rover stops 404'ing on
layers-2x.png.
The 18-hour forecast chart on /path now shows the same weighted-
criteria panel the /map popup builds when the user clicks a grid
cell. Hovering a dot fires a server roundtrip that fetches
Propagation.point_detail at the path's midpoint for that valid_time;
results are cached client-side so re-hovering is instant.
`Propagation.point_detail/4` now returns a `:profile_source` tag —
`:exact`, `{:fallback, fallback_valid_time}`, or `:unavailable` —
distinguishing real per-hour breakdowns (Rust pipeline writes a
profile file for every f00..f18 step) from rare cycle-miss fallbacks
where we approximate from a recent analysis profile. Stale comment
about "only f00 persists profiles" updated.
Map utilities (FACTOR_META, FACTOR_ORDER, factorBar,
factorExplanation, scoreTier, FactorMeta, ScoreTier) are now exported
from propagation_map_hook so the new path hook reuses the exact same
look + copy.
Two follow-ups to the earlier /skewt work:
1. Interactive crosshair (matches the SPC-viewer feel from the
reference screenshot). Mousing over the diagram now drops a
horizontal pressure cursor with a top-left readout box showing the
pressure at the cursor's altitude (mb), the height in m and ft,
and the linearly interpolated temperature and dewpoint at that
level. Two coloured dots track the T and Td traces along the
cursor so the user can read the spread visually.
Mechanics: SkewtSvg.geometry/0 exposes the plot geometry (left,
right, top, bottom, p_bottom, p_top, t_min, t_max, skew) as JSON;
SkewtSvg emits an initially-hidden <g class="skewt-cursor"> layer.
A new Skewt hook in assets/js/app.ts inverts pressure_y to recover
pressure from the cursor's viewBox-Y, walks the (descending-pres-
sorted) profile to interp T/Td/height, and updates the elements.
The hook also handles mouseleave to hide the cursor and uses the
SVG's screen-CTM so the math stays correct under any container
aspect-ratio resize.
2. DB fallback for `available_valid_times`/`load_profile`. When the
on-disk `ProfilesFile` has nothing for the location-and-window
(true on dev, transient on prod between chain runs), SkewtLive
now falls back to `HrrrProfileLookup.{list_valid_times_near,
read_point_near}/3` against the `hrrr_profiles` Postgres table.
The lookup uses the same ±0.07° spatial tolerance and the
`(lat, lon, valid_time)` unique index that `Weather.find_nearest_
hrrr/3` already relies on, so the queries are index-only.
Returned shape matches `ProfilesFile.read_point/3` so SkewtLive
reads from either source transparently. NaiveDateTime → UTC
normalisation is centralised in `ensure_utc/1` so the LiveView
sees `%DateTime{time_zone: "Etc/UTC"}` regardless of which side
produced the value.
Tests: 6/6 new HrrrProfileLookupTest cases (list/limit/tolerance,
read nearest cell, nil for far cells, picks closest among matches),
3/3 SkewtLiveTest still green, 2,894 total Elixir tests + 221
properties green via mix test (one pre-existing flaky test in
admin/contact_edit_live re-runs clean). mix credo --strict clean.
mix assets.build clean.
850 mb T alone shows that warm air is present aloft but doesn't reveal
whether it's the textbook cap structure — a warm 700 mb above an EML
plume. The three new layers expose the canonical capping diagnostics
visually:
* T @ 700 mb — ≥10 °C is the southern Plains "moderate cap" threshold
* Td @ 700 mb — wide T-Td depression at 700 mb signals the EML plume
* Lapse 850→700 — steep mid-layer lapse rate (≥7 °C/km) over a moist
boundary layer is the cap mechanism itself
All three derive purely from the existing pressure-level profile (Rust
already fetches up through 700 mb), so no Rust pipeline changes needed.
The Rust f01..f18 pipeline writes ProfilesFile cells, and the Rust
hrrr_points worker writes hrrr_profiles rows, with profile entries
keyed "pres_mb" / "hght_m" / "tmpc" / "dwpc" (string keys pre-
atomization; atom keys post-ProfilesFile.read). Elixir's
SoundingParams.derive and WeatherLayers.sort_profile filtered on
"pres" / "hght" — so every derived field returned nil for any
Rust-provided profile.
Visible symptoms on /weather: N-gradient, Refractivity, T @ 850mb,
Td @ 850mb, Lapse Rate, Inversion, Inv. Base all rendered no overlay.
For per-contact analysis, min_refractivity_gradient from the Rust
HRRR point worker's rows silently dropped.
- SoundingParams.normalize_profile_entry/1: single-source normalizer
accepting legacy, Rust-string, and Rust-atom shapes; returns the
canonical "pres"/"hght"/"tmpc"/"dwpc"/"drct"/"sknt" shape.
- SoundingParams.derive/1 and WeatherLayers.sort_profile/1 both run
every entry through it before the rest of the existing logic.
- Weather.build_grid_cache_row/4 now falls back to native_min_gradient
when SoundingParams can't derive one (surface-only profiles).
Also:
- Default /weather overlay changed to Temperature (LiveView assign +
JS fallback in the hook's dataset default) per user request.
Replaces the SVG schematic on /eme with a textured WebGL scene: NASA
Blue Marble Earth, real-size Moon, outbound beam, dashed return ray
to the sub-lunar point, and a red shading on the anti-moon hemisphere
that marks who can't see the Moon (outside the bounce footprint).
Three.js (~680 kB) now lives in its own chunk; esbuild --splitting
keeps it out of the main bundle so it only loads when a user hits
/eme. Main app.js drops from 1.6 MB to 922 kB.
Also swaps the Elevation/SNR rows so the badge comes before the number.
**Wire format — /map scores payload.** Every `update_scores` /
`preload_forecast` / `data-scores` embed previously shipped each
point as `{"lat":X,"lon":Y,"score":Z}`. At 95k cells in full CONUS
that's ~36 bytes/point × 95k = 3.4 MB of JSON, ~50% of it repeating
the three key names. Packing as `[lat, lon, score]` tuples drops
per-point overhead to ~19 bytes → ~1.8 MB on the wire, ~45% smaller.
New `Propagation.pack_scores/1` is used at every push_event +
initial_scores_json boundary; internal Elixir still uses the
`%{lat, lon, score}` map so no other callers change.
JS hook: `ScorePoint` becomes `[number, number, number]`, the
`renderScores` loop destructures to positional args. Regression
test in map_live_test updated to match the new shape.
**ProfilesFile compression.** Rust writer was at gzip level 6
(default). Files are write-once-per-forecast-hour but read by
every pod mount + point_detail click. Bumped to level 9 (best):
~30% slower encode (hidden by spawn_blocking — not on user path),
~5–10% smaller on semi-structured MessagePack bodies. No reader
changes needed — same gzip stream.
**TypeScript strictness.**
- `app.ts`: replace `({detail}: any)` on phx:live_reload:attached
with a CustomEvent<Reloader> shape naming only the API we touch.
- `weather_map_hook.ts`: drop the `(this as any)._map` pair by
declaring a `GridLayerWithMap = L.GridLayer & { _map: L.Map }`
intersection on the overlay's createTile `this` parameter.
- `global.d.ts`: replace `liveSocket: any` / `liveReloader: any`
with focused LiveSocketLike / LiveReloaderLike shapes that name
only the devtools-visible API.
No more `any` in the codebase (verified via rg).
