The 1-hour valid_time filter inherited from /weather is wrong for
HRDPS: the model publishes 4x/day with 3-4h publication latency and
the chain step adds ~13 min on top, so the freshest data on disk is
routinely 4-6h old. With the 1h cutoff in place, /weather-ca's
timeline was empty whenever there wasn't an active chain step running,
which left the JS hook with nothing to fetch. The sidebar already
shows the selected valid_time prominently — users can see for
themselves how stale the data is and don't need the page to hide it.
New LiveView at /weather-ca duplicates the /weather UI but defaults the
viewport to central Canada (55N, -100W, z=4) and renders the map div
with data-source="hrdps". The JS hook reads that attribute and appends
&source=hrdps to every cell fetch; the WeatherTileController routes
those reads through Weather.weather_grid_hrdps_at/2, which only reads
the <vt>.hrdps scalar dir and skips HRRR merging entirely. Timeline is
sourced from ScalarFile.list_valid_times_hrdps/0 so HRDPS-only hours
show up even when no HRRR file exists for the same valid_time.
Also drops "— Unified" from the algo.md H1.
The HRDPS pipeline as written was unusable in production: each chain
step took ~5 hours (not 30-90s as the dev-bench claim) because every
`wgrib2 -lon` batch redundantly JPEG2000-decodes ~150 records. With 57
batches × 18 forecast hours, a single cycle would take days to drain.
The /weather Canadian view only needs the current hour, so the cheapest
fix is to stop seeding the rest of the chain entirely:
* Rust grid: HRDPS_STEP=0.5 (was 0.125) drops point count 16× to ~3.5k.
Coarser cells than HRRR, but /weather gets visible Canadian coverage
inside one chain step (~20 min) instead of never.
* GridTaskEnqueuer.seed_current_hour/2 inserts exactly one forecast row
whose valid_time is closest to `now`. fh is clamped to 1..18 so the
Rust F00Reserved branch never fires.
* HrdpsGridWorker switches to seed_current_hour and drops the
6-hour-cycle cron in favor of HH:35 hourly fires. Reseeds are
idempotent on the 4-column unique key.
Also fixes a pre-existing credo "nested too deep" in
ScalarFile.read_point by extracting lookup_in_chunk + hit_or_false
helpers — happened to be on the read path that surfaces this work, so
folding it into the same commit.
Adds lat/lon/zoom URL params to /weather so reload / share-link
restores the exact map state. The existing ?layer= param keeps
working alongside.
LiveView side:
- mount() reads lat/lon/zoom from query params, clamps to valid
ranges (lat -90..90, lon -180..180, zoom 4..10), and falls back
to DFW@z7 when missing or invalid. Initial values flow to the
hook via three new data attributes.
- handle_event("viewport_changed", ...) accepts {lat, lon, zoom}
pushed by the hook on debounced moveend, clamps + rounds, and
push_patches the URL with replace: true so browser history
doesn't grow an entry per pan tick.
- build_path/2 helper centralises URL construction so select_layer
and viewport_changed both produce a deterministic param order
(layer, lat, lon, zoom). Tests assert exact URLs.
JS hook side:
- mounted() reads data-initial-lat/lon/zoom from the el dataset
with parseFloat/parseInt + Number.isFinite gates, then seeds
Leaflet's center/zoom from those instead of the previous
hardcoded (32.897, -97.038, 7).
- moveend handler now also schedules a 400 ms-debounced
pushEvent("viewport_changed", {lat, lon, zoom}) so the URL
catches up with whatever the user panned/zoomed to. Separate
debounce timer from the cells fetch since the URL push and the
network fetch have different cadence sweet spots.
Tests cover: initial seeding from URL params, clamping of bad
values, viewport_changed event producing the right patched URL,
and the original layer-shareability test (now expecting the full
viewport in the URL).
The /weather map now shows HRRR-derived data wherever HRRR has
coverage and falls back to HRDPS-derived data for Canadian cells
outside HRRR's bbox. North of the 49°N HRRR-coverage line the map
stays populated instead of going blank.
Rust:
- weather_scalar_file::dir_for_hrdps + write_atomic_hrdps land HRDPS
scalar chunks at `<vt>.hrdps/` so HRRR's `<vt>/` write isn't
clobbered. The HRRR writer wipes its dir before each write — both
sources need their own dir to coexist.
- pipeline::run_chain_step_hrdps now derives a ScalarRow for each
scored cell (same derive_row that HRRR uses; same wire format) and
writes them via write_atomic_hrdps alongside the .hrdps.prop score
files.
Elixir:
- ScalarFile.dir_for_hrdps + read_bounds + read_point + exists? all
walk both `<vt>/` (HRRR) and `<vt>.hrdps/` (HRDPS). read_bounds
concatenates with HRRR-precedence on overlap (defensive — Rust
pipeline doesn't write HRRR-overlap cells, but cheap to enforce).
- Weather.weather_grid_at unchanged: it routes through ScalarFile,
which now transparently returns the union. GridCache stays
per-valid_time so a single cache entry holds both regions.
Tests cover four scenarios: HRRR-only, HRDPS-only, both-present
union, and HRRR-precedence on the same cell.
Stages 6, 7, 10 of the HRDPS plan, plus the Elixir read-side merge
that makes Rust's `.hrdps.prop` writes show up on /map.
Read side:
- `ScoresFile.path_for_hrdps/2` + `read_hrdps/2` — companions to the
HRRR equivalents. Same decode path; different filename.
- `ScoresFile.read_bounds/3` now reads HRRR + HRDPS files and
concatenates the cells. Files are disjoint by construction
(Grid.hrdps_only_points excludes CONUS) so no de-dup needed.
- `ScoresFile.read_point/4` checks `.prop`, then `.hrdps.prop`, then
legacy `.ntms` — first hit wins. Cells outside their owning region
read as no-data in the other file's grid, so the order doesn't
matter for correctness.
- `Propagation.warm_cache_and_broadcast/2` reads both files and
merges before broadcasting to the cluster ScoreCache. A single
missing file (HRDPS pre-cycle, HRRR briefly absent) is now OK —
the cache warms with whichever side is available.
Activation:
- runtime.exs cron: HRDPS seed worker fires at HH:35 of each cycle
hour (05:35Z, 11:35Z, 17:35Z, 23:35Z) — 5h after cycle, 30 min
staggered from HRRR's */15 schedule.
- HrdpsGridWorker moduledoc updated: no longer dormant; Rust
prop-grid-rs handles source='hrdps' rows.
- map_live North America bbox extended to (24.5, 60.0, -141.0, -52.0)
for the out-of-coverage banner check; visitors anywhere in CONUS
or the Canadian extent now stay banner-free.
Cleanup:
- mix hrdps.probe deleted — its risks are retired and the production
HrdpsClient covers the same surface.
What's still in motion (not blocking the Canadian /map):
- Per-valid_time profile + scalar artifacts for HRDPS (would let
/weather show Canadian temperature/refractivity layers). Rust
pipeline currently skips those so the HRRR-written companion files
aren't clobbered — separate stage when /weather UX is decided.
- FreshnessMonitor HRDPS staleness tracking — HRDPS has its own cron
and a different cadence than HRRR; HRRR-stale logic doesn't apply
cleanly. Defer until prod tells us what alarms we actually want.
Stage 3 (Elixir foundation) of the HRDPS plan. Lands the scaffolding
that lets the Rust prop-grid-rs HRDPS branch be wired up cleanly when
it ships, without further DB or Elixir-side changes.
What ships:
- `grid_tasks.source` column (default 'hrrr', backfill is a no-op).
Replaces the (run_time, forecast_hour, kind) unique index with a
4-column key (..., source) so HRRR and HRDPS analysis/forecast rows
for the same cycle coexist.
- `Grid.hrdps_only_points/0` — Canadian cells inside HRDPS bbox
(49-60°N, -141 to -52°W) but outside HRRR's CONUS bbox. Disjoint
from `conus_points/0` by construction so the two grids never
double-write the same (lat, lon).
- `Grid.point_source/1` — `:hrrr | :hrdps | :neither` classification
for a (lat, lon) pair. Routes per-QSO enrichment to the right NWP
source.
- `GridTaskEnqueuer.seed_with_analysis/2` and `seed/2` accept a
`:source` option; defaults to "hrrr" so existing callers are
unchanged.
- `HrdpsGridWorker` cycle seeder mirroring PropagationGridWorker's
shape but for HRDPS's 4×/day cadence. `:hrdps` queue added to base
Oban config.
What's deferred:
- The Rust `prop-grid-rs` worker doesn't yet branch on
`task.source`. Until that ships, `HrdpsGridWorker` stays out of
runtime.exs cron — it would just produce rows that the Rust worker
mishandles as HRRR. Module doc explicitly notes the dormant state.
- Map overlay extension and FreshnessMonitor HRDPS staleness deferred
to follow-up sessions once real data flows.
Activation path documented in the updated plan file.
Coverage cap recorded as Stage 8 decision: 60°N for v1 (SRTM stops
there). Arctic CDEM deferred to a follow-up plan.
Stage 2 of the HRDPS plan. Mirror of hrrr_profiles for HRDPS-derived
rows so HRDPS can be dropped/reprocessed without disturbing the 4500+
HRRR profiles already in prod. Same column set, same indexes,
PARTITION BY RANGE (valid_time) with quarterly partitions starting at
the deploy quarter.
Schema follows the @primary_key {:id, :binary_id, autogenerate: true}
convention and the same set of derived fields HrrrProfile carries
(surface_refractivity, min_refractivity_gradient, ducting_detected,
duct_characteristics, etc.) so the rest of the pipeline can consume
HRDPS-derived rows interchangeably with HRRR-derived rows.
Unique constraint on (lat, lon, valid_time) is enforced at the DB
level via the partitioned index; schema-level unique_constraint/2
matching against changeset errors doesn't apply because the
partition's index name varies per partition. Production upserts go
through `on_conflict: {:replace_all_except, ...}` per project
convention.
The test computed recent_t = (truncated_to_top_of_hour now) - 30min,
which is 30-90 min before real utc_now() depending on what minute the
test runs at. The LiveView's `cutoff = utc_now() - 1h` then dropped
recent_t whenever the real-time minute exceeded ~30, so the test
failed half the time.
Anchor recent_t at the top of the current hour — that's always within
the 1-hour window regardless of minute. The test still exercises the
"keep within last hour" path; just stops being clock-flaky.
Stage 1 of the HRDPS plan. Mirrors HrrrClient.fetch_grid/3's signature
so PropagationGridWorker can call either polymorphically.
Architectural shape per the probe wedge findings:
- ECCC's date-prefixed datamart URL structure
(dd.weather.gc.ca/{YYYYMMDD}/WXO-DD/model_hrdps/...)
- One-variable-per-file model — fetch ~14 small GRIB2s concurrently,
byte-concatenate into a single multi-record binary, hand to
Wgrib2.extract_points_from_file. wgrib2 reads multi-record files
natively so no -merge step is needed.
- wgrib2's -lon flag handles the rotated-lat/lon reprojection
internally — no manual rotation math.
Variable catalog covers the surface set the propagation scorer reads
plus the 1000-700 mb pressure-level set used by the refractivity
gradient. PWAT is unavailable in HRDPS f000 inventory; the scorer's
PWAT factor will fall back to its default for HRDPS-derived points.
DEPR (T-Td depression) is treated as an alternate primary; when DPT
is absent build_profile_from_extracted derives dewpoint as
temp - depression so downstream consumers see the same shape as HRRR.
Plan task 1.6 (CYYR projection sanity check vs UWYO sounding) is
covered by the live probe at lib/mix/tasks/hrdps_probe.ex which
already validated wgrib2's rotated-grid handling against five
Canadian cities; the production implementation routes through the
same Wgrib2.extract_points_from_file path that the probe used.
Bucket each cache entry into 5°×5° spatial chunks matching the layout
used by Weather.GridCache. fetch_bounds/3 now walks only the chunks
intersecting the viewport instead of the full 92k-cell CONUS map; a
typical zoomed viewport hits ~1500 cells per chunk × the chunks the
viewport overlaps instead of scanning all 92k.
Public API and observable behavior unchanged. Add cross-chunk-boundary
regression tests so future refactors can't silently flatten the layout.
Apply the same overlay refactor we did for /weather to the propagation
map. Score data used to flow as JSON via LiveView push_event
(update_scores + the 18-hour preload_forecast batch); switching bands
or scrubbing time meant a synchronous server roundtrip + ~MB JSON push
per change.
Now:
1. New /scores/cells endpoint returns a "PSCR" binary cell-pack —
columnar f32 lats/lons + u8 scores, ~3-4× smaller than JSON and
parsed via DataView + typed-array views with no JSON.parse.
2. JS hook owns score fetching. mount/moveend/band/time changes
trigger HTTP fetches against /scores/cells. After the initial
paint, every other forecast hour for the current band is fetched
in parallel in the background — timeline scrubs after the prefetch
finishes are pure cache hits with no network at all.
3. LiveView no longer pushes scores. select_band emits update_band_info
with band_mhz so the client knows what to refetch; select_time and
set_selected_time only update server-side state (URL, point detail
bbox). map_bounds, propagation_updated, and advance_now_cursor all
skip the score push — pack_scores, schedule_bounds_update,
schedule_preload_forecast, the :flush_bounds and :preload_forecast
handle_info clauses, and the start_async :initial_scores lifecycle
(incl. retry_initial_scores event) are all deleted.
Net result: instant band toggles after the first paint, no LiveView
WebSocket payloads larger than the timeline metadata, and ~370 fewer
lines in MapLive.
The HRRR scalar cache can hold a previous run's forecast hours past
their useful life — the timeline was offering "current conditions"
points that were 2-3 hours old, which is more misleading than helpful
on a map labeled "Now".
Filter Weather.available_weather_valid_times() through a `now - 1h`
cutoff in mount, :weather_updated, and :propagation_pipeline_progress
so the timeline only ever shows the most recent analysis hour plus
forecast hours.
The client moved to the binary cell-pack and canvas overlay last
commit, so the per-tile SVG endpoint, the TileRenderer module, and the
JSON shape of /weather/cells are all dead code. Remove them.
/weather/cells now always returns the binary WCEL pack (the ?format=bin
toggle is gone — there's only one format). The data-weather-tile-url
attribute and the weather_tile_url assign are dropped from the LiveView.
Also fix a latent bug surfaced once tests covered the default-layers
path: nil row values were writing the atom :nan into the binary segment
at runtime. Replaced with the explicit f32 quiet-NaN bit pattern so
Number.isNaN flags them correctly on the JS side.
Three changes that shrink first-paint and make layer toggles instant:
1. Server: /weather/cells now supports ?format=bin and ?layers=a,b,c. The
binary "WCEL" pack is ~3× smaller than JSON (Float32 columns, no key
overhead) and parses with DataView + typed-array views — essentially
memcpy on the client.
2. Client: replaced JSON parse + per-cell SVG <rect> with a custom
Leaflet canvas layer. The pack is columnar (lats/lons + Float32Array
per layer) and each draw is one fillRect per cell against a
precomputed 256-step color LUT. At low zoom (CONUS, ~80k cells) SVG
paint cost dominated; canvas keeps it under one frame.
3. Lazy prefetch: the initial fetch only requests the *current* layer
(~1/19 the bytes). Right after paint, a single background fetch
pulls every other layer into the same pack. Layer switches that land
after the prefetch finishes are pure recolor — no network, no
server CPU.
The /weather/tiles endpoint stays for backward compat. The JSON path of
/weather/cells stays too — the binary path is opt-in via ?format=bin.
Previously every layer toggle triggered 21 fresh tile requests with the
new layer in the URL — ~21 server-side SVG renders, plus the round-trip
overhead. Even with the GridCache fix the server still spent real CPU
generating thousands of <rect>s per tile, and the user felt it.
Add /weather/cells: a single JSON endpoint returning every layer field
for every cell in the requested viewport. The hook fetches once on
mount, time change, or pan/zoom, caches the cells in memory, and paints
them via L.svgOverlay. Layer switches now re-color the same cached
cells locally — zero network, zero server CPU.
The single SVG uses lat/lon coords with preserveAspectRatio="none";
Leaflet stretches it linearly to the bbox. There is mild equirect→
mercator distortion at low zoom over CONUS but it's imperceptible at
typical use (z6+) and the layer-toggle UX win is large.
Tile endpoint kept for back-compat. svgOverlay opacity matches the
prior tile renderer's 0.55 fill-opacity.
Each /weather/tiles request was running the full ScalarFile decode path
(File.read → gunzip → Msgpax.unpack → normalize) for any non-analysis
valid_time, because GridCache deliberately skipped forecast hours to keep
memory low. With 21 simultaneous viewport tiles all hitting the same few
chunk files, each tile took 1.5-3.2s of contended IO + CPU.
On a GridCache miss when a ScalarFile exists, hydrate GridCache from the
file once (deduped via the existing claim_fill primitive) and serve all
subsequent reads from ETS. Bound memory with prune_keep_latest/1 — keep
the 24 most-recently-touched valid_times, which covers a full HRRR run
(analysis + 18 forecasts) plus a few stragglers from the previous run.
In the steady state, the first viewport read for a given forecast hour
takes one full ScalarFile decode (~50-100 ms for 92k cells); every
subsequent tile, point lookup, and viewport read for the same hour
serves from ETS in <1 ms.
Move ScalarFile from a dual-writer dual-format setup (ETF on the Elixir
side, materialized via NotifyListener; nothing on the Rust side) to a
single chunked-MessagePack format that both languages produce and
consume.
Elixir side:
- ScalarFile now writes gzipped MessagePack `.mp.gz` per chunk via
Msgpax. Reader decodes with Msgpax + zlib. Atom keys are stripped to
strings on encode and re-atomized via a whitelist on read so callers
see the same shape regardless of which side wrote the bytes.
- DateTime values round-trip through ISO8601 strings.
- New cross-language wire-format test that decodes a hand-rolled
payload mirroring exactly what `rmp_serde::to_vec_named` emits, so a
format regression on either side fails the suite.
Rust side:
- New `prop_grid_rs::weather_scalar_file` module:
- 1:1 port of `Microwaveprop.Weather.WeatherLayers.derive/1` plus
the surface fields `Weather.build_grid_cache_row/4` adds on top
(lapse rate, mid lapse rate, inversion strength + base, 850/700 mb
interpolation, surface RH, surface refractivity, ducting flag).
- `write_atomic` chunks rows into 5°×5° buckets, writes one
`<lat_band>_<lon_band>.mp.gz` per chunk via tmp + rename, drops
pre-existing chunks first to keep writes canonical.
- 5 unit tests covering surface-only rows, upper-air derivation,
write/read round-trip, chunk band boundaries, and the
out-of-physical-range surface-temp filter.
- Pipeline integration: both `run_chain_step` (f01..f18) and the f00
analysis path now build `Vec<ScalarRow>` in the same merged-cell
pass that builds `CellEntry` and `Conditions`, then fire a parallel
`spawn_blocking` write that overlaps with scoring. Awaiting the
scalar future after the score-band writes keeps the failure-surface
symmetric with the existing profile_future.
Result: every new forecast hour lands with the scalar artifact already
on disk, so /weather reads never fall through to ProfilesFile decode +
per-cell SoundingParams + WeatherLayers derivation. The Elixir-side
`NotifyListener.handle_propagation_ready/1` materialization remains as
an idempotent safety net for hours that pre-date this commit.
Move the /weather render path off raw HRRR profile decoding + per-cell
SoundingParams + WeatherLayers derivation. The dominant cost was
re-deriving the same scalar fields on every viewport pan and timeline
scrub.
Server side:
- New Microwaveprop.Weather.ScalarFile persists pre-derived scalar
rows on NFS, bucketed into 5°×5° chunk files under
<base>/weather_scalars/<iso>/<lat_band>_<lon_band>.etf.gz. Viewport
reads only decode the chunks that overlap the requested bounds;
point-detail clicks read exactly one chunk.
- weather_grid_at/2 and weather_point_detail/3 prefer ScalarFile;
ProfilesFile is now the cold-start fallback only.
- warm_grid_cache_and_broadcast/1 and warm_grid_cache_from_latest_profile/0
also persist the scalar artifact, and the cold-derive path kicks
off a per-valid_time-locked async materialization so the next
reader gets the cheap path.
- NotifyListener.handle_propagation_ready/1 fires
Weather.materialize_scalar_file/1 in a detached Task on the Rust
pipeline's NOTIFY propagation_ready, so steady-state forecast hours
arrive with their scalar artifact already on disk.
- available_weather_valid_times/0 unions ScalarFile + ProfilesFile so
the timeline survives an aggressive retention sweep on either side.
Browser side (finding #8):
- Mount the legend Leaflet control once and patch its inner content
on layer changes instead of remove + re-add on every renderLayer.
- renderTimeline now keys off the rendered button set: selection-only
updates take an applyTimelineSelection patch path that restyles
existing buttons in place. Full innerHTML rebuild only fires when
timelineData itself changes.
Also fixes a credo nesting-depth flag in tile_renderer.ex by extracting
interpolate_pair/2 from the inner anonymous function.
Refit on the current contact corpus via mix recalibrate_scorer (val loss
0.140 vs 0.146 train, initial 0.434). Mostly small adjustments — the
notable shifts are time_of_day −0.0116 (now the smallest factor) and
rain +0.0069 (continues to be the largest single weight).
Sums to 1.0 (within rounding); per-band overrides not affected.
`mix prop.compare` runs both scorers against a recent contact sample
and measures both against achieved contact distance. Each run appends
to priv/calibration/history.jsonl, so trends in alg-ML divergence and
algorithm-distance correlation surface over weeks of running.
When drift exceeds threshold (alg-ML RMSE > 8 score points, or current
correlation > 0.10 below the history median), the task writes a
recommendation pointing at the right remediation:
mix recalibrate_scorer # algorithm drift → refit weights
mix propagation_train # ML drift → retrain on the new algorithm
That is the feedback loop: measure → recommend → recalibrate/retrain →
loop. Operational rather than automatic, so a bad data window can't
silently corrupt the model.
Pure analysis lives in Microwaveprop.Propagation.Calibration with its
own unit tests; the Mix task only handles data loading, file I/O, and
console formatting.
- Unify haversine_km / deg_to_rad: Geo is the canonical home (atan2
form for antipodal stability); Radio, common_volume, rain_scatter,
ionosphere, terrain/srtm, terrain/elevation_client now delegate.
- Drop safe_min/safe_max wrappers in favor of Enum.min/max defaults.
- Move parse_float / parse_int to MicrowavepropWeb.LiveHelpers; eme,
path, and rover LiveViews import from there.
- Extract MicrowavepropWeb.LocationResolver from the eme/path
resolve_source / resolve_location duplicate. Standardize the kind
key and "Invalid grid square" error message.
- Convert Scorer cond chains (humidity, td_depression, sky, wind,
rain, pwat, pressure) to multi-clause guarded defs, matching the
existing classify_time_period style.
- extract_profile_fields uses a named map accumulator instead of a
6-tuple; build_path_conditions guards on %{temps: []} / %{dewpoints: []}.
- Collapse scores_file clamp/3 to max |> min.
- humidity_effect_label lives in BandConfig; map_live and path_live
both use it.
Retrained on 60k month-balanced HRRR profiles through 2026-04-05
(test RMSE 1.7 score points, R² 0.97). Added explain_prediction/4
returning ranked feature contributions via batched finite-difference
attribution, so the UI can show users which weather factors drove
each prediction.
Adds Microwaveprop.Radio.TextSanitizer to remove BOM, zero-width
spaces/joiners, C0 control bytes (except CR/LF/TAB), and DEL, and to
collapse NBSP and other unicode whitespace into a regular space.
CSV: applied to the whole upload at the top of preview/2.
ADIF: applied per-field-value after extraction so the byte-counted
length tags still slice the right substring.
New globally-shared list of rover-friendly (or off-limits) parking
spots. Anyone can view; logged-in users can add new locations and
edit/delete their own. Each location stores lat/lon, free-text notes,
and a status enum (:ideal | :off_limits).
* Microwaveprop.Rover.Location schema + migration
* Rover.list_locations/0, create_location/2, update_location/3,
delete_location/2 (mutations require User; ownership-scoped)
* /rover-locations LiveView in the public live_session — Add button
is disabled for anonymous visitors, Edit/Delete buttons render
only on the user's own entries
* Tests for context + LiveView covering anonymous read, owner-only
edit/delete, and form submission
Adds Microwaveprop.Canopy module that reads 1° × 1° uint8 per-degree
canopy-height tiles (mirrors SRTM .hgt naming/layout, 30 m resolution).
Returns 0 m for areas with no tile on disk so the lookup is safe to
thread through the existing path-clearance pipeline before any data
lands.
Wires canopy into:
- PathTerrain.obstacle_top: per-sample blocker = ground +
max(building_m, canopy_m), so the rover ranks paths through forests
as obstructed even when no buildings sit on the line
- CandidateDetail profile: each sample now carries canopy_m alongside
building_m
- Rover-detail SVG: green "trees" polygon stacked on terrain, under
the red building polygon
- Path calculator elevation chart: green Tree Canopy dataset between
terrain and buildings
Data prep (download Potapov 2019 / Lang 2022 GeoTIFF, slice to per-degree
uint8 tiles, drop in /data/canopy/) is a separate one-shot operation —
without tiles, lookups return 0 and the existing behavior is unchanged.
Compute.run now accepts a `progress` callback and reports human-readable
labels before each pipeline step (loading grid, looking up elevation,
checking road access, scoring cells, etc.). RoverLive feeds it via
send/2 to itself and renders the latest label as small text to the left
of the Calculate button while it's running.
Adds a per-cell building-clutter penalty so the algorithm down-ranks
spots in built environments where buildings on multiple sides
scatter/block signal regardless of which station you're aiming at.
Penalty = max_height_within_75m / 5 dB, capped at 6 dB. Path-clearance
already accounts for buildings ON the link path; this is the
"surrounded by stuff" signal.
Phase 2/3/5 of the building-blockage support:
* Parser streams csv.gz tiles into compact records (centroid, radius,
height) — keeps RAM ~32 B per polygon, drops -1.0-height entries.
* Index buckets records into 0.01° (~1 km) grid cells in a public ETS
table for concurrent reads; max_height_near/3 and records_near/3
scan only the 9 nearest buckets.
* Loader lazily parses any cached quadkey before a Calculate so the
scorer is terrain-only when tiles aren't on disk yet.
* Rover.PathTerrain now treats each path-sample obstacle as
terrain_elev + max_local_building_height, so blocked LOS through
recent construction actually reduces clearance.
* Selecting a candidate pushes a candidate_buildings event with the
buildings near each rover→station path; the hook renders them as
height-coloured circles (yellow <10 m, orange 10-30 m, red >=30 m)
with a tooltip showing height in meters.
Phase 1 of building-blockage support: implements quadkey math (Bing/MS
tile encoding), looks up the MS dataset-links index for UnitedStates,
and downloads csv.gz quadkey tiles to /data/buildings (overridable via
:buildings_cache_dir). Index is cached 24h; tiles are written once and
reused. No path-analysis wiring yet — that's the next slice.
- Local prominence: each cell's elevation vs. its 8-neighbor ring
(~1.3 km radius) feeds a small additive bonus so broad hilltops
rank above isolated SRTM voxels
- Road proximity: one Overpass call per Calculate fetches drivable
ways inside the bbox; cells beyond ~0.5 km of any road get a
light dB penalty, gated off in tests
For each (rover cell, fixed station) pair, sample SRTM along the
great-circle path and add a per-link dB bonus proportional to the
rover's elevation above the highest intermediate terrain. Rover spots
on hilltops with clear sight to multiple stations now rise to the top.
Also vendor Leaflet's layers control PNGs and copy them under
priv/static/assets/css/images/ on build so /rover stops 404'ing on
layers-2x.png.
