Python pskr_mqtt_listen.py:
- Guard against malformed CONNACK/SUBACK/PUBLISH packets
- Fix _s() truthiness: is not None instead of if v (zero SNR was lost)
- EINTR-safe select loop, OOB data handling, VBInt overflow check
- Proper socket cleanup with try/finally + DISCONNECT on shutdown
Elixir:
- Log dropped spot errors in aggregator (was silently discarded)
- Wire retain_scores_window into NotifyListener chain completion
- Recurse sweep_tmp_dir into band/weather_scalars subdirectories
- Rescue recalibrator.run/0 to write failed status row on crash
- Handle nil in fmt_snr/fmt_callsigns (no more nil dB crashes)
- Replace Stream.run with Enum.reduce logging exits in poll_worker
- Handle File.stat race in ms_footprints prune, grid_center nil log
- Fix unused variables, stale comments, missing get_path!/1
Rust:
- Graceful JoinError handling in fetcher/hrdps_fetcher (no more panics)
- round_to_5min returns Option (leap-second safe)
- Acquire/Release ordering on shutdown flags (was Relaxed)
- Parameterized NOTIFY in db.rs, defensive scalar sweep, NaN guard
- OsString::push for tmp naming, clippy fixes
Previously required ≥6-char locators; 4-char fields (~70×100 km)
were dropped. They're less precise for HRRR calibration but still
useful for spot display and coarse path analysis. Lowers the
minimum from 6 to 4 characters.
Co-Authored-By: Claude <noreply@anthropic.com>
Some FT8/Q65 multi-decoder spots omit sender/receiver callsigns
from the JSON payload, but the MQTT topic routing path always
carries them in positions 5 (sc) and 6 (rc):
pskr/filter/v2/<band>/<mode>/<sc>/<rc>/<sl>/<rl>/<sa>/<ra>
Client now extracts callsigns from the topic and threads them
through Aggregator.ingest → Pskr.parse_spot, where they act as
fallback when the JSON sc/rc keys are nil or empty.
JSON payload values still take precedence when present — the topic
is only used as a safety net.
Co-Authored-By: Claude <noreply@anthropic.com>
A 4-char locator covers ~70 km × 100 km, which dwarfs HRRR's 3 km
native cell and any midpoint we'd derive for the calibration corpus.
Storing those samples gives the recalibrator geometry that's
indistinguishable from noise.
fetch_grid/2 now returns {:error, {:short_grid, key, grid}} for any
locator under 6 chars after Maidenhead validation. The spot is
dropped before it reaches the aggregator, so neither pskr_spots_hourly
nor pskr_calibration_samples ever see it.
Spots that include subsquare (EM12kl), extended-square (EM12kl37),
or extended-extended-subsquare (EM12kl37ab) precision now keep
that resolution end-to-end. Previously we truncated everything to
4 chars at parse time, which collapsed distinct paths into the same
row whenever two spotters shared a field.
The aggregation key uses the full grid as reported, so two
spotters in different subsquares of `EM12` produce two rows —
correct, since they are physically different paths. Any spot that
fails Maidenhead validation (odd length, illegal char at position
N) is still dropped via the same error path.
Storage normalizes to uppercase so the unique index doesn't split
on case. Maidenhead.valid?/1 already accepts both cases on input.
Subscribes to mqtt.pskreporter.info:1883 over plain TCP and folds
every reception report into hourly aggregates per
(band, sender_grid_4, receiver_grid_4). Each spot is the empirical
"propagation actually occurred on this path right now" signal we'll
correlate against HRRR atmospheric state to recalibrate the scoring
weights. Aggregating at the path-hour bucket collapses 50-reporter
QSOs to one row instead of 50.
Topic filter anchors on USA (DXCC 291) on either sender or receiver
side so the broker pre-filters out everything outside our HRRR
domain. Bands subscribed: VHF and up (6m, 2m, 70cm, 23cm, +
microwave) — HF is dominated by ionospheric propagation, which this
project doesn't model.
Pskr.Client cluster-elects a singleton via :global.register_name —
all replicas start the GenServer but only one connects to MQTT;
the rest stay in :standby and watch for the leader's nodedown to
re-run election. Off in dev/test, on by default in prod
(PSKR_MQTT_ENABLED=false as kill-switch).
Pskr.Aggregator buffers in memory keyed by Pskr.path_key/1 and
flushes every 60s via Repo.insert_all/3 with an additive
ON CONFLICT (count summed, SNR envelope by GREATEST/LEAST,
modes unioned via unnest+DISTINCT). Idempotent across overlapping
flushes and across leader handover.
Dockerfile sets BUILD_WITHOUT_QUIC=1 — emqtt's transitive quicer
dep wants CMake + OpenSSL headers we'd otherwise have to add to
the builder image just to never use QUIC over plain MQTT. Base
image is unchanged; the new dep compiles cleanly into the existing
prop-base runtime.
