prop/lib/microwaveprop/space_weather
Graham McIntire 26ef50c965
fix(ingest): dedupe ASOS duplicates + shrink SWPC decode errors
Two unrelated production warnings from the same log window:

WeatherFetchWorker raised Postgrex 21000 cardinality_violation on
ASOS upserts when IEM returned two METAR records for the same
timestamp (routine + special at the same minute). Dedupe by
(station_id, observed_at) in upsert_surface_observations, keeping
the last occurrence since IEM's ordering is chronological.

SwpcClient.fetch_xrays warned with the full truncated response
body (~650 KB of GOES X-ray JSON) dumped into the log aggregator
whenever SWPC's CDN cut the response mid-stream. Intercept
Jason.DecodeError in fetch_and_parse and format it short: keep
the byte position and an 80-char preview, drop the rest.
2026-04-21 16:39:17 -05:00
..
geomagnetic_observation.ex SpaceWeather: SWPC JSON ingestion (Kp, F10.7, GOES X-ray) 2026-04-15 14:55:24 -05:00
solar_flux_observation.ex SpaceWeather: SWPC JSON ingestion (Kp, F10.7, GOES X-ray) 2026-04-15 14:55:24 -05:00
solar_xray_observation.ex SpaceWeather: SWPC JSON ingestion (Kp, F10.7, GOES X-ray) 2026-04-15 14:55:24 -05:00
swpc_client.ex fix(ingest): dedupe ASOS duplicates + shrink SWPC decode errors 2026-04-21 16:39:17 -05:00