|
|
2da74c5cd8
|
feat(telemetry): wide instrumentation + bump hrrr to 2 per pod
Config:
- runtime.exs hrrr queue 1 → 2 (6 concurrent HRRR jobs across 3 pods)
New helper Microwaveprop.Instrument.span/3 wraps :telemetry.span with
a [:microwaveprop | event_suffix] prefix so metrics can key off it.
Spans added (each emits duration + result tag):
- HrrrClient.fetch_grid / fetch_profile / fetch_idx
- NexradClient.fetch_frame + decode_png
- IemClient.fetch_asos / fetch_raob
- GefsClient.fetch_grid_profiles
- NarrClient.fetch_profile_at
- UwyoSoundingClient.fetch_sounding
- ElevationClient.fetch_elevation_profile
- Weather.upsert_hrrr_profiles_batch / upsert_gefs_profiles_batch
- Propagation.replace_scores
- PropagationGridWorker.compute_scores_algorithm
- TerrainAnalysis.analyse
- CommonVolumeRadarWorker.aggregate_stats
Telemetry catalog (MicrowavepropWeb.Telemetry):
- Oban job duration / queue_time / count / exception by (worker, queue, state)
- Per-span summary metrics for every instrumented phase above
- Periodic (10s) poller emits oban queue depth by (queue, state) — drops
into the /admin/dashboard Metrics tab immediately
Also drops the now-redundant "fetching n0q frame" and "fetching <url>"
info lines from CommonVolumeRadarWorker / NexradClient; the span events
cover that and the worker's "ingested" line stays for per-job signal.
|
2026-04-18 16:33:34 -05:00 |
|
