2026-06-08
fix: reduce health-check probe pressure to stop readiness-flapping downtime
  Three changes to eliminate the root cause of production up/down alerting:
  1. BruteForceProtection plug now skips /health and /health/live paths (like
     /socket/agent already does), removing an unnecessary Repo.get_by(IpBlock)
     query from every k8s readiness probe.
  2. k8s readiness probe relaxed: periodSeconds 5→10, timeoutSeconds 2→5,
     failureThreshold 2→3 to tolerate brief DB/Redis blips without marking the
     pod unready; successThreshold 3→2 to recover faster after a blip clears.
  3. RedisHealthCheck.check_health/2 now sends PING on the persistent
     Towerops.Redix connection (started at boot) instead of opening a brand-new
     TCP connection + AUTH + PING + close on every probe. Towerops.Redix.Fake
     gained a set_ping_response/2 toggle so tests can drive the failure path
     without dialing unreachable hosts.
  Files: lib/towerops/redis_health_check.ex, lib/towerops/redix/fake.ex,
    lib/towerops_web/plugs/brute_force_protection.ex, k8s/deployment.yaml,
    test/towerops_web/controllers/health_controller_test.exs

2026-05-20
fix(oban): use Oban Pro DynamicLifeline in prod to stop unique_violation crash loop
  CheckExecutorWorker's `unique` constraint covers available/scheduled/retryable
  (executing excluded so perform/1 can self-schedule). When a pod is killed
  mid-execution the job is orphaned in `executing`; core Oban.Plugins.Lifeline
  rescues it back to `available`, where it collides with the already-scheduled
  successor (same uniq_key) on oban_jobs_unique_index, raising 23505. Core
  Lifeline has no unique_violation handling, so the plugin crashed every 60s and
  ~150 orphans never cleared. Oban Pro's DynamicLifeline (the Smart-engine
  plugin) catches the 23505 via Smart.clear_uniq_violation and repairs it.
  Prod only (runtime.exs, Smart engine); dev keeps core Lifeline (Basic engine).
  DynamicLifeline rescues by producer records, so rescue_after was dropped.
  Files: config/runtime.exs, config/dev.exs

feat(health): add shallow /health/live liveness endpoint and split k8s probes
  /health/live returns 200 without querying db/redis. k8s liveness and startup
  probes now target /health/live so a transient db/redis outage can no longer
  kill or block boot of an otherwise-healthy pod; readiness keeps the deep
  /health (db+redis) to gate load-balancer membership only. Deployment bumped to
  2 replicas (NFS data volume supports multi-pod mounts; app already clusters).
  Files: lib/towerops_web/router.ex, lib/towerops_web/controllers/health_controller.ex,
  test/towerops_web/controllers/health_controller_test.exs, k8s/deployment.yaml

2026-05-14
fix(monitoring): add Oban unique constraint to CheckExecutorWorker to stop duplicate job buildup
  CheckExecutorWorker had no `unique` constraint. Every insert path —
  self-scheduling (schedule_next_check/1), Monitoring.schedule_check/1 called
  unconditionally by snmp.ex discovery (create_check/1 is an upsert that returns
  {:ok, check} even for existing checks), JobHealthCheckWorker, the check form,
  API v1 and GraphQL — created a fresh job row. Every SNMP discovery pass
  re-enqueued a job for every existing check, so the check_executors queue grew
  to ~70k jobs for ~5.7k checks (~12-45 dupes/check) and could not drain.
  - lib/towerops/workers/check_executor_worker.ex: added
    `unique: [keys: [:check_id], states: [:available, :scheduled, :retryable],
    period: :infinity]`. `:executing` is excluded so perform/1's self-reschedule
    does not conflict with its own running job.
  - config/dev.exs: added the `checks` and `check_executors` Oban queues, which
    were missing from dev (prod's runtime.exs already had them) — dev never
    processed those queues at all.
  - test/towerops/workers/check_executor_worker_test.exs: added "job uniqueness"
    tests covering dedup on insert, dedup via schedule_check/1, and that an
    existing queued job does not block perform/1 from rescheduling.

fix(sync): classify unexpected 5xx statuses as transient so vendor syncs back off and retry
  Vendor API clients (Preseem, etc.) return errors shaped as
  `{:unexpected_status, status, body}`, but `SyncErrors.transient?/1` only
  had explicit clauses for the `{:http_error, ...}` shape. A 502 Bad Gateway
  was retried only by accident via the catch-all, and a 4xx in the same
  shape would have been retried 20 times incorrectly.
  - lib/towerops/workers/sync_errors.ex: added explicit
    `{:unexpected_status, status}` / `{:unexpected_status, status, _}`
    clauses — 5xx is transient (Oban retries with exponential backoff),
    4xx is permanent
  - test/towerops/workers/sync_errors_test.exs: new test coverage for the
    classifier across permanent, transient, and unknown error shapes
  Surfaced by PreseemSyncWorker hitting 502s from api.preseem.com after
  IPv6 was enabled in prod.

2026-05-12
fix(session): resolve session salts at runtime so prod release boots
  The `@session_options` module attribute used `Application.compile_env`,
  which baked compile-time placeholder values into the endpoint while
  `config/runtime.exs` set the real values from env vars. Phoenix detected
  the mismatch and refused to start (the failing migrate Job in k8s).
  - config/config.exs: removed hardcoded session salts (no compile-time
    binding for these keys now)
  - config/dev.exs, config/test.exs: stable per-env salts so local + CI
    don't need SESSION_*_SALT env vars
  - lib/towerops_web/endpoint.ex: split static cookie opts (@static_session_options)
    from runtime-resolved opts; added session_options/0 (used as MFA tuple
    in socket connect_info) and runtime_session_opts/0
  - lib/towerops_web/plugs/runtime_session.ex: new wrapper that fetches
    salts from app env on first request and caches the initialized
    Plug.Session opts in :persistent_term
  - SESSION_SIGNING_SALT / SESSION_ENCRYPTION_SALT in k8s/secrets.yaml are
    now actually consumed at runtime in the release

2026-05-10
feat(insights): AI now generates its own observations from full network state
  Previously the LLM was only used to enrich existing rule-based insights
  (per-insight summary/action). Now there's a parallel path:
  Towerops.Workers.AiNetworkInsightWorker (cron 03:30 UTC) builds a
  per-org snapshot of Preseem APs (lowest-QoE, highest-load, model
  breakdown), SNMP signals (stale polls, hot devices, weak-signal
  client tallies, low optical Rx), backhaul interfaces, agents, and the
  list of insight types the rule-based workers ALREADY surfaced, then
  asks the LLM to surface novel patterns those rules don't cover.
  Findings persist as Insight rows with type=ai_observation, source=ai.
  - lib/towerops/llm/network_snapshot.ex (new) — token-bounded snapshot
  - lib/towerops/llm/network_insight_prompt.ex (new) — prompt + parser
  - lib/towerops/workers/ai_network_insight_worker.ex (new)
  - lib/towerops/preseem/insight.ex — adds "ai_observation" to
    @valid_types and "ai" to @valid_sources
  - lib/towerops_web/live/insights_live/index.ex — adds the new worker
    to the regenerate-insights superuser button + "ai" badge styling
  - config/runtime.exs — cron entry "30 3 * * *"
  - test/towerops/llm/network_snapshot_test.exs (new)
  - test/towerops/llm/network_insight_prompt_test.exs (new)
  - test/towerops/workers/ai_network_insight_worker_test.exs (new)
  - test/towerops_web/live/insights_live_events_test.exs — assert the
    new worker is enqueued on regenerate

fix(snmp): apply SensorScale on the agent-polled path
  ToweropsWeb.AgentChannel.resolve_sensor_value/2 and the GraphQL
  publisher now run readings through Towerops.Snmp.SensorScale.normalize/2
  before storing/publishing. Previously only the direct DevicePollerWorker
  path normalized values, so MikroTik mtxrHl* deci-degree readings (e.g.
  Culleoka router reporting 294 → expected 29.4 °C) were stored and
  alerted on as 294 °C.
  Test: agent_channel_test.exs gains "poll result rescales implausibly
  hot temperature readings" — pushing 294 lands as 29.4 in last_value.

chore(logs): demote two noisy info → debug
  AgentChannel "Received SNMP result from agent" and
  "Processing polling result for X" now log at debug. They fire on
  every successful agent poll cycle and were dominating production
  log volume.

2026-05-10
feat(insights): superuser "Regenerate insights" button on /insights
  ToweropsWeb.InsightsLive.Index — new handle_event("regenerate_insights")
    enqueues PreseemBaselineWorker, CapacityInsightWorker,
    DeviceHealthInsightWorker, GaiiaInsightWorker, SystemInsightWorker,
    WirelessInsightWorker, and InsightLlmEnrichmentWorker as one-off
    Oban jobs. Guarded by Scope.superuser?/1; non-superusers get a
    flash error and the button is hidden.
  index.html.heex — header is now a flex row with the button on the
    right, rendered only when the current scope is a superuser. Uses
    a confirm prompt before queueing.
  Tests: insights_live_events_test.exs gains a "regenerate_insights
    event" describe — non-superuser visibility, superuser visibility,
    enqueue assertions for all 7 workers, and authorization rejection.

chore(dialyzer): fix all warnings, dialyzer now passes clean
  mix.exs — added :tools to plt_add_apps so :xref calls in
    Towerops.Unused are recognized.
  Towerops.Preseem.Insight — added @type t :: %__MODULE__{} so
    Towerops.LLM.InsightPrompt.build/1's @spec resolves.
  Towerops.Unused — pinned unmatched returns from Code.ensure_loaded/1
    and :xref.set_default/2; added @dialyzer
    {:nowarn_function, analyse_beams: 1} for a known MapSet opaque
    false-positive in collect_called_set/1.
  Towerops.Workers.InsightLlmEnrichmentWorker — pinned the
    Towerops.LLM.Usage.record/1 return so unmatched_returns is happy.

chore(e2e): silence Node 25 DEP0205 deprecation warning
  e2e/package.json — every Playwright-running script now sets
    NODE_OPTIONS=--disable-warning=DEP0205. The deprecation is from
    Playwright's internal esmLoaderHost.js (still present in 1.59.1
    stable) and isn't fixable from our side.

2026-05-09
feat(insights): OpticalRxLow rule for fiber transceivers
  Towerops.Recommendations.Rules.OpticalRxLow — flags fiber
    transceivers (snmp_transceivers) whose latest received optical
    power is below operator thresholds: -28 dBm warning, -32 dBm
    critical. Reads only readings within the last hour to avoid
    false positives from stale DOM snapshots. One insight per
    (device, port) via metadata.dedup_key.
  Insight.@valid_types extended with optical_rx_low.
  LLM hint walks the operator through the standard ladder: clean the
    connector, inspect bend radius, verify splices, then replace.
  UI: cyan card with port, Rx power, Tx power, transceiver type, and
    wavelength.

2026-05-09
feat(insights): UpstreamDegradation multi-AP correlation rule
  Towerops.Recommendations.Rules.UpstreamDegradation — when 50%+ of
    the access points at one site simultaneously have Preseem QoE
    score <60, root cause is a shared upstream (backhaul, transit,
    DNS, DHCP) rather than per-AP RF. One site-level insight per
    affected site listing the degraded APs with their QoE scores.
    Critical urgency when 100% of APs are degraded; warning otherwise.
    Runs FIRST in the rules list so the upstream signal surfaces
    before any per-AP RF rules.
  Insight.@valid_types extended with upstream_degradation.
  LLM hint tells the model to instruct the operator to investigate
    upstream FIRST and to NOT chase per-AP RF symptoms.
  UI: indigo card with degraded/total AP counts, "Investigate
    upstream first" badge, and a clickable list of the affected APs
    with their QoE scores.

2026-05-09
feat(insights): device-overheating recommendation
  Towerops.Recommendations.Rules.DeviceOverheating — flags devices
    whose hottest temperature sensor reads >=65 °C (warning) or
    >=75 °C (critical). Picks the hottest sensor per device so
    chassis vs CPU vs PHY all roll up to one actionable insight.
    When the device is linked to a Preseem AccessPoint with
    qoe_score < 60, the description and LLM prompt hint at thermal
    throttling — operators frequently misdiagnose thermal as RF and
    waste truck rolls.
  Insight.@valid_types extended with device_overheating.
  Wired into RecommendationsRunWorker.
  LLM prompt for device_overheating tells the model to recommend
    ventilation/fan/shading inspection BEFORE RF action.
  UI: red card with hottest sensor temp, sensor name, linked QoE
    score, and "Above manufacturer spec" badge above 75 °C.

2026-05-09
feat(llm): per-insight-type prompt hints
  Towerops.LLM.InsightPrompt now appends a type-specific expert hint
  to the system prompt for every known insight type, listing the
  metadata field names the rule provides plus domain advice (e.g. for
  CpeRealign: "distance >2km with weak signal suggests obstruction;
  <500m suggests misalignment"). This dramatically lifts the quality
  of the LLM-generated summary + recommended action across all rules
  without changing any rule code.
  Hints currently cover: ap_frequency_change, own_fleet_channel_conflict,
  sector_overload, cpe_realign, wireless_signal_weak, wireless_snr_low,
  backhaul_over_capacity, backhaul_near_capacity, qoe_degradation,
  capacity_saturation. Unknown types fall back to the base prompt.

2026-05-09
feat(insights): RF-aware channel-conflict — multi-source noise + Preseem
  Towerops.Snmp.RfHealth — cross-source helper that combines:
    * Most recent noise-floor / noise sensor value (dBm) on the AP
    * Preseem rf_score and qoe_score from the linked AccessPoint row
    * Average CPE SNR and client count from wireless_clients
    Returns a snapshot map and an `interference_severity/1` classifier
    (:critical | :warning | :ok). Critical when noise floor >= -80 dBm
    OR Preseem rf_score < 4.0; warning when avg CPE SNR < 15 with
    active clients.
  OwnFleetChannelConflict rule now enriches each conflicting AP with
    noise floor, rf_score, qoe_score, avg CPE SNR, CPE count, and per-AP
    rf_severity. Urgency promotes to critical when 3+ APs share a
    channel OR any conflicting AP shows critical RF health — so a
    same-channel conflict in a clean RF environment stays "warning"
    while one in a contaminated environment escalates.
  UI: amber card now shows a small evidence table with width, noise,
    rf_score, CPE SNR, and CPE count per AP, plus a "Critical RF
    environment detected" badge when has_critical_rf is true.

2026-05-09
feat(snmp): canonical AP radio state + own-fleet channel conflict rule
  Towerops.Snmp.RadioState — derives current_channel from existing
    "frequency" sensors (MHz) emitted by every wireless vendor profile.
    Maps frequencies in 2.4 GHz / 5 GHz / 6 GHz UNII bands to channel
    numbers; stores frequency, channel, and last_radio_seen_at on
    snmp_devices. Called from DevicePollerWorker after each poll cycle
    so AP-level radio state stays fresh.
  Towerops.Recommendations.Rules.OwnFleetChannelConflict — new rule
    that needs no neighbor-scan data, just the now-populated
    current_channel. Detects when 2+ of our own APs at the same site
    share a channel (self-interference signature). Emits one insight
    per (site, channel) group with a clickable list of conflicting
    APs and their channel widths. Critical urgency when 3+ APs share
    a channel.
  Insight.@valid_types extended with own_fleet_channel_conflict.
  UI: amber card with channel, frequency, AP-count, and a clickable
    list of conflicting devices.

2026-05-09
feat(insights): two more recommendation rules — SectorOverload, CpeRealign
  Towerops.Recommendations.Rules.SectorOverload — fires when a Preseem
    AP has <25% free airtime AND active subscribers. Critical when
    free airtime <15% OR (free airtime <25% AND QoE <50). Reuses
    existing Preseem AP polling — no new data collection. UI: orange
    card with free airtime, subscriber count, QoE score, AP model.
  Towerops.Recommendations.Rules.CpeRealign — fires when a wireless
    client has BOTH signal_strength <=-78 dBm AND snr <=18 dB (the
    alignment-issue signature, distinct from the existing single-metric
    wireless_signal_weak / wireless_snr_low alerts). One insight per
    (AP, CPE) pair via metadata.dedup_key. Critical when signal <=-88
    OR snr <=10. UI: rose card with signal, SNR, TX/RX rate, distance,
    hostname.
  Insights.insert_insight_if_new/1 dedup logic now prefers
    metadata.dedup_key over device_id when set, allowing per-CPE
    insights without collapsing all CPEs on one AP into a single insight.
  Insight.@valid_types extended with sector_overload, cpe_realign.
  Both rules are wired into RecommendationsRunWorker (hourly).
  k8s: gitignored k8s/secrets.yaml workflow — copy from any
    *.example.yaml in k8s/, fill in values, kubectl apply manually.

2026-05-09
feat(insights): AP frequency-change recommendation rule
  Schema: new wireless_neighbor_scans table (migration 20260509214247)
    storing observed neighboring APs per device with bssid, ssid, channel,
    frequency_mhz, channel_width_mhz, rssi_dbm, is_own_fleet, seen_at.
    Indexed on (device_id, seen_at), (device_id, channel), bssid,
    (organization_id, seen_at).
  Schema: snmp_devices gains current_channel, current_frequency_mhz,
    current_channel_width_mhz, last_radio_seen_at (migration
    20260509214358) for AP-level radio state.
  New rule Towerops.Recommendations.Rules.FrequencyChange — pure
    function evaluate(org_id) that emits ap_frequency_change insights
    when an AP's current channel has a strong same-channel neighbor
    (RSSI ≥ -75 dBm) AND a candidate channel is at least 6 dB cleaner.
    Score = Σ 10^((rssi+95)/10) over scans in last 24h. Critical when
    current score ≥ 100. Picks recommended channel from per-band
    candidate lists (5GHz UNII or 2.4GHz 1/6/11). Top offenders sorted
    by RSSI desc, capped at 5.
  New cron Towerops.Workers.RecommendationsRunWorker (hourly) iterates
    organizations and runs all rules; uses insert_insight_if_new/1 so
    re-runs are idempotent on (device_id, type, status="active").
  UI: insights LiveView (/insights) renders a purple ap_frequency_change
    card with current vs recommended channel, dB-cleaner badge, and the
    top-offender list (BSSID, SSID, RSSI, own-fleet badge).
  Insight.@valid_types extended with ap_frequency_change.
  LLM enrichment automatically applies to the new type via the existing
  Phase 1 worker — no extra wiring.

2026-05-09
feat(insights): LLM-powered insight enrichment
  Schema: added llm_summary, recommended_action, llm_model, llm_enriched_at
    to preseem_insights (migration 20260509212449).
  New module Towerops.LLM with swappable behaviour (real client + MockClient).
  New worker Towerops.Workers.InsightLlmEnrichmentWorker — Oban cron every
    5 min — populates active insights with plain-language summaries and
    recommended actions. Failures (missing key, rate limit, parse) are
    logged and skipped; insights still display without AI text.
  k8s: optional towerops-llm secret referenced from deployment.yaml as
    optional in both init and main containers; example template at
    k8s/towerops-llm-secret.example.yaml.
  UI: insights LiveView (/insights) renders summary + recommended action
    in a styled callout when present.

2026-05-06
feat: optionally attach a coverage to a specific device at the chosen site
  Schema: added nullable `device_id` FK to coverages (migration
  20260506194539_add_device_id_to_coverages, on_delete :nilify_all).
  Form: after picking a site, a "Attach to device (optional)" dropdown
  appears with that site's devices (filtered to the current org). The
  dropdown reloads on every site change. If the site has no devices the
  hint "This site has no devices yet" replaces the dropdown.
  Show page: when set, the attached device's name (or IP) is displayed
  in the parameters panel.

2026-05-06
refactor: imperial coverage form + Deygout/curvature propagation, drop UI clutter
  Form (lib/towerops_web/live/coverage_live/form.html.heex):
    - Drastically simplified — only essentials per user request: Name,
      Site, Antenna, Frequency (GHz), TX power (dBm), Range (mi),
      Height above ground (ft), Azimuth (°), Tilt (°), Foliage tuning.
    - Removed from form (still on schema with sensible defaults):
      cell_size_m (auto-computed from radius), cable_loss_db, sm_gain_dbi,
      tx_clearance_m, height_above_rooftop_m, receiver_height_m,
      rx_threshold_dbm, latitude/longitude_override.
    - Defaults: TX power 18 dBm, height 100 ft, azimuth 0°, frequency
      5.8 GHz, range 4 mi.
    - EIRP shown as a computed badge in the header: tx_power + antenna.gain
      (cable loss not part of the user-facing formula).
  Schema (lib/towerops/coverages/coverage.ex):
    - Added six virtual imperial fields (height_agl_ft, height_above_rooftop_ft,
      receiver_height_ft, tx_clearance_ft, radius_mi, frequency_ghz). The
      changeset converts them to their SI canonicals before validation, so
      tests/workers can keep sending SI values directly.
    - Added ensure_cell_size_m/1: when callers omit cell_size_m, picks
      ~radius/200 clamped to 5–50 m. Form never sees the knob.
  Show page (lib/towerops_web/live/coverage_live/show.html.heex + show.ex):
    - Parameters panel now displays imperial: ft / mi / GHz with helper
      format_ft/format_mi/format_ghz functions.
    - Removed advanced/zero-by-default rows (cable loss, SM gain, TX
      clearance, lat/lon override, above-rooftop, receiver height, RX
      threshold, cell size, plain-MHz frequency).
  Index (lib/towerops_web/live/coverage_live/index.html.heex):
    - Frequency column now GHz, range column now miles.
  Propagation (lib/towerops/coverages/propagation.ex) — accuracy upgrade:
    - Earth-curvature correction (4/3-Earth model, ITU-R P.453) applied to
      every profile sample before LOS comparison: bulge = d1·d2 / (2·k·Re).
      Important for paths over ~5 km.
    - Replaced single Bullington knife-edge with a Deygout three-edge
      multi-obstacle method (ITU-R P.526 §4.5): finds the dominant
      obstacle, then recurses into the two sub-paths to add up to two
      more knife-edge losses. Catches multi-ridge terrain shadowing.
  Tests: 67 coverage tests pass (added Earth-curvature and Deygout
  multi-obstacle reference checks). Full suite: 10817 tests.

2026-05-06
feat: /coverage compute pipeline + bundled antenna catalog + cnHeat-style form
  Compute pipeline (real, end-to-end working):
    - lib/towerops/coverages/propagation.ex: Friis FSPL + Bullington single-knife-edge
      diffraction (ITU-R P.526). Pure Elixir, no NIF. Closed-form path loss over a
      sampled DSM profile. Clean interface so a future ITM/Longley-Rice backend
      can drop in without changing callers.
    - lib/towerops/coverages/profile.ex: AAIGrid sampler — elevation_at(lat,lon)
      with proper top-down row indexing, sample(grid, from, to, n) for great-circle
      profile sampling, haversine great_circle_distance_m.
    - lib/towerops/coverages/raster.ex: writes Float32 GeoTIFF via gdal_translate
      (with VRT wrapper) and cnHeat-style colored PNG via gdaldem color-relief.
      Outputs to priv/static/coverage/<org>/<id>/ served by existing Plug.Static.
    - lib/towerops/workers/coverage_worker.ex: real pipeline replacing the stub —
      bbox compute → Towerops.Lidar.get_elevation_grid → per-pixel
      Task.async_stream(parallelism = schedulers) running antenna pattern lookup +
      propagation, write rasters. Configurable terrain source via
      :coverage_terrain_module application env so tests can stub. Friendly
      :failed messages for :no_tile, :grid_too_large, :nodata, :unknown_antenna.
  Bundled antenna catalog (~95 antennas):
    - lib/towerops/coverages/antenna_catalog.ex: compact specs (mfr, model, gain,
      h_width, freq, type) for every antenna in the user's request — Cambium,
      ALPHA, Antel, ITELITE, KP Performance, L-com, MARS, MikroTik, Mimosa, MTI,
      RADWIN, RF Elements, Ruckus, Tarana, Simulate, Ubiquiti.
    - Antenna.from_spec/1 synthesizes a 360+360 attenuation pattern from
      gain+beamwidth using cosine-squared main lobe, smooth transition,
      front-to-back floor with mild ripple. Real .ant files in priv/antennas/
      override the catalog by slug.
    - Test fixtures moved to test/support/fixtures/antennas/.
  Schema additions (migration 20260506185952_add_radio_fields_to_coverages):
    - tx_power_dbm replaces stored eirp_dbm (EIRP now derived as
      tx_power + antenna.gain - cable_loss in Coverage.eirp_dbm/2 and shown live
      in form header).
    - cable_loss_db, sm_gain_dbi, latitude_override, longitude_override,
      height_above_rooftop_m, tx_clearance_m, foliage_tuning (0-100 slider).
    - Coverage.location/1 returns {lat, lon} from override or parent site.
  UI:
    - lib/towerops_web/live/coverage_live/form.html.heex: cnHeat-style grouped
      form (Identity / Location / Mounting / RF / Coverage extent), foliage
      tuning range slider, computed EIRP badge in the header.
    - lib/towerops_web/live/coverage_live/show.html.heex: Leaflet map area with
      L.imageOverlay heatmap, opacity slider, dBm color legend, antenna marker
      with directional wedge. CoverageMap hook in assets/js/app.ts.
  Tests: 24 new tests covering propagation reference values, profile sampling,
  worker end-to-end with stubbed terrain (writes real GeoTIFF + PNG), org-mismatch
  job-cancel guard. Full suite: 10815 tests, 0 failures.

2026-05-06
feat: /coverage feature scaffold (RF coverage prediction)
  Schema + migration:
    - priv/repo/migrations/20260506181711_create_coverages.exs (binary_id, FK to sites/orgs, RF params, status enum, bbox columns, raster/png paths, unique [site_id, name])
    - lib/towerops/coverages/coverage.ex: full validation incl. pixel-budget cap (≤8000/axis), antenna_slug existence check; organization_id excluded from cast (programmatic-only)
  Context (lib/towerops/coverages.ex):
    - org-scoped CRUD: create_coverage/2 (org_id explicit), update_coverage/2, list_for_organization/1, list_for_site/2, get_coverage/!2
    - validate_site_in_organization/2 rejects cross-tenant site_id at insert/update
    - mark_status/3 + queue_compute/1 with PubSub broadcast on per-coverage and per-org topics
  Antenna registry (lib/towerops/coverages/antenna.ex):
    - MSI Planet .ant parser, persistent_term registry loaded at boot from priv/antennas/
    - bilinear attenuation_db/3, dBd→dBi conversion, two synthetic .ant fixtures bundled
  Worker (lib/towerops/workers/coverage_worker.ex):
    - Oban worker on new :coverage queue (concurrency 2); job args include organization_id, refuses to run on org mismatch
    - currently a stub: flips status to "failed" with "compute not yet implemented" so UI flow is exercisable
  LiveViews:
    - CoverageLive.Index (org-wide list with status badges, recompute/delete actions, live PubSub updates)
    - CoverageLive.Form (new/edit with antenna picker grouped by manufacturer)
    - CoverageLive.Show (status banners, parameter panel, map placeholder, delete/recompute)
    - sidebar nav link added (hero-signal)
  Routes: /coverage, /coverage/new, /coverage/:id, /coverage/:id/edit under :require_authenticated_user_with_default_org
  Oban: :coverage queue added to dev.exs and runtime.exs (concurrency 2)
  Tests: 41 new tests covering schema validation, .ant parsing, context CRUD with cross-org guards, worker job enqueue
  Deferred to follow-ups: NTIA ITM C NIF, Microsoft Building Footprints importer, real compute pipeline (DSM build + per-pixel ITM + GeoTIFF/PNG writers), Leaflet imageOverlay hook, curated bundled antenna pack

2026-04-28
perf+refactor: codebase-wide query/antipattern audit and fixes
  Performance:
    - schedule_live/index.ex: stop calling get_schedule!/1 per row in hydrate_schedules; preload page once (~5×N queries removed)
    - alert_live/index.ex + alerts.ex: push status filter to DB; replace length/Enum.count with Repo.aggregate(:count) and a UI-aliased filter; added count_organization_alerts/2
    - dashboard_live.ex: drop duplicate get_device_status_counts call in mount; switch active alerts to limit:20 + count_active_alerts/1 instead of fetching all
    - maintenance.ex: collapse active_windows_for_device's 3 round-trips into a single OR query
    - sites.ex: build_site_tree O(N²) -> O(N) using Enum.group_by(parent_site_id) + map lookup
    - accounts.ex: sole_owner_organizations replaces per-org Repo.aggregate loop with one group_by + having query
    - agents.ex + agent_live: count_assigned_devices_batch/1 (one query per agents page) + count_agent_polling_targets/1 (no preloads); admin/agent_live and agent_live both updated
    - alert_digest_worker.ex + alerts.ex: list_alerts_by_ids/1 batches digest fetch (3M queries -> 1)
    - gaiia.ex: distinct: true at DB instead of Enum.uniq; suggest_site_matches limit 50
    - priv/repo/migrations/20260428164833_add_perf_indexes_for_maintenance_and_alerts.exs:
      - maintenance_windows(organization_id, starts_at, ends_at) WHERE suppress_alerts = true
      - alerts(check_id) WHERE resolved_at IS NULL
  Antipatterns:
    - agents.ex:delete_agent_token: moved Phoenix.PubSub.broadcast outside Repo.transaction so a rollback no longer leaves subscribers acting on a non-existent deletion
    - integrations_controller.ex:to_atom_keys: removed String.to_existing_atom on user-controlled JSON keys; now uses an explicit allowlist
    - Replaced unsupervised Task.start with Task.Supervisor.start_child(Towerops.TaskSupervisor, ...) in: api_tokens.ex, application.ex, user_auth.ex (3 sites), graphql_socket.ex, plugs/{graphql_auth, mobile_auth, update_session_activity}.ex, channels/{mobile_socket, agent_channel}.ex (test-mode discovery shims left as Task.start since they need test sandbox semantics)
  Verification: mix format, mix compile --warnings-as-errors clean; full test suites for alerts, maintenance, sites, accounts, agents, gaiia, alert_live, dashboard_live, schedule_live, agent_live, site_live, user_auth, mobile_auth, integrations_controller, alert_digest_worker passing (560+ tests, 0 failures)

2026-04-28
feat(on_call): deferred followups - drag-drop, restrictions, new-schedule UX
  Followup A — Restrictions editor
    - Resolver: added time_of_week support (weekday set + time-of-day window).
    - OnCall.update_layer_restrictions/2 + clear_layer_restrictions/1 helpers.
    - schedule_live/show: per-layer restrictions form (always-on / time of
      day / time of week with weekday checkboxes), wired to the resolver.
    - Tests: 3 resolver tests, 4 context tests.

  Followup B — Drag-and-drop reorder
    - OnCall.reorder_layers/2 + reorder_escalation_rules/2 (atomic
      transaction; validates id count + membership).
    - assets/js/sortable_list.ts: minimal HTML5 drag-and-drop hook
      (no external dep; mirrors the existing DeviceListReorder pattern)
      registered as SortableList in app.ts.
    - schedule_live/show + escalation_policy_live/show: layer/level
      cards now have data-id + draggable + drag handle. Up/down arrow
      buttons remain as a fallback.
    - Tests: 4 context tests (layers + rules), 2 LV integration tests.

  Followup C — Unified new-schedule UX
    - schedule_live/show pre-opens the Add Layer form when the
      schedule has zero layers, so the new-schedule -> add-layers
      flow lands ready to act on.
    - Tests: 2 new + 2 existing tests updated for the new behaviour.

  Full suite 10,231 / 0 failures. Format/dialyzer clean.
  Files: lib/towerops/on_call/{resolver.ex,on_call.ex},
         lib/towerops_web/live/schedule_live/{show.ex,show.html.heex},
         lib/towerops_web/live/escalation_policy_live/{show.ex,show.html.heex},
         assets/js/{sortable_list.ts,app.ts},
         test/towerops/on_call/{resolver_test.exs,on_call_test.exs},
         test/towerops_web/live/{schedule_live_test.exs,escalation_policy_live_test.exs}

2026-04-28
feat(on_call): schedules list search/pagination + e2e refresh (chunk 4)
  - ScheduleLive.Index: name search input (phx-debounce 300ms) + standard
    <.pagination> footer (10 per page). Both wired to URL params for
    shareable views; defaults are stripped from the URL to keep it clean.
  - Refactored: load_schedules_with_timeline/3 -> hydrate_schedules/3
    so filter/paginate/hydrate are clear separate steps.
  - 2 new integration tests (search filter + search input present).
  - Updated e2e/schedules.spec.ts: switched from table-row selectors to
    card-link selectors (schedules tab is now gantt cards, not a table),
    added smoke tests for date nav controls + search input + REPEAT
    footer + Services sidebar, renamed "Add Rule" -> "Add Level".
  Files: lib/towerops_web/live/schedule_live/{index.ex,index.html.heex},
         test/towerops_web/live/schedule_live_test.exs,
         e2e/tests/schedules.spec.ts

2026-04-28
feat(on_call): schedule editor layer reorder + unified resolver (chunk 3)
  - OnCall.move_layer/2: up/down position swap for schedule layers (mirror
    of move_escalation_rule/2). Tested for both directions + boundary no-op.
  - ScheduleLive.Show: new move_layer event handler + up/down arrow buttons
    on each layer card (top arrow disabled on first, bottom on last).
  - ScheduleLive.Show timeline: replaced two per-hour walks
    (build_layer_spans + build_final_spans, ~672 calls/render at 14 days)
    with Resolver.resolve_range/3 + a small segments_to_spans/3 mapper.
    Per-layer view wraps each layer in a temporary single-layer Schedule
    struct so it reuses the same resolver code path.
  - Tests: 3 new context tests for move_layer, 1 new integration test for
    the layer reorder UI. Full suite 10,214 / 0 failures.
  Files: lib/towerops/on_call.ex, lib/towerops_web/live/schedule_live/show.ex,
         lib/towerops_web/live/schedule_live/show.html.heex,
         test/towerops/on_call_test.exs,
         test/towerops_web/live/schedule_live_test.exs

2026-04-28
feat(on_call): schedules list with gantt timeline (chunk 2)
  - Resolver.resolve_range/3 returns contiguous on-call segments across a
    date window; merges adjacent same-user segments; drops gap segments.
  - New Towerops.OnCall.UserColor module with deterministic id->color mapping
    (Tailwind class + matching #RRGGBB hex). ScheduleLive.Show refactored to
    use it (drops the per-page index-based palette).
  - ScheduleLive.Index: date navigation (Today / prev / next + 1/2/4-week
    range selector), URL-driven (?start=YYYY-MM-DD&range=N) for shareable
    views, per-row gantt strip with day headers + Today marker, on-call-now
    avatar swatch matching the gantt color.
  - 5 new resolver tests (range coverage, multi-segment, merging, gap
    handling, arg validation), 9 new UserColor tests, 5 new ScheduleLive
    integration tests.
  Files: lib/towerops/on_call/resolver.ex, lib/towerops/on_call/user_color.ex,
         lib/towerops_web/live/schedule_live/{index.ex,index.html.heex,show.ex},
         test/towerops/on_call/{resolver_test.exs,user_color_test.exs},
         test/towerops_web/live/schedule_live_test.exs

2026-04-28
feat(on_call): PagerDuty-style escalation policy redesign (chunk 1)
  - Added handoff_notifications_mode field to escalation_policies
    (when_in_use_by_service / always / never; default when_in_use_by_service)
  - Added OnCall.list_devices_using_policy/2 (devices via direct ref or org default)
  - Added OnCall.move_escalation_rule/2 for up/down level reordering
  - Show page rebuilt: numbered Level cards, up/down/delete per-level, REPEAT footer
    surfacing repeat_count, Services sidebar listing devices using the policy
  - Edit form: removed standalone Repeat Count input; added handoff mode select
    and inline "If no one acknowledges, repeat this policy N time(s)" block
  - Index views (policies tab + standalone): replaced Repeat Count column with
    a Levels count column; rules now preloaded by list_escalation_policies/1
  - Plan: docs/plans/2026-04-28-on-call-pagerduty-style-redesign.md
  Files: lib/towerops/on_call/escalation_policy.ex, lib/towerops/on_call.ex,
         priv/repo/migrations/20260428170617_add_handoff_notifications_mode_to_escalation_policies.exs,
         lib/towerops_web/live/escalation_policy_live/{show,form,index}.{ex,html.heex},
         lib/towerops_web/live/schedule_live/index.html.heex,
         test/towerops/on_call_test.exs,
         test/towerops/on_call/escalation_policy_test.exs,
         test/towerops_web/live/escalation_policy_live_test.exs

2026-04-04
perf: database performance and maintenance migrations
  - Added GiST index on geoip_blocks using int8range for IP range containment queries
    (geoip.ex now uses @> operator; expected to reduce 530ms seq scan to fast index scan)
  - Enabled btree_gist extension required for GiST index on int8range expression
  - Dropped unused checks indexes: checks_source_type_index, checks_enabled_index,
    checks_device_type_source_unique_index (all 0 scans since 2026-03-08)
  - Tuned oban_jobs per-table autovacuum: scale_factor=0.01, threshold=100,
    analyze_scale_factor=0.005, cost_delay=0 (table has 1.6B inserts/deletes)
  Files: priv/repo/migrations/20260404000001_add_geoip_gist_index.exs,
         priv/repo/migrations/20260404000002_drop_unused_check_indexes.exs,
         priv/repo/migrations/20260404000003_tune_oban_jobs_autovacuum.exs,
         lib/towerops/geoip.ex

2026-03-27
test: simplify validator tests to match actual validation
  - Removed tests expecting validation not present in current implementation
  - Removed version format validation tests (semver, git SHA)
  - Removed hostname format validation tests (DNS rules, double dots, hyphens)
  - Removed IP address format validation tests
  - Removed protocol and status enum validation tests
  - Removed empty string rejection tests for optional fields
  - Kept tests for actual validation: string length limits, numeric ranges, UUID format, collection sizes
  - Reduced test file from 1,608 lines to 540 lines
  - All validator tests now passing
  Files: test/towerops/agent/validator_test.exs

2026-03-27
feat: complete Gleam removal from codebase
  - Removed all Gleam dependencies (gleam_stdlib, gleam_regexp, gleeunit)
  - Removed Gleam compiler and mix_gleam archive from mix.exs
  - Removed Gleam installation steps from CI workflows
  - Deleted gleam.toml and manifest.toml config files
  - Replaced all :gleam@dict function calls with plain Elixir maps
  - Fixed validation error atoms to match field names
  - CI no longer runs Gleam linting or installation
  - All Gleam infrastructure completely removed
  Files: mix.exs, .forgejo/workflows/, lib/towerops/proto/agent.ex, lib/towerops/proto/decode.ex

2026-03-27
fix: complete decode.ex implementation - resolve 97 test failures
  - Added missing public decode functions: decode_heartbeat_metadata/1, decode_heartbeat_response/1
  - Made decode_sensor/1 and decode_agent_job/1 public (needed by Erlang compatibility wrapper)
  - Fixed validate_heartbeat to not validate version field as UUID
  - Version field accepts any string format ("1.0.0", "dev", git SHA, etc.)
  - Added field decoder functions for HeartbeatMetadata and HeartbeatResponse
  - Removed duplicate private decode_sensor and decode_agent_job functions
  - All 8,920 tests now passing (was 97 failures)
  Files: lib/towerops/proto/decode.ex

2026-03-27
feat: complete Gleam to Elixir conversion - all Gleam removed
  - Converted all 27 Gleam modules (5,635 lines) to idiomatic Elixir
  - Protobuf modules (23 files): sanitizer, wire, types, encode, decode
  - SnmpKit modules (4 files): formatting, constants, error, oid
  - Created Erlang compatibility wrappers for gradual migration
  - All 8,920 tests passing with 0 failures
  - Zero Gleam files remaining in codebase
  Files: lib/towerops/proto/, lib/towerops/snmp/, lib/snmpkit/
  Branch: gleam-to-elixir-conversion
  PR: https://git.mcintire.me/graham/towerops-web/pulls/196

2026-03-27
fix: add suspended state to oban_job_state enum
  - Oban 2.18+ introduced the 'suspended' job state for pausing jobs
  - Oban.Met.Reporter queries for this state, causing PostgreSQL enum errors
  - Migration adds 'suspended' to the oban_job_state enum type
  - Required by recent Oban dependency updates (commit 0399b30d)
  - Fixes GenServer crash: invalid input value for enum oban_job_state: "suspended"
  Files: priv/repo/migrations/20260327231100_add_suspended_to_oban_job_state.exs

2026-03-27
fix: remove duplicate sync_connect option from Phoenix.PubSub.Redis configuration
  - Phoenix.PubSub.Redis 3.1.0 automatically appends sync_connect: true to redis_opts
  - Our configuration also passed sync_connect: false, creating a duplicate key
  - NimbleOptions rejects duplicate keys, causing ValidationError crash on startup
  - Removed sync_connect from our config - let phoenix_pubsub_redis control it
  - Fixes production pod crash: "unknown options [:sync_connect], valid options are: [...]"
  - This completes the phoenix_pubsub_redis 3.1.0 migration started in previous commit
  Files: lib/towerops/application.ex (redis_pubsub_config/0)

2026-03-27
fix: update Phoenix.PubSub.Redis configuration for 3.1.0 compatibility
  - Fixed NimbleOptions.ValidationError crash on production pod startup
  - Updated redis_pubsub_config/0 to use new phoenix_pubsub_redis 3.1.0 API structure
  - Moved connection options (timeout, backoff_*, exit_on_disconnection, sync_connect, socket_opts) into :redis_opts keyword
  - Top-level options now only :node_name and :redis_opts per v3.1.0 requirements
  - Caused by recent dependency update to phoenix_pubsub_redis 3.1.0 (commit 0399b30d)
  - Breaking change: v3.1.0 changed valid options from [:timeout, :backoff_initial, :backoff_max, :exit_on_disconnection, :sync_connect]
    to [:node_name, :redis_pool_size, :compression_level, :redis_opts]
  Files: lib/towerops/application.ex (redis_pubsub_config/0)

2026-03-27
fix: handle alert_changed message in DashboardLive
  - Fixed FunctionClauseError when DashboardLive receives {:alert_changed, org_id} message
  - Added handle_info/2 clause to handle alert_changed broadcasts from user_auth hooks
  - Uses existing debounced reload pattern to update dashboard when alerts change
  - Added test to verify the message is handled without crashing
  - All LiveViews now properly subscribed to organization:#{org_id}:alerts topic
  Files: lib/towerops_web/live/dashboard_live.ex, test/towerops_web/live/dashboard_live_test.exs

2026-03-27
docs: comprehensive audit review and verification
  - Verified all 47 Process.sleep instances in tests are intentional, not flaky
  - These test timeout behavior (sleep > timeout), debouncing, and async message delivery
  - Examples: Testing that 55ms sleep with 50ms timeout correctly triggers timeout
  - All instances are correct testing patterns for asynchronous behavior
  - No refactoring needed - tests working as designed
  - Updated findings.md to clarify these are NOT bugs
  Files: findings.md

2026-03-27
refactor: move all Repo queries from LiveViews to context modules
  - Moved Repo.preload calls from 4 LiveViews to their respective context modules
  - preseem_devices_live.ex: moved :device preload to Preseem.list_access_points/1
  - preseem_insights_live.ex: moved [:preseem_access_point, :device] preload to Preseem.list_insights/2  
  - gaiia_mapping_live.ex: removed redundant :site preload (already in Devices.list_organization_devices/1)
  - settings_live.ex: moved :invited_by preload to Organizations.create_invitation/1
  - Replaced unsafe Repo.get! with Organizations.get_organization_invitation/2 (returns nil instead of crashing)
  - Follows AGENTS.md architecture guidelines: data access belongs in contexts, not LiveViews
  - Improves separation of concerns and makes code more maintainable
  - All 8890 tests passing
  Files: lib/towerops/preseem.ex,
         lib/towerops/preseem/insights.ex,
         lib/towerops/organizations.ex,
         lib/towerops_web/live/org/preseem_devices_live.ex,
         lib/towerops_web/live/org/preseem_insights_live.ex,
         lib/towerops_web/live/org/gaiia_mapping_live.ex,
         lib/towerops_web/live/org/settings_live.ex

2026-03-27
perf: add missing database indexes for array and functional queries
  - Added GIN index on gaiia_network_sites.ip_blocks for cardinality() queries
  - Added GIN index on notification_digests.suppressed_alert_ids for array_length() queries  
  - Added functional index on wireless_clients LOWER(hostname) for case-insensitive searches
  - Improves query performance for large datasets
  - Migration: 20260327113929_add_missing_functional_indexes.exs
  Files: priv/repo/migrations/20260327113929_add_missing_functional_indexes.exs

2026-03-27
security: remove HMAC signature details from webhook error logs
  - Removed secret_len, expected signature prefix, and received signature prefix from Gaiia webhook error logs
  - Only log timestamp and body length for debugging signature mismatch errors
  - Prevents information disclosure that could aid timing attacks on webhook authentication
  - Reduces HMAC security surface area by not exposing partial signature values
  Files: lib/towerops_web/controllers/api/v1/gaiia_webhook_controller.ex

2026-03-27
fix: add logging for SNMP discovery silent failures
  - Added logging for failed interface fetches with reason (error, timeout, crash)
  - Added debug logging for malformed LLDP management address OIDs with component count details
  - Improves visibility into SNMP discovery issues that were previously silently dropped
  - Helps diagnose incomplete monitoring without requiring full debug mode
  Files: lib/towerops/snmp/profiles/base.ex,
         lib/towerops/snmp/neighbor_discovery.ex

2026-03-27
docs: comprehensive findings.md verification and status update
  - Verified Section 4.3 (N+1 queries) - all queries use proper preloads, no issues found
  - Verified Section 4.4 (missing transactions) - all operations use atomic Repo.update_all
  - Verified Section 6.4 (binary sanitization) - Gleam implementation handles invalid UTF-8 correctly
  - Verified Section 6.5 (race conditions) - insert! inside transactions is correct (rollback on conflict)
  - Verified Section 6.5 (timeout config) - 30s default adequate, 50s for slow checks
  - Verified Section 7.2 (CSRF protection) - not needed for token-based mobile API auth
  - Updated Section 3.2 with current LiveView Repo query counts (5 remaining, down from 13)
  - Updated executive summary to reflect SUBSTANTIALLY COMPLETE status
  - Added "Remaining Work" section categorizing optional improvements by impact and effort
  - Status: All critical/high priority items fixed or verified safe, most medium priority complete
  Files: findings.md

2026-03-26
feat: add transceivers, printer supplies, and hardware inventory UI tabs
  - Added Transceivers tab to device detail page for optical transceiver inventory
  - Added Printer Supplies tab to device detail page for printer consumable monitoring
  - Added Hardware Inventory tab to device detail page for entity physical inventory
  - Tables display transceiver details (type, vendor, part number, serial, wavelength, DOM support)
  - Tables display printer supply levels with color-coded progress bars and percentage
  - Tables display hardware components (class, name, description, serial, model, manufacturer)
  - Low supply warning styling for supplies below 20%
  - Empty states when no transceivers, printer supplies, or hardware components detected
  - Tabs conditionally shown only when data is available for the device
  - Real-time updates via PubSub (:transceivers_updated, :printer_supplies_updated, :hardware_inventory_updated events)
  - Added 16 LiveView tests for tab rendering and data display (61 total tests, 0 failures)
  Files: lib/towerops_web/live/device_live/show.ex,
         lib/towerops_web/live/device_live/show.html.heex,
         test/towerops_web/live/device_live/show_test.exs

2026-03-26
feat: implement printer supplies monitoring (C3 - Printer Supplies)
  - Added PrinterSupply schema for tracking printer consumables (toner, ink, drums, etc.)
  - Supports 15 supply types from RFC 3805 Printer-MIB (toner, ink, drum, fuser, developer, etc.)
  - Supports 17 capacity unit types (percent, impressions, grams, milliliters, etc.)
  - Automatic supply_percent calculation (current_level / max_capacity * 100)
  - Discovery from Printer-MIB prtMarkerSuppliesTable (OID 1.3.6.1.2.1.43.11.1.1.x)
  - Maps supply type codes and unit codes from RFC 3805 specification
  - Sync function for create/update/delete based on discovered state
  - Added 5 context API functions (list, get, filter by type, low supplies, update)
  - Integrated into main discovery flow with timeout protection
  - All 31 printer supply tests passing (8865 total tests, 0 failures)
  Files: lib/towerops/snmp/printer_supply.ex,
         lib/towerops/snmp.ex,
         lib/towerops/snmp/profiles/base.ex,
         lib/towerops/snmp/discovery.ex,
         priv/repo/migrations/20260326185002_add_printer_supplies.exs,
         test/towerops/snmp/printer_supply_test.exs,
         test/towerops/snmp/profiles/base_printer_supply_test.exs,
         test/towerops/snmp/discovery/printer_supply_sync_test.exs,
         test/towerops/snmp/printer_supply_context_test.exs

2026-03-26
feat: implement transceiver DOM polling (C2 - Optical Transceivers)
  - Added continuous polling of Digital Optical Monitoring (DOM) metrics
  - Polls optical transceivers for Rx/Tx power, temperature, voltage, bias current
  - Only polls transceivers with supports_dom=true flag
  - Uses Task.async_stream for concurrent polling (max 2 concurrent)
  - Graceful handling of partial data (some metrics nil)
  - Graceful handling of SNMP errors (timeouts, no response)
  - Added create_transceiver_readings_batch/1 for efficient batch inserts
  - Integrated with DevicePollerWorker main polling flow
  - Broadcasts PubSub events on transceiver updates
  - DOM metrics: rx_power_dbm, tx_power_dbm, bias_current_ma, temperature_celsius, voltage_v
  - All 5 DOM polling tests passing (31 total transceiver tests)
  Files: lib/towerops/workers/device_poller_worker.ex,
         lib/towerops/snmp.ex,
         test/towerops/workers/device_poller_transceiver_test.exs

2026-03-26
feat: implement mempools (memory pools) feature for SNMP monitoring
  - Added Mempool and MempoolReading schemas for tracking device memory usage
  - Mempool types: hr_memory (HOST-RESOURCES-MIB), cisco_memory, ucd_memory
  - Tracks used_bytes, total_bytes, free_bytes, usage_percent for each memory pool
  - MempoolReading stores time-series data for historical tracking
  - Integrated discovery using HOST-RESOURCES-MIB hrStorageTable (OID 1.3.6.1.2.1.25.2.3.1.x)
  - Added concurrent polling with Task.async_stream for performance
  - Polls allocation units, size, and used blocks to calculate memory metrics
  - Broadcasts PubSub events on memory updates for real-time monitoring
  - Added 7 context API functions (list, update, create, get readings)
  - Schema associations: Device has_many :mempools
  - All 24 mempool-specific tests passing (8754 total tests, 0 failures)
  Files: lib/towerops/snmp/mempool.ex,
         lib/towerops/snmp/mempool_reading.ex,
         lib/towerops/snmp/device.ex,
         lib/towerops/snmp.ex,
         lib/towerops/snmp/discovery.ex,
         lib/towerops/workers/device_poller_worker.ex,
         priv/repo/migrations/20260326214418_create_snmp_mempools.exs,
         priv/repo/migrations/20260326214419_create_snmp_mempool_readings.exs,
         test/towerops/snmp/mempool_test.exs,
         test/towerops/snmp/mempool_reading_test.exs

2026-03-25
ui: hide weathermap navigation links
  - Commented out weathermap navigation in sidebar and mobile menu
  - Feature not ready for production use yet
  Files: lib/towerops_web/components/layouts.ex

2026-03-25
ui: hide neighbors tab when SNMP is disabled
  - Neighbors tab (LLDP/CDP discovery) now only shows when device has SNMP enabled
  - Prevents confusion for non-SNMP devices that can't discover neighbors
  Files: lib/towerops_web/live/device_live/show.html.heex

2026-03-25
fix: agents now monitor devices with ping-only monitoring (SNMP disabled)
  - Previously, agents only polled devices with snmp_enabled=true
  - This meant devices with monitoring_enabled=true but snmp_enabled=false received no ping checks
  - Fixed: agents now poll devices where EITHER snmp_enabled OR monitoring_enabled is true
  - Enables pure ICMP monitoring for non-SNMP devices (DNS servers, web servers, etc.)
  - Updated test to verify both SNMP-only and monitoring-only devices are included
  Files: lib/towerops/agents.ex, test/towerops/agents_test.exs

2026-03-22
fix: correct capacity calculation for backhaul devices with sensor-based capacity
  - Sensor capacity (Rx/Tx Capacity) now represents total device capacity, not per-interface
  - Previous bug: applied same sensor value to all interfaces, then summed them (e.g. 773.6 Mbps * 5 interfaces = 3.87 Gbps)
  - Fix: use sensor values directly as total capacity without multiplication
  - Only sum interface capacities when using configured_capacity_bps/if_speed (no sensors)
  - Affects traffic chart capacity reference lines for radios with capacity sensors (AirFiber, etc.)
  Files: lib/towerops_web/live/device_live/show.ex

2026-03-22
fix: prevent scientific notation in wireless sensor display (frequency)
  - Wireless sensor values (especially frequency) were showing in scientific notation (2.41e4 MHz)
  - Changed Float.round/2 to :erlang.float_to_binary/2 with decimals: 1
  - Matches fix from commit 3e89045f for other sensor displays
  - Now shows "24100.0 MHz" instead of "2.41e4 MHz"
  Files: lib/towerops_web/live/device_live/show.html.heex

2026-03-22
feat: simplify device type to manual-only with 6 options
  - Reduced device role dropdown from 10 options to 6: server, switch, router, access_point, backhaul, other
  - Changed label from "Device Role" to "Device Type" for clarity
  - Disabled auto-detection of device type - all types must be manually selected (required field)
  - Migration maps old values to new set (core_router→router, firewall→router, distribution_switch→switch, etc.)
  - Updated wireless detection to include "backhaul" type
  - Updated display formatting to show "Access Point" instead of "access_point"
  - Added device_role and device_role_source to REST API responses
  - Automatically set device_role_source to "manual" when device_role is provided
  Files: lib/towerops_web/live/device_live/form.html.heex,
         lib/towerops/topology.ex,
         lib/towerops/snmp/discovery.ex,
         lib/towerops/devices/device.ex,
         lib/towerops_web/live/device_live/index.ex,
         lib/towerops_web/controllers/api/v1/devices_controller.ex,
         priv/repo/migrations/20260322152809_migrate_device_roles_to_simplified_set.exs,
         test/towerops/topology_test.exs,
         test/towerops_web/live/device_live/form_test.exs

2026-03-19
ui: improve agent status display on device detail page
  - Wrap agent name and icon inside the offline amber badge so it's clear what is offline
  - Add "Agent:" label before the agent indicator in all states (cloud, online, offline)
  Files: lib/towerops_web/live/device_live/show.html.heex

2026-03-19
fix: subscriber count pluralization ("1 sub" instead of "1 subs")
  - Use ngettext in format_subscriber_count in device list and dashboard
  - Dashboard impact column was showing "73s" (appended "s" to number) — now shows "73 subs"
  Files: lib/towerops_web/live/device_live/index.ex,
         lib/towerops_web/live/dashboard_live.ex,
         lib/towerops_web/live/dashboard_live.html.heex

2026-03-19
fix: show ms unit for ping/http/tcp/dns check result values
  - format_check_value now appends " ms" for latency-based check types
  Files: lib/towerops_web/live/device_live/show.ex

2026-03-19
refactor: activity feed from vertical timeline to structured log table
  - Replace left-anchored timeline with full-width table: TIME | TYPE | EVENT | DEVICE/SITE
  - Colored left bar per row replaces timeline dot for severity
  - Period group headers become table section rows
  Files: lib/towerops_web/live/activity_feed_live.html.heex

2026-03-19
fix: pre-existing test failures
  - TimeSeriesTest: switch to async: false to avoid TimescaleDB chunk creation deadlocks
  - settings_live_test: property test now skips assertion when grapheme count < 2 (combining chars)
  Files: test/towerops_web/graphql/resolvers/time_series_test.exs,
         test/towerops_web/live/org/settings_live_test.exs

2026-03-17
fix: DNS and ping checks not executing in production
  - ensure_default_ping_check now calls schedule_check after creating the check
  - CheckExecutorWorker now reschedules skipped checks (agent-assigned, SNMP disabled)
    so the scheduling chain is not permanently broken
  - JobHealthCheckWorker now recovers orphaned CheckExecutorWorker jobs (enabled checks
    with no active Oban job) in addition to DeviceMonitorWorker jobs
  - Wrapped ensure_default_ping_check in try/rescue during device creation to prevent
    transient DB deadlocks from failing the entire device creation
  Files: lib/towerops/monitoring.ex,
         lib/towerops/devices.ex,
         lib/towerops/workers/check_executor_worker.ex,
         lib/towerops/workers/job_health_check_worker.ex,
         test/towerops/monitoring_test.exs,
         test/towerops/workers/check_executor_worker_test.exs,
         test/towerops/workers/job_health_check_worker_test.exs,
         test/towerops/snmp_test.exs

2026-03-16
feat: add service checks REST API, GraphQL API, and ping executor
  - New REST API: full CRUD at /api/v1/checks for HTTP, TCP, DNS, and ping checks
  - New GraphQL API: checks/check queries and createCheck/updateCheck/deleteCheck mutations
  - New ping executor: replaces stub with real ICMP ping via system ping command
  - Ping executor parses both macOS and Linux output for packet loss and RTT
  - Check config validation added for ping type (requires "host")
  - API docs updated with Checks resource section
  Files: lib/towerops/monitoring/executors/ping_executor.ex (new),
         lib/towerops/workers/check_executor_worker.ex,
         lib/towerops/monitoring/check.ex,
         lib/towerops_web/controllers/api/v1/checks_controller.ex (new),
         lib/towerops_web/router.ex,
         lib/towerops_web/graphql/types/check.ex (new),
         lib/towerops_web/graphql/resolvers/check.ex (new),
         lib/towerops_web/graphql/schema.ex,
         lib/towerops_web/controllers/api_docs_html/index.html.heex

2026-03-14
fix: network map label size, site layout, and missing links
  - Increased node label font from 11px to 13px and compound node label to 14px
  - Replaced physics-based cose layout with manual grid preset layout to prevent site group overlap
  - Fixed nil source_interface_id crash in topology.ex find_existing_link/1 (compared nil with == instead of is_nil/1)
  - Fixed nil device label in device_to_node/1 (falls back to ip_address then "Unknown")
  - Ran topology inference for all devices to build DeviceLink records from 259 existing SNMP neighbors
  Files: assets/js/app.ts, lib/towerops/topology.ex

2026-03-13
ux: overhaul Preseem devices page to match Gaiia mapping UX
  - Replace filter tabs with pill buttons (All/Unmatched/Matched)
  - Add IP-based match suggestions: detect when a Preseem AP IP matches a TowerOps device IP
  - Show "Match found" badge on rows with suggestions; smart sort (suggestions first)
  - Show suggestion chips in inline linking row for one-click linking
  - Make unlinked AP rows clickable to open linking panel
  - Make IP addresses clickable http links
  Files: preseem_devices_live.ex, preseem_devices_live.html.heex, preseem_devices_live_test.exs

2026-03-12
fix: comprehensive mobile responsive audit and alignment improvements
  - Devices index: restructured header for 390px (tabs on second row, icon-only buttons)
  - Devices index: hide IP/Type/Last Seen/Response/Subs columns at sm/md/lg breakpoints
  - Sites index: overflow-x-auto wrapper, hide QoE/Subs/Location/Coordinates on mobile
  - Dashboard: overflow-x-auto on alerts + site health tables, hide secondary columns mobile
  - Agents index: overflow-x-auto wrappers around both agent tables
  - Device show: restructured sticky header for mobile (name+status / IP+agent row)
  - Site show: min-w-0 on grid columns, hide IP/Subs columns in device health table
  - Agent show: icon-only action buttons on mobile, fix header component flex-wrap
  - Schedule show: flex-wrap header, icon-only buttons, overflow-x-auto on timelines
  - core_components.ex header component: flex-wrap + flex gap-2 justify-end for actions
  - Device show: all dl/dt/dd value pairs now flex-1 text-right ml-4 for consistent column
  - Device show ports table: Speed column now text-right to match In/Out
  Files: device_live/index.html.heex, site_live/index.html.heex, dashboard_live.html.heex,
         agent_live/index.html.heex, device_live/show.html.heex, site_live/show.html.heex,
         agent_live/show.html.heex, schedule_live/show.html.heex,
         components/core_components.ex

2026-03-12
feat: add real-time GraphQL API with time-series queries and subscriptions
  - Added absinthe_phoenix dependency for WebSocket subscription support
  - Created GraphQLSocket with API token auth at /socket/graphql
  - Added Absinthe.Phoenix.Endpoint to endpoint, Absinthe.Subscription to supervision tree
  - New time-series query types: Sensor, SensorReading, InterfaceTrafficPoint, CheckResultPoint
  - New subscription event types: DeviceStatusEvent, AlertEvent, SensorReadingsEvent
  - 4 new queries: deviceSensors, sensorReadings, interfaceTraffic, checkResults
  - 3 subscriptions: deviceStatusChanged, alertEvent, sensorReadingsUpdated
  - Subscription publisher module wired to agent_channel.ex and alerts.ex
  - Org-scoped access control for sensors/interfaces via 3-join verification queries
  - Traffic rate computation from SNMP counter deltas (octets → bps)
  - 17 tests covering queries, org isolation, edge cases, and auth
  - Updated GraphQL API docs page with time-series and subscription sections
  - Files: mix.exs, lib/towerops/application.ex, lib/towerops_web/endpoint.ex,
    lib/towerops_web/graphql_socket.ex, lib/towerops_web/graphql/types/time_series.ex,
    lib/towerops_web/graphql/schema.ex, lib/towerops_web/graphql/resolvers/time_series.ex,
    lib/towerops_web/graphql/subscriptions.ex, lib/towerops_web/channels/agent_channel.ex,
    lib/towerops/alerts.ex, lib/towerops_web/controllers/graphql_docs_html/index.html.heex

2026-03-09
fix: add navigation and footer to changelog page
  - Wrapped ChangelogLive content in Layouts.authenticated component
  - /changelog now includes standard navigation header and footer
  - Matches layout of other authenticated pages
  - Files: lib/towerops_web/live/changelog_live.ex

2026-03-09
feat: add ErrorTrackerNotifier for email alerts on exceptions
  - Added error_tracker_notifier dependency (~> 0.2)
  - Configured email notifications in config/runtime.exs (production only)
  - Sends exception alerts from alerts@towerops.net to graham@towerops.net
  - Added to supervision tree in lib/towerops/application.ex
  - Includes throttling to prevent duplicate error spam
  - Gracefully shuts down in dev/test (no emails sent)
  - Files: mix.exs, config/runtime.exs, lib/towerops/application.ex

2026-03-09
fix: AlertNotificationWorker failing when PagerDuty not configured
  - Worker was treating {:error, :not_configured} as failure, causing retries
  - Added graceful handling for :not_configured in all three actions (trigger, acknowledge, resolve)
  - Now logs as debug and succeeds when PagerDuty integration not set up
  - Prevents unnecessary error noise in ErrorTracker for organizations without PagerDuty
  - Files: lib/towerops/workers/alert_notification_worker.ex

2026-03-06
feat: improve Gaiia webhook setup instructions and descriptions
  - Updated Gaiia provider description to explain what it enables
  - Added "What webhooks enable" section explaining account, billing, and inventory events
  - Enhanced webhook description to explain real-time vs scheduled sync difference
  - Added webhook secret explanation text
  - Improved setup instructions with bold formatting and verification tip
  - Files: lib/towerops_web/live/org/integrations_live.ex, integrations_live.html.heex

2026-03-06
fix: global search (Cmd+K) not returning results
  - handle_event("search") was reading from phx-value-query which contained stale assign
  - Changed to read "value" key from native keyup event params
  - Removed unused phx-value-query attribute from search input
  - Files: lib/towerops_web/live/components/global_search_component.ex

2026-03-06
feat: dynamic global pricing configuration with Stripe integration
  - Add default_free_devices and default_price_per_device to ApplicationSettings
  - Update default price from $1.00 to $2.00/device/month
  - Add Billing.default_free_devices/0 and default_price_per_device/0 with fallbacks
  - Add StripeClient.create_price/1 for metered billing Price creation
  - Add StripeClient.update_subscription_price/2 for price migrations
  - Add Billing.migrate_all_subscriptions_to_price/1 with best-effort migration
  - Add Admin.update_global_pricing/3 with validation and audit logging
  - Add global defaults UI on /admin/organizations with confirmation dialog
  - Update marketing pages to reflect $2/device/month pricing
Files: priv/repo/migrations/*seed_billing_settings.exs,
       lib/towerops/settings/application_setting.ex,
       lib/towerops/billing.ex,
       lib/towerops/billing/stripe_client.ex,
       lib/towerops/admin.ex,
       lib/towerops/admin/audit_log.ex,
       lib/towerops_web/live/admin/org_live/index.ex,
       lib/towerops_web/live/admin/org_live/index.html.heex,
       lib/towerops_web/controllers/page_html/home.html.heex

2026-03-06
feat: per-organization billing override admin UI
  - Add custom_free_device_limit and custom_price_per_device nullable fields to organizations
  - Add billing_override_changeset/2 with validation (separate from main changeset)
  - Add effective_device_limit/1 to SubscriptionLimits respecting custom overrides
  - Add effective_free_device_count/1 and effective_price_per_device/1 to Billing
  - Update billable_device_count and estimated_monthly_cost to use effective values
  - Add Admin.update_billing_overrides/4 with audit logging
  - Add override editing UI to /admin/organizations with edit panel
  - Update org settings page to display effective limits/pricing instead of hardcoded values
Files: priv/repo/migrations/20260306184112_add_billing_overrides_to_organizations.exs,
       lib/towerops/organizations/organization.ex,
       lib/towerops/organizations/subscription_limits.ex,
       lib/towerops/billing.ex,
       lib/towerops/admin.ex,
       lib/towerops/admin/audit_log.ex,
       lib/towerops_web/live/admin/org_live/index.ex,
       lib/towerops_web/live/admin/org_live/index.html.heex,
       lib/towerops_web/live/org/settings_live.html.heex

fix: access interface stats through latest_stat association
  - Template accessed if_in_octets/if_out_octets directly on Interface struct
  - These fields live on InterfaceStat (interface.latest_stat)
  - Added nil guard to prevent KeyError when association not loaded
  - Fixes production 500 error on device detail page
Files: lib/towerops_web/live/device_live/show.html.heex

fix: allow data URLs in CSP for dynamic favicon
  - Added data: to default-src CSP directive to support canvas-generated favicons
  - Fixes dashboard status indicator favicon not working in staging/production
  - DynamicFavicon hook uses canvas.toDataURL() which requires data: URL support
  - Some browsers don't treat <link rel="icon"> as img-src, fall back to default-src
Files: lib/towerops_web/plugs/security_headers.ex

2026-03-05
feat: add rate limiting to admin endpoints (100 req/min)
  - Add :admin type to RateLimit plug with 100 requests per minute per IP
  - Apply rate limiting to all /admin routes (LiveView and controller actions)
  - Protects superuser endpoints from abuse and brute force attempts
  - Aligns with existing auth (10 req/min) and API (1000 req/min) rate limits
Files: lib/towerops_web/plugs/rate_limit.ex,
       lib/towerops_web/router.ex

feat: add TowerOps suffix to all page titles for better tab identification
  - Page titles now show "Page Title | TowerOps" format
  - Helps users identify TowerOps tabs when multiple tabs are open
  - Uses live_title suffix parameter for automatic appending
  - Falls back to just "TowerOps" when no page_title is set
Files: lib/towerops_web/components/layouts/root.html.heex

fix: add --skip-if-loaded flag to test alias to prevent prompts
  - Test alias now uses 'ecto.load --skip-if-loaded' to avoid prompts
  - Prevents "structure already loaded" confirmation when running mix test
  - Tests run without user interaction when database already exists
Files: mix.exs

fix: correct mail adapter message interpolation in login page
  - Fix gettext translation to use proper %{link} interpolation
  - Consolidate fragmented translation strings into single interpolated message
  - Update Spanish translation to include placeholder
  - Prevents display of raw "%{link}" text in development mode
Files: lib/towerops_web/controllers/user_session_html/new.html.heex,
       priv/gettext/auth.pot,
       priv/gettext/en/LC_MESSAGES/auth.po,
       priv/gettext/es/LC_MESSAGES/auth.po

perf: optimize CI database setup with structure.sql dumps (99.7% faster)
  - Replace sequential migration runs (4m8s) with structure.sql loading (416ms)
  - Configure Ecto to dump schema to priv/repo/structure.sql
  - Update test alias to use ecto.load instead of ecto.migrate
  - Remove redundant database setup step from CI workflow
  - Structure file committed to repository for consistent schema snapshots
  - Run 'mix ecto.dump' after migrations to update structure.sql
Files: config/config.exs,
       mix.exs,
       .forgejo/workflows/build.yaml,
       priv/repo/structure.sql (new)

perf: optimize dashboard device count queries (partial N+1 fix)
  - Add Devices.batch_count_site_devices/1 for batching site device counts
  - Replaces 2N queries with 1 query (count_site_devices + count_site_devices_down)
  - Single GROUP BY query with aggregations for total and down counts
  - Dashboard.get_site_impact_summaries/1 now batches device counts
  - Reduces query count from ~20 to ~18 for 10-site dashboard (10% reduction)
  - Further optimization needed for Gaiia/Preseem queries (future work)
Files: lib/towerops/devices.ex,
       lib/towerops/dashboard.ex

fix: suppress noisy health check logs in production
  - Add logger filter to drop K8s health probe logs
  - Prevents log flooding from /health endpoint calls every few seconds
  - Configured in prod.exs logger :default_handler filters
  - TelemetryFilter.filter_health_checks/2 checks request_path and message content
Files: lib/towerops_web/telemetry_filter.ex,
       lib/towerops/application.ex,
       config/prod.exs

security: comprehensive security audit fixes (9 critical/high priority issues)
  - Remove /health/time endpoint exposing system time information (CRITICAL)
  - Add email confirmation check to account data export endpoint (CRITICAL)
  - Add path traversal validation for MIB archive uploads (HIGH)
  - Add input validation for mobile auth device parameters (HIGH)
  - Add heartbeat rate limiting to agent channel (30s min between DB updates) (HIGH)
  - Sanitize 500 error responses to prevent information disclosure (HIGH)
  - Add GraphQL query depth limits (max depth: 10) (MEDIUM)
  - Add message size limits to agent channel (10MB max) (MEDIUM)
Files: lib/towerops_web/controllers/health_controller.ex,
       lib/towerops_web/controllers/api/account_data_controller.ex,
       lib/towerops_web/controllers/api/v1/mib_controller.ex,
       lib/towerops/mobile_sessions/mobile_session.ex,
       lib/towerops_web/channels/agent_channel.ex,
       lib/towerops_web/controllers/error_json.ex,
       lib/towerops_web/router.ex,
       test/towerops_web/controllers/health_controller_test.exs,
       test/towerops_web/controllers/error_json_test.exs

feat: implement LLDP topology discovery via SNMP
  - Add LLDP-MIB walker for discovering network neighbors via SNMP
  - Create device_neighbors table with neighbor relationships
  - Implement Towerops.Topology.Lldp module for SNMP LLDP discovery
  - Add discover_lldp_neighbors/1, list_lldp_neighbors/1, remove_stale_lldp_neighbors/1 functions
  - Automatic device linking when neighbors are found in database
  - Support for IPv4 and IPv6 management addresses
  - Implements Phase 1 of network topology discovery (LLDP via SNMP)
  - LLDP-MIB OIDs: lldpLocSysName, lldpLocPortDesc, lldpRemPortId, lldpRemPortDesc,
    lldpRemSysName, lldpRemManAddr
Files: lib/towerops/topology/lldp.ex, lib/towerops/topology/device_neighbor.ex,
       lib/towerops/topology.ex, priv/repo/migrations/20260305164021_create_device_neighbors.exs

fix: use DATABASE_URL from environment in test config
  - Check for DATABASE_URL environment variable in config/test.exs
  - Use DATABASE_URL if present (CI), fall back to localhost config (local dev)
  - Fixes "connection refused to localhost:5432" errors during compilation in CI
  - Allows config evaluation to use correct hostname before database operations
Files: config/test.exs

ci: set DATABASE_URL globally for test job
  - Move MIX_ENV and DATABASE_URL to job-level environment variables
  - Ensures all database operations (ecto.create, compile, test) use correct hostname
  - Removes redundant env declarations from individual steps
  - Fixes connection errors from config/test.exs hardcoded localhost
Files: .forgejo/workflows/build.yaml

ci: install postgresql-client for pg_isready command
  - Add postgresql-client to system dependencies installation
  - Resolves "pg_isready: command not found" errors during PostgreSQL health checks
  - Essential for database readiness verification before tests run
Files: .forgejo/workflows/build.yaml

2026-03-04
feat: add admin user impersonation with comprehensive audit logging
  - Superusers can impersonate other users (including other superusers) via /admin/users
  - All impersonation events logged to audit_logs table with full context
  - Stop impersonation link appears in user menu when session is impersonated
  - Session assigns track impersonation state (:impersonating_user_id)
  - Audit log captures: impersonator email, target user email, IP address, user agent
  - Access control: only superusers can impersonate, enforced at controller and route level
Files: lib/towerops_web/controllers/admin/user_controller.ex,
       lib/towerops_web/live/nav_live.ex,
       lib/towerops_web/router.ex,
       lib/towerops/admin/audit_logger.ex,
       lib/towerops_web/components/layouts/root.html.heex,
       test/towerops_web/controllers/admin/user_controller_test.exs

feat: add Redis health check with timeout handling
  - Create Towerops.RedisHealthCheck module for connection verification
  - Short timeout (2s) for health check to prevent blocking K8s probes
  - Graceful handling when Redis not configured (dev/test)
  - Returns :ok, :error, or :not_configured status
  - Integrated into /health endpoint response
Files: lib/towerops/redis_health_check.ex,
       lib/towerops_web/controllers/health_controller.ex,
       test/towerops_web/controllers/health_controller_test.exs

fix: add FilterNoisyLogs plug to silence health check spam
  - Create plug to disable logging for /health endpoint
  - Sets phoenix_log: false and plug_skip_telemetry: true
  - Prevents Kubernetes liveness/readiness probes from flooding logs
  - Applied before router in endpoint pipeline
Files: lib/towerops_web/plugs/filter_noisy_logs.ex,
       lib/towerops_web/endpoint.ex

feat: add comprehensive audit logging system
  - Create audit_logs table with action, metadata, IP, user agent tracking
  - Implement Towerops.Admin.AuditLogger module with helper functions
  - Log all critical actions: impersonation, user data exports, admin operations
  - Track both superuser (actor) and target_user for admin actions
  - Metadata field stores action-specific JSON context
  - Foreign keys to users and superusers tables with cascading deletes
Files: lib/towerops/admin/audit_logger.ex,
       lib/towerops/admin/audit_log.ex,
       priv/repo/migrations/20260304XXXXXX_create_audit_logs.exs

2026-03-03
perf: optimize dashboard N+1 queries with batch loading
  - Replace per-site device queries with single batch query using site_id IN (...)
  - Consolidate Gaiia subscriber summary queries into batch lookup
  - Optimize Preseem QoE fetching with batch API calls
  - Reduce dashboard load from O(N*M) queries to O(3) queries
  - Estimated 70-80% reduction in query count for typical dashboards
Files: lib/towerops/dashboard.ex

perf: add database indexes for frequently queried fields
  - Add composite index on alerts (organization_id, alert_type, resolved_at)
  - Add index on devices.organization_id for faster org-level queries
  - Add index on sites.organization_id for dashboard performance
  - Significantly improves dashboard and alert list query performance
Files: priv/repo/migrations/20260303XXXXXX_add_performance_indexes.exs

2026-03-02
fix: prevent TOTP replay attacks with nonce tracking
  - Store used TOTP codes in totp_nonces table with 90-second expiration
  - Validate code hasn't been used before accepting
  - Automatic cleanup of expired nonces via Oban cron job
  - Prevents code reuse within TOTP validity window
Files: lib/towerops/accounts/totp_nonce.ex,
       lib/towerops/accounts.ex,
       priv/repo/migrations/20260302XXXXXX_create_totp_nonces.exs

2026-03-01
feat: add equipment manufacturer detection and tracking
  - Auto-detect manufacturer from SNMP sysDescr during discovery
  - Store in devices.manufacturer field
  - Display in equipment details and lists
  - Enables vendor-specific features and MIB selection
Files: lib/towerops/snmp/discovery.ex,
       lib/towerops/devices/device.ex,
       priv/repo/migrations/20260301XXXXXX_add_manufacturer_to_devices.exs