perf: optimize dashboard device count queries (partial N+1 fix)

Batch device count queries to reduce N+1 problem in dashboard.

Changes:
- Add Devices.batch_count_site_devices/1 function
  * Uses single GROUP BY query instead of N separate queries
  * Returns map of {site_id => %{total: count, down: count}}
  * Aggregates both total devices and down devices in one query
- Update Dashboard.get_site_impact_summaries/1 to use batch function
  * Eliminates 2N queries (count_site_devices + count_site_devices_down)
  * Reduces to 1 query regardless of site count

Performance impact:
- 10 sites: Reduces from 20 device count queries to 1 (95% reduction)
- 50 sites: Reduces from 100 device count queries to 1 (99% reduction)

Remaining N+1 opportunities:
- Gaiia subscriber summaries (N queries)
- Preseem QoE data (M queries where M = total devices)
- Down device listings (N queries)

Files:
- lib/towerops/devices.ex
- lib/towerops/dashboard.ex
- CHANGELOG.txt
This commit is contained in:
Graham McIntire 2026-03-05 13:15:55 -06:00
parent 4ce1155548
commit f9efdc7339
No known key found for this signature in database
3 changed files with 39 additions and 2 deletions

View file

@ -1,4 +1,14 @@
2026-03-05
perf: optimize dashboard device count queries (partial N+1 fix)
- Add Devices.batch_count_site_devices/1 for batching site device counts
- Replaces 2N queries with 1 query (count_site_devices + count_site_devices_down)
- Single GROUP BY query with aggregations for total and down counts
- Dashboard.get_site_impact_summaries/1 now batches device counts
- Reduces query count from ~20 to ~18 for 10-site dashboard (10% reduction)
- Further optimization needed for Gaiia/Preseem queries (future work)
Files: lib/towerops/devices.ex,
lib/towerops/dashboard.ex
fix: suppress noisy health check logs in production
- Add logger filter to drop K8s health probe logs
- Prevents log flooding from /health endpoint calls every few seconds

View file

@ -192,10 +192,15 @@ defmodule Towerops.Dashboard do
"""
def get_site_impact_summaries(organization_id) do
sites = Sites.list_organization_sites(organization_id)
site_ids = Enum.map(sites, & &1.id)
# Batch query for device counts (replaces 2N queries with 1)
device_counts = Devices.batch_count_site_devices(site_ids)
Enum.map(sites, fn site ->
device_count = Devices.count_site_devices(site.id)
down_count = Devices.count_site_devices_down(site.id)
counts = Map.get(device_counts, site.id, %{total: 0, down: 0})
device_count = counts.total
down_count = counts.down
gaiia_summary = Gaiia.get_site_subscriber_summary(site.id)
# Get impact from down devices at this site

View file

@ -106,6 +106,28 @@ defmodule Towerops.Devices do
)
end
@doc """
Batch count devices for multiple sites.
Returns a map of %{site_id => %{total: count, down: count}}
"""
def batch_count_site_devices(site_ids) when is_list(site_ids) do
# Single query to get both total and down counts grouped by site
from(d in DeviceSchema,
where: d.site_id in ^site_ids,
group_by: d.site_id,
select: %{
site_id: d.site_id,
total: count(d.id),
down: filter(count(d.id), d.status == :down)
}
)
|> Repo.all()
|> Map.new(fn %{site_id: site_id, total: total, down: down} ->
{site_id, %{total: total, down: down}}
end)
end
@doc """
Returns the count of devices that is down for a site.
"""