perf: optimize dashboard device count queries (partial N+1 fix)
Batch device count queries to reduce N+1 problem in dashboard.
Changes:
- Add Devices.batch_count_site_devices/1 function
* Uses single GROUP BY query instead of N separate queries
* Returns map of {site_id => %{total: count, down: count}}
* Aggregates both total devices and down devices in one query
- Update Dashboard.get_site_impact_summaries/1 to use batch function
* Eliminates 2N queries (count_site_devices + count_site_devices_down)
* Reduces to 1 query regardless of site count
Performance impact:
- 10 sites: Reduces from 20 device count queries to 1 (95% reduction)
- 50 sites: Reduces from 100 device count queries to 1 (99% reduction)
Remaining N+1 opportunities:
- Gaiia subscriber summaries (N queries)
- Preseem QoE data (M queries where M = total devices)
- Down device listings (N queries)
Files:
- lib/towerops/devices.ex
- lib/towerops/dashboard.ex
- CHANGELOG.txt
This commit is contained in:
parent
4ce1155548
commit
f9efdc7339
3 changed files with 39 additions and 2 deletions
|
|
@ -1,4 +1,14 @@
|
|||
2026-03-05
|
||||
perf: optimize dashboard device count queries (partial N+1 fix)
|
||||
- Add Devices.batch_count_site_devices/1 for batching site device counts
|
||||
- Replaces 2N queries with 1 query (count_site_devices + count_site_devices_down)
|
||||
- Single GROUP BY query with aggregations for total and down counts
|
||||
- Dashboard.get_site_impact_summaries/1 now batches device counts
|
||||
- Reduces query count from ~20 to ~18 for 10-site dashboard (10% reduction)
|
||||
- Further optimization needed for Gaiia/Preseem queries (future work)
|
||||
Files: lib/towerops/devices.ex,
|
||||
lib/towerops/dashboard.ex
|
||||
|
||||
fix: suppress noisy health check logs in production
|
||||
- Add logger filter to drop K8s health probe logs
|
||||
- Prevents log flooding from /health endpoint calls every few seconds
|
||||
|
|
|
|||
|
|
@ -192,10 +192,15 @@ defmodule Towerops.Dashboard do
|
|||
"""
|
||||
def get_site_impact_summaries(organization_id) do
|
||||
sites = Sites.list_organization_sites(organization_id)
|
||||
site_ids = Enum.map(sites, & &1.id)
|
||||
|
||||
# Batch query for device counts (replaces 2N queries with 1)
|
||||
device_counts = Devices.batch_count_site_devices(site_ids)
|
||||
|
||||
Enum.map(sites, fn site ->
|
||||
device_count = Devices.count_site_devices(site.id)
|
||||
down_count = Devices.count_site_devices_down(site.id)
|
||||
counts = Map.get(device_counts, site.id, %{total: 0, down: 0})
|
||||
device_count = counts.total
|
||||
down_count = counts.down
|
||||
gaiia_summary = Gaiia.get_site_subscriber_summary(site.id)
|
||||
|
||||
# Get impact from down devices at this site
|
||||
|
|
|
|||
|
|
@ -106,6 +106,28 @@ defmodule Towerops.Devices do
|
|||
)
|
||||
end
|
||||
|
||||
@doc """
|
||||
Batch count devices for multiple sites.
|
||||
|
||||
Returns a map of %{site_id => %{total: count, down: count}}
|
||||
"""
|
||||
def batch_count_site_devices(site_ids) when is_list(site_ids) do
|
||||
# Single query to get both total and down counts grouped by site
|
||||
from(d in DeviceSchema,
|
||||
where: d.site_id in ^site_ids,
|
||||
group_by: d.site_id,
|
||||
select: %{
|
||||
site_id: d.site_id,
|
||||
total: count(d.id),
|
||||
down: filter(count(d.id), d.status == :down)
|
||||
}
|
||||
)
|
||||
|> Repo.all()
|
||||
|> Map.new(fn %{site_id: site_id, total: total, down: down} ->
|
||||
{site_id, %{total: total, down: down}}
|
||||
end)
|
||||
end
|
||||
|
||||
@doc """
|
||||
Returns the count of devices that is down for a site.
|
||||
"""
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue