From 4c7c4d7714ecbc3b1972a012b01594e538a65244 Mon Sep 17 00:00:00 2001 From: Graham McIntire Date: Thu, 5 Mar 2026 10:39:09 -0600 Subject: [PATCH] docs: add LLDP topology discovery analysis from lldp2map research - Comprehensive analysis of lldp2map's discovery algorithms - BFS recursive topology discovery with depth limits - LLDP-MIB walking with smart fallback strategies - Recommended Elixir/Phoenix implementation phases - Database schema for device neighbors - Topology visualization options - Auto-discovery workflow enhancements - Estimated 8-11 days implementation effort Based on research of https://github.com/buraglio/lldp2map --- docs/lldp-topology-discovery-analysis.md | 400 +++++++++++++++++++++++ 1 file changed, 400 insertions(+) create mode 100644 docs/lldp-topology-discovery-analysis.md diff --git a/docs/lldp-topology-discovery-analysis.md b/docs/lldp-topology-discovery-analysis.md new file mode 100644 index 00000000..6ed18b5b --- /dev/null +++ b/docs/lldp-topology-discovery-analysis.md @@ -0,0 +1,400 @@ +# LLDP Topology Discovery Analysis for Towerops + +## Executive Summary + +`lldp2map` is a well-architected Go CLI tool that performs **recursive network topology discovery** using LLDP (Link Layer Discovery Protocol) via SNMP. This analysis examines how its approach could significantly enhance Towerops' device discovery and network visualization capabilities. + +## Current Towerops Discovery Limitations + +1. **Manual Device Addition**: Users must manually add each device's IP address +2. **No Topology Awareness**: No understanding of physical network connections +3. **No Automatic Neighbor Discovery**: Missing devices that are reachable via LLDP neighbors +4. **No Visual Network Maps**: No built-in topology visualization +5. **Limited Relationship Data**: Device relationships not captured or displayed + +## lldp2map Key Features & Algorithms + +### 1. Recursive BFS (Breadth-First Search) Discovery + +**Algorithm** (from `cmd/root.go:138-215`): +```go +queue := []queueItem{{host: seedHost, depth: 0}} +visited := map[string]bool{} + +for len(queue) > 0 { + item := queue[0] + queue = queue[1:] + + if visited[item.host] { + continue + } + visited[item.host] = true + + // Walk LLDP neighbors + info, err := lldp.Walk(client) + + // Add discovered neighbors to queue + for _, neighbor := range info.Neighbors { + for _, ip := range neighbor.MgmtAddrs { + if !visited[ipStr] && item.depth < maxHops { + queue = append(queue, queueItem{ + host: ipStr, + depth: item.depth + 1 + }) + } + } + } +} +``` + +**Benefits**: +- Automatically discovers entire network from a single seed device +- Configurable depth limit prevents runaway discovery +- Handles network cycles via visited tracking +- Scales to large networks efficiently + +### 2. Comprehensive LLDP MIB Walking + +**OIDs Used** (from `internal/lldp/walker.go:13-45`): +``` +Local System: +- 1.0.8802.1.1.2.1.3.3.0 lldpLocSysName (device hostname) +- 1.0.8802.1.1.2.1.3.7.1.4 lldpLocPortDesc (local port names) + +Remote Neighbors: +- 1.0.8802.1.1.2.1.4.1.1.7 lldpRemPortId (remote port ID) +- 1.0.8802.1.1.2.1.4.1.1.8 lldpRemPortDesc (remote port name) +- 1.0.8802.1.1.2.1.4.1.1.9 lldpRemSysName (neighbor hostname) +- 1.0.8802.1.1.2.1.4.2.1.3 lldpRemManAddrIfId (management IPs) +- 1.0.8802.1.1.2.1.4.1.1.4 lldpRemChassisIdSubtype +- 1.0.8802.1.1.2.1.4.1.1.5 lldpRemChassisId + +Interface Addresses: +- 1.3.6.1.2.1.4.34.1.3 ipAddressIfIndex (IPv4+IPv6, RFC 4293) +- 1.3.6.1.2.1.4.20.1.1 ipAdEntAddr (IPv4 fallback, RFC 1213) + +ARP Resolution: +- 1.3.6.1.2.1.4.22.1.2 ipNetToPhysAddr (MAC→IP resolution) +``` + +**Smart Fallback Strategy**: +1. Try modern management address table (`lldpRemManAddrIfId`) +2. Fall back to chassis ID with network address type +3. For MAC chassis IDs, resolve via device ARP table +4. This ensures maximum compatibility across device vendors + +### 3. Advanced OID Index Parsing + +**Example: Management Address Index** (`internal/lldp/walker.go:242-284`): +``` +Index format: timeMark.portNum.remIndex.addrSubtype.addrLen.addr[bytes] +- addrSubtype: 1=IPv4 (4 bytes), 2=IPv6 (16 bytes) +- Handles both IPv4 and IPv6 natively +- Extracts neighbor identification across multiple MIB entries +``` + +**Parsing robustness**: +- Handles malformed OID responses gracefully +- Filters out loopback and link-local addresses +- Deduplicates addresses across multiple discovery paths + +### 4. Topology Graph Structure + +**Data Model** (`internal/graph/topology.go`): +```go +type Node struct { + Name string // Device hostname + IP string // Management IP for SNMP + Addrs []string // All interface IPs (optional) +} + +type Edge struct { + From string // Source device name + To string // Destination device name + FromPort string // Local port (eth0, ge-0/0/1, etc) + ToPort string // Remote port +} +``` + +**Edge Deduplication**: +- Automatically removes A→B when B→A exists +- Prevents duplicate links in visual diagrams +- Maintains bidirectional link information + +## Recommended Implementation for Towerops + +### Phase 1: Core LLDP Discovery (Elixir Migration) + +**New Elixir Modules**: + +```elixir +# lib/towerops/snmp/lldp.ex +defmodule Towerops.Snmp.Lldp do + @moduledoc """ + LLDP topology discovery via SNMP. + Walks LLDP-MIB tables to discover network neighbors. + """ + + @oid_loc_sys_name "1.0.8802.1.1.2.1.3.3.0" + @oid_rem_sys_name "1.0.8802.1.1.2.1.4.1.1.9" + @oid_rem_man_addr "1.0.8802.1.1.2.1.4.2.1.3" + + def discover_neighbors(device_id) do + # Walk LLDP tables + # Parse neighbor data + # Return list of %{hostname, ports, management_ips} + end +end + +# lib/towerops/topology/discovery.ex +defmodule Towerops.Topology.Discovery do + @moduledoc """ + Recursive network topology discovery using LLDP. + """ + + def discover_from_seed(device_id, opts \\\\ []) do + max_depth = Keyword.get(opts, :max_depth, 10) + + # BFS queue implementation + # Track visited devices + # Build topology graph + end +end + +# lib/towerops/topology/neighbor.ex (new schema) +defmodule Towerops.Topology.Neighbor do + @moduledoc """ + Represents an LLDP neighbor relationship between devices. + """ + + schema "device_neighbors" do + belongs_to :device, Device + belongs_to :neighbor_device, Device + + field :local_port, :string + field :remote_port, :string + field :discovered_at, :utc_datetime + field :last_seen_at, :utc_datetime + + timestamps() + end +end +``` + +**Database Schema**: +```sql +CREATE TABLE device_neighbors ( + id UUID PRIMARY KEY, + device_id UUID NOT NULL REFERENCES devices(id) ON DELETE CASCADE, + neighbor_device_id UUID REFERENCES devices(id) ON DELETE SET NULL, + local_port TEXT NOT NULL, + remote_port TEXT, + discovered_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + last_seen_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + inserted_at TIMESTAMPTZ NOT NULL, + updated_at TIMESTAMPTZ NOT NULL, + + UNIQUE(device_id, local_port, neighbor_device_id) +); + +CREATE INDEX idx_neighbors_device ON device_neighbors(device_id); +CREATE INDEX idx_neighbors_neighbor ON device_neighbors(neighbor_device_id); +``` + +### Phase 2: Topology Worker (Background Discovery) + +**New Oban Worker**: +```elixir +defmodule Towerops.Workers.TopologyDiscoveryWorker do + use Oban.Worker, queue: :discovery, max_attempts: 3 + + @impl Oban.Worker + def perform(%Oban.Job{args: %{"seed_device_id" => device_id}}) do + # Run discovery from seed device + # Update device_neighbors table + # Create new devices for unknown neighbors + # Broadcast topology update to LiveView + :ok + end +end +``` + +**Trigger Options**: +1. **Manual**: User clicks "Discover Topology" button +2. **Scheduled**: Nightly cron job for each site +3. **Event-Driven**: When new device added with LLDP enabled + +### Phase 3: Topology Visualization (LiveView Component) + +**New LiveView**: +```elixir +defmodule ToweropsWeb.TopologyLive.Show do + use ToweropsWeb, :live_view + + def mount(%{"site_id" => site_id}, _session, socket) do + neighbors = Topology.list_site_neighbors(site_id) + + {:ok, + socket + |> assign(:site_id, site_id) + |> assign(:neighbors, neighbors) + |> assign(:layout_format, "force_directed")} + end + + def handle_event("run_discovery", %{"device_id" => device_id}, socket) do + TopologyDiscoveryWorker.enqueue(%{seed_device_id: device_id}) + {:noreply, put_flash(socket, :info, "Topology discovery started")} + end +end +``` + +**Visualization Options**: +1. **SVG Force-Directed Graph**: Use D3.js or Cytoscape.js via Alpine.js hook +2. **Interactive Canvas**: vis-network or sigma.js +3. **Export to Draw.io**: Generate XML for external editing +4. **Export to PNG/PDF**: Server-side Graphviz rendering + +### Phase 4: Device Auto-Discovery Enhancement + +**Current Flow**: +``` +User manually adds device → SNMP discovery runs → Sensors created +``` + +**Enhanced Flow with LLDP**: +``` +User adds seed device → + SNMP discovery runs → + LLDP neighbor discovery runs → + New devices auto-created (pending approval) → + Recursive SNMP discovery on approved neighbors +``` + +**UI Changes**: +```elixir +# New "Pending Devices" tab on Devices page +def handle_event("approve_discovered_device", %{"id" => id}, socket) do + case Devices.approve_discovered_device(id) do + {:ok, device} -> + # Enable SNMP polling + # Run discovery on new device + # Continue topology discovery if depth allows + {:error, _} -> + # Show error + end +end +``` + +## Benefits for Towerops + +### For Network Operators + +1. **Faster Onboarding**: Add one device, discover entire network +2. **Topology Awareness**: Visual understanding of network structure +3. **Change Detection**: Alert when topology changes (new/missing links) +4. **Documentation**: Auto-generated network diagrams for customers +5. **Troubleshooting**: See physical path between devices + +### For Developers + +1. **Path Analysis**: Calculate shortest path between devices +2. **Impact Analysis**: "If this device fails, what's affected?" +3. **Capacity Planning**: Identify bottleneck links +4. **Configuration Validation**: Verify redundancy in topology + +### Technical Advantages + +1. **Standards-Based**: LLDP is IEEE 802.1AB standard, vendor-neutral +2. **Low Overhead**: Single SNMP walk per device +3. **Incremental Updates**: Only re-scan changed devices +4. **Scalable**: BFS with depth limit prevents resource exhaustion + +## Implementation Complexity + +**Estimated Effort**: +- Phase 1 (Core Discovery): 2-3 days +- Phase 2 (Background Worker): 1 day +- Phase 3 (Visualization): 3-5 days +- Phase 4 (Auto-Discovery): 2 days +- **Total**: 8-11 days + +**Dependencies**: +- No new external libraries needed (use existing SnmpKit) +- Graphviz optional (only for export features) +- D3.js or Cytoscape.js for visualization (already using Alpine.js) + +## Risks & Mitigation + +**Risks**: +1. **LLDP Not Enabled**: Some devices don't have LLDP enabled + - *Mitigation*: Provide instructions to enable, support manual links + +2. **Management Address Missing**: Neighbor has no mgmt IP + - *Mitigation*: Use chassis ID fallback like lldp2map does + +3. **Large Network Scan**: Could overwhelm SNMP on many devices + - *Mitigation*: Rate limiting, depth limits, async background jobs + +4. **Stale Data**: Topology changes not reflected in real-time + - *Mitigation*: Periodic re-scan, "last_seen_at" timestamps + +## Comparison: lldp2map vs Towerops Integration + +| Feature | lldp2map (Go CLI) | Towerops Enhancement | +|---------|-------------------|----------------------| +| Discovery | One-time, manual | Background cron + manual trigger | +| Storage | In-memory graph | PostgreSQL with TimescaleDB | +| Visualization | Static PNG/PDF | Interactive LiveView with real-time updates | +| Integration | Standalone | Integrated with existing devices, sites, alerts | +| Multi-tenancy | None | Organization/site scoped | +| Authentication | CLI flags | Inherited from device SNMP credentials | +| History | None | Track topology changes over time | +| Alerting | None | Alert on topology changes | + +## Recommended Next Steps + +1. **Prototype** (1 day): + - Create basic `Lldp.discover_neighbors/1` function + - Test on production devices to verify LLDP support + - Validate OID parsing logic + +2. **Schema & Migration** (0.5 days): + - Add `device_neighbors` table + - Create Ecto schema and context functions + +3. **Worker Implementation** (1 day): + - Build `TopologyDiscoveryWorker` + - Add manual trigger UI + +4. **Visualization** (3-5 days): + - Choose visualization library (recommend Cytoscape.js) + - Build interactive topology view + - Add export options + +5. **Auto-Discovery** (2 days): + - Pending devices workflow + - Approval UI and logic + +## Code Quality Notes + +**lldp2map Strengths to Adopt**: +1. ✅ **Comprehensive OID coverage** - handles multiple MIB table formats +2. ✅ **Smart fallbacks** - gracefully degrades when data missing +3. ✅ **Clean separation** - SNMP, LLDP, Graph, Render all separate +4. ✅ **Robust parsing** - handles malformed/unexpected SNMP responses +5. ✅ **IPv6 support** - first-class support, not an afterthought + +**Additional Improvements for Towerops**: +1. **Historical tracking** - store topology snapshots for trend analysis +2. **Change alerting** - notify when new neighbors appear/disappear +3. **Multi-site correlation** - show connections between sites +4. **Performance metrics** - link utilization on topology edges +5. **Agent integration** - prefer local agent polling for LLDP data + +## Conclusion + +lldp2map demonstrates a **production-ready approach to network topology discovery** that would significantly enhance Towerops' value proposition. The recursive BFS algorithm, comprehensive LLDP MIB coverage, and smart fallback mechanisms provide a solid foundation. + +By adapting this approach to Elixir/Phoenix with background workers, persistent storage, and interactive visualization, Towerops can offer **automated network discovery and topology mapping** as a key differentiator from manual monitoring tools. + +**Recommendation**: Implement Phases 1-2 (8-11 days) to provide immediate value, then iterate on visualization and auto-discovery based on customer feedback.