docs: add LLDP topology discovery analysis from lldp2map research

- Comprehensive analysis of lldp2map's discovery algorithms - BFS recursive topology discovery with depth limits - LLDP-MIB walking with smart fallback strategies - Recommended Elixir/Phoenix implementation phases - Database schema for device neighbors - Topology visualization options - Auto-discovery workflow enhancements - Estimated 8-11 days implementation effort Based on research of https://github.com/buraglio/lldp2map
2026-03-05 10:39:09 -06:00 · 2026-03-05 10:39:09 -06:00 · 4c7c4d7714
commit 4c7c4d7714
parent b216da96f5
1 changed files with 400 additions and 0 deletions
--- a/docs/lldp-topology-discovery-analysis.md
+++ b/docs/lldp-topology-discovery-analysis.md
@ -0,0 +1,400 @@
+# LLDP Topology Discovery Analysis for Towerops
+
+## Executive Summary
+
+`lldp2map` is a well-architected Go CLI tool that performs **recursive network topology discovery** using LLDP (Link Layer Discovery Protocol) via SNMP. This analysis examines how its approach could significantly enhance Towerops' device discovery and network visualization capabilities.
+
+## Current Towerops Discovery Limitations
+
+1. **Manual Device Addition**: Users must manually add each device's IP address
+2. **No Topology Awareness**: No understanding of physical network connections
+3. **No Automatic Neighbor Discovery**: Missing devices that are reachable via LLDP neighbors
+4. **No Visual Network Maps**: No built-in topology visualization
+5. **Limited Relationship Data**: Device relationships not captured or displayed
+
+## lldp2map Key Features & Algorithms
+
+### 1. Recursive BFS (Breadth-First Search) Discovery
+
+**Algorithm** (from `cmd/root.go:138-215`):
+```go
+queue := []queueItem{{host: seedHost, depth: 0}}
+visited := map[string]bool{}
+
+for len(queue) > 0 {
+    item := queue[0]
+    queue = queue[1:]
+
+    if visited[item.host] {
+        continue
+    }
+    visited[item.host] = true
+
+    // Walk LLDP neighbors
+    info, err := lldp.Walk(client)
+
+    // Add discovered neighbors to queue
+    for _, neighbor := range info.Neighbors {
+        for _, ip := range neighbor.MgmtAddrs {
+            if !visited[ipStr] && item.depth < maxHops {
+                queue = append(queue, queueItem{
+                    host: ipStr,
+                    depth: item.depth + 1
+                })
+            }
+        }
+    }
+}
+```
+
+**Benefits**:
+- Automatically discovers entire network from a single seed device
+- Configurable depth limit prevents runaway discovery
+- Handles network cycles via visited tracking
+- Scales to large networks efficiently
+
+### 2. Comprehensive LLDP MIB Walking
+
+**OIDs Used** (from `internal/lldp/walker.go:13-45`):
+```
+Local System:
+- 1.0.8802.1.1.2.1.3.3.0         lldpLocSysName (device hostname)
+- 1.0.8802.1.1.2.1.3.7.1.4       lldpLocPortDesc (local port names)
+
+Remote Neighbors:
+- 1.0.8802.1.1.2.1.4.1.1.7       lldpRemPortId (remote port ID)
+- 1.0.8802.1.1.2.1.4.1.1.8       lldpRemPortDesc (remote port name)
+- 1.0.8802.1.1.2.1.4.1.1.9       lldpRemSysName (neighbor hostname)
+- 1.0.8802.1.1.2.1.4.2.1.3       lldpRemManAddrIfId (management IPs)
+- 1.0.8802.1.1.2.1.4.1.1.4       lldpRemChassisIdSubtype
+- 1.0.8802.1.1.2.1.4.1.1.5       lldpRemChassisId
+
+Interface Addresses:
+- 1.3.6.1.2.1.4.34.1.3           ipAddressIfIndex (IPv4+IPv6, RFC 4293)
+- 1.3.6.1.2.1.4.20.1.1           ipAdEntAddr (IPv4 fallback, RFC 1213)
+
+ARP Resolution:
+- 1.3.6.1.2.1.4.22.1.2           ipNetToPhysAddr (MAC→IP resolution)
+```
+
+**Smart Fallback Strategy**:
+1. Try modern management address table (`lldpRemManAddrIfId`)
+2. Fall back to chassis ID with network address type
+3. For MAC chassis IDs, resolve via device ARP table
+4. This ensures maximum compatibility across device vendors
+
+### 3. Advanced OID Index Parsing
+
+**Example: Management Address Index** (`internal/lldp/walker.go:242-284`):
+```
+Index format: timeMark.portNum.remIndex.addrSubtype.addrLen.addr[bytes]
+- addrSubtype: 1=IPv4 (4 bytes), 2=IPv6 (16 bytes)
+- Handles both IPv4 and IPv6 natively
+- Extracts neighbor identification across multiple MIB entries
+```
+
+**Parsing robustness**:
+- Handles malformed OID responses gracefully
+- Filters out loopback and link-local addresses
+- Deduplicates addresses across multiple discovery paths
+
+### 4. Topology Graph Structure
+
+**Data Model** (`internal/graph/topology.go`):
+```go
+type Node struct {
+    Name  string      // Device hostname
+    IP    string      // Management IP for SNMP
+    Addrs []string    // All interface IPs (optional)
+}
+
+type Edge struct {
+    From     string   // Source device name
+    To       string   // Destination device name
+    FromPort string   // Local port (eth0, ge-0/0/1, etc)
+    ToPort   string   // Remote port
+}
+```
+
+**Edge Deduplication**:
+- Automatically removes A→B when B→A exists
+- Prevents duplicate links in visual diagrams
+- Maintains bidirectional link information
+
+## Recommended Implementation for Towerops
+
+### Phase 1: Core LLDP Discovery (Elixir Migration)
+
+**New Elixir Modules**:
+
+```elixir
+# lib/towerops/snmp/lldp.ex
+defmodule Towerops.Snmp.Lldp do
+  @moduledoc """
+  LLDP topology discovery via SNMP.
+  Walks LLDP-MIB tables to discover network neighbors.
+  """
+
+  @oid_loc_sys_name "1.0.8802.1.1.2.1.3.3.0"
+  @oid_rem_sys_name "1.0.8802.1.1.2.1.4.1.1.9"
+  @oid_rem_man_addr "1.0.8802.1.1.2.1.4.2.1.3"
+
+  def discover_neighbors(device_id) do
+    # Walk LLDP tables
+    # Parse neighbor data
+    # Return list of %{hostname, ports, management_ips}
+  end
+end
+
+# lib/towerops/topology/discovery.ex
+defmodule Towerops.Topology.Discovery do
+  @moduledoc """
+  Recursive network topology discovery using LLDP.
+  """
+
+  def discover_from_seed(device_id, opts \\\\ []) do
+    max_depth = Keyword.get(opts, :max_depth, 10)
+
+    # BFS queue implementation
+    # Track visited devices
+    # Build topology graph
+  end
+end
+
+# lib/towerops/topology/neighbor.ex (new schema)
+defmodule Towerops.Topology.Neighbor do
+  @moduledoc """
+  Represents an LLDP neighbor relationship between devices.
+  """
+
+  schema "device_neighbors" do
+    belongs_to :device, Device
+    belongs_to :neighbor_device, Device
+
+    field :local_port, :string
+    field :remote_port, :string
+    field :discovered_at, :utc_datetime
+    field :last_seen_at, :utc_datetime
+
+    timestamps()
+  end
+end
+```
+
+**Database Schema**:
+```sql
+CREATE TABLE device_neighbors (
+  id UUID PRIMARY KEY,
+  device_id UUID NOT NULL REFERENCES devices(id) ON DELETE CASCADE,
+  neighbor_device_id UUID REFERENCES devices(id) ON DELETE SET NULL,
+  local_port TEXT NOT NULL,
+  remote_port TEXT,
+  discovered_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
+  last_seen_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
+  inserted_at TIMESTAMPTZ NOT NULL,
+  updated_at TIMESTAMPTZ NOT NULL,
+
+  UNIQUE(device_id, local_port, neighbor_device_id)
+);
+
+CREATE INDEX idx_neighbors_device ON device_neighbors(device_id);
+CREATE INDEX idx_neighbors_neighbor ON device_neighbors(neighbor_device_id);
+```
+
+### Phase 2: Topology Worker (Background Discovery)
+
+**New Oban Worker**:
+```elixir
+defmodule Towerops.Workers.TopologyDiscoveryWorker do
+  use Oban.Worker, queue: :discovery, max_attempts: 3
+
+  @impl Oban.Worker
+  def perform(%Oban.Job{args: %{"seed_device_id" => device_id}}) do
+    # Run discovery from seed device
+    # Update device_neighbors table
+    # Create new devices for unknown neighbors
+    # Broadcast topology update to LiveView
+    :ok
+  end
+end
+```
+
+**Trigger Options**:
+1. **Manual**: User clicks "Discover Topology" button
+2. **Scheduled**: Nightly cron job for each site
+3. **Event-Driven**: When new device added with LLDP enabled
+
+### Phase 3: Topology Visualization (LiveView Component)
+
+**New LiveView**:
+```elixir
+defmodule ToweropsWeb.TopologyLive.Show do
+  use ToweropsWeb, :live_view
+
+  def mount(%{"site_id" => site_id}, _session, socket) do
+    neighbors = Topology.list_site_neighbors(site_id)
+
+    {:ok,
+     socket
+     |> assign(:site_id, site_id)
+     |> assign(:neighbors, neighbors)
+     |> assign(:layout_format, "force_directed")}
+  end
+
+  def handle_event("run_discovery", %{"device_id" => device_id}, socket) do
+    TopologyDiscoveryWorker.enqueue(%{seed_device_id: device_id})
+    {:noreply, put_flash(socket, :info, "Topology discovery started")}
+  end
+end
+```
+
+**Visualization Options**:
+1. **SVG Force-Directed Graph**: Use D3.js or Cytoscape.js via Alpine.js hook
+2. **Interactive Canvas**: vis-network or sigma.js
+3. **Export to Draw.io**: Generate XML for external editing
+4. **Export to PNG/PDF**: Server-side Graphviz rendering
+
+### Phase 4: Device Auto-Discovery Enhancement
+
+**Current Flow**:
+```
+User manually adds device → SNMP discovery runs → Sensors created
+```
+
+**Enhanced Flow with LLDP**:
+```
+User adds seed device →
+  SNMP discovery runs →
+  LLDP neighbor discovery runs →
+  New devices auto-created (pending approval) →
+  Recursive SNMP discovery on approved neighbors
+```
+
+**UI Changes**:
+```elixir
+# New "Pending Devices" tab on Devices page
+def handle_event("approve_discovered_device", %{"id" => id}, socket) do
+  case Devices.approve_discovered_device(id) do
+    {:ok, device} ->
+      # Enable SNMP polling
+      # Run discovery on new device
+      # Continue topology discovery if depth allows
+    {:error, _} ->
+      # Show error
+  end
+end
+```
+
+## Benefits for Towerops
+
+### For Network Operators
+
+1. **Faster Onboarding**: Add one device, discover entire network
+2. **Topology Awareness**: Visual understanding of network structure
+3. **Change Detection**: Alert when topology changes (new/missing links)
+4. **Documentation**: Auto-generated network diagrams for customers
+5. **Troubleshooting**: See physical path between devices
+
+### For Developers
+
+1. **Path Analysis**: Calculate shortest path between devices
+2. **Impact Analysis**: "If this device fails, what's affected?"
+3. **Capacity Planning**: Identify bottleneck links
+4. **Configuration Validation**: Verify redundancy in topology
+
+### Technical Advantages
+
+1. **Standards-Based**: LLDP is IEEE 802.1AB standard, vendor-neutral
+2. **Low Overhead**: Single SNMP walk per device
+3. **Incremental Updates**: Only re-scan changed devices
+4. **Scalable**: BFS with depth limit prevents resource exhaustion
+
+## Implementation Complexity
+
+**Estimated Effort**:
+- Phase 1 (Core Discovery): 2-3 days
+- Phase 2 (Background Worker): 1 day
+- Phase 3 (Visualization): 3-5 days
+- Phase 4 (Auto-Discovery): 2 days
+- **Total**: 8-11 days
+
+**Dependencies**:
+- No new external libraries needed (use existing SnmpKit)
+- Graphviz optional (only for export features)
+- D3.js or Cytoscape.js for visualization (already using Alpine.js)
+
+## Risks & Mitigation
+
+**Risks**:
+1. **LLDP Not Enabled**: Some devices don't have LLDP enabled
+   - *Mitigation*: Provide instructions to enable, support manual links
+
+2. **Management Address Missing**: Neighbor has no mgmt IP
+   - *Mitigation*: Use chassis ID fallback like lldp2map does
+
+3. **Large Network Scan**: Could overwhelm SNMP on many devices
+   - *Mitigation*: Rate limiting, depth limits, async background jobs
+
+4. **Stale Data**: Topology changes not reflected in real-time
+   - *Mitigation*: Periodic re-scan, "last_seen_at" timestamps
+
+## Comparison: lldp2map vs Towerops Integration
+
+| Feature | lldp2map (Go CLI) | Towerops Enhancement |
+|---------|-------------------|----------------------|
+| Discovery | One-time, manual | Background cron + manual trigger |
+| Storage | In-memory graph | PostgreSQL with TimescaleDB |
+| Visualization | Static PNG/PDF | Interactive LiveView with real-time updates |
+| Integration | Standalone | Integrated with existing devices, sites, alerts |
+| Multi-tenancy | None | Organization/site scoped |
+| Authentication | CLI flags | Inherited from device SNMP credentials |
+| History | None | Track topology changes over time |
+| Alerting | None | Alert on topology changes |
+
+## Recommended Next Steps
+
+1. **Prototype** (1 day):
+   - Create basic `Lldp.discover_neighbors/1` function
+   - Test on production devices to verify LLDP support
+   - Validate OID parsing logic
+
+2. **Schema & Migration** (0.5 days):
+   - Add `device_neighbors` table
+   - Create Ecto schema and context functions
+
+3. **Worker Implementation** (1 day):
+   - Build `TopologyDiscoveryWorker`
+   - Add manual trigger UI
+
+4. **Visualization** (3-5 days):
+   - Choose visualization library (recommend Cytoscape.js)
+   - Build interactive topology view
+   - Add export options
+
+5. **Auto-Discovery** (2 days):
+   - Pending devices workflow
+   - Approval UI and logic
+
+## Code Quality Notes
+
+**lldp2map Strengths to Adopt**:
+1. ✅ **Comprehensive OID coverage** - handles multiple MIB table formats
+2. ✅ **Smart fallbacks** - gracefully degrades when data missing
+3. ✅ **Clean separation** - SNMP, LLDP, Graph, Render all separate
+4. ✅ **Robust parsing** - handles malformed/unexpected SNMP responses
+5. ✅ **IPv6 support** - first-class support, not an afterthought
+
+**Additional Improvements for Towerops**:
+1. **Historical tracking** - store topology snapshots for trend analysis
+2. **Change alerting** - notify when new neighbors appear/disappear
+3. **Multi-site correlation** - show connections between sites
+4. **Performance metrics** - link utilization on topology edges
+5. **Agent integration** - prefer local agent polling for LLDP data
+
+## Conclusion
+
+lldp2map demonstrates a **production-ready approach to network topology discovery** that would significantly enhance Towerops' value proposition. The recursive BFS algorithm, comprehensive LLDP MIB coverage, and smart fallback mechanisms provide a solid foundation.
+
+By adapting this approach to Elixir/Phoenix with background workers, persistent storage, and interactive visualization, Towerops can offer **automated network discovery and topology mapping** as a key differentiator from manual monitoring tools.
+
+**Recommendation**: Implement Phases 1-2 (8-11 days) to provide immediate value, then iterate on visualization and auto-discovery based on customer feedback.