docs: add LLDP topology discovery analysis from lldp2map research

- Comprehensive analysis of lldp2map's discovery algorithms - BFS recursive topology discovery with depth limits - LLDP-MIB walking with smart fallback strategies - Recommended Elixir/Phoenix implementation phases - Database schema for device neighbors - Topology visualization options - Auto-discovery workflow enhancements - Estimated 8-11 days implementation effort Based on research of https://github.com/buraglio/lldp2map
2026-03-05 10:39:09 -06:00 · 2026-03-05 10:39:09 -06:00 · 4c7c4d7714
commit 4c7c4d7714
parent b216da96f5
1 changed files with 400 additions and 0 deletions
--- a/docs/lldp-topology-discovery-analysis.md
+++ b/docs/lldp-topology-discovery-analysis.md
@ -0,0 +1,400 @@
 # LLDP Topology Discovery Analysis for Towerops
 ## Executive Summary
 `lldp2map` is a well-architected Go CLI tool that performs **recursive network topology discovery** using LLDP (Link Layer Discovery Protocol) via SNMP. This analysis examines how its approach could significantly enhance Towerops' device discovery and network visualization capabilities.
 ## Current Towerops Discovery Limitations
 1. **Manual Device Addition**: Users must manually add each device's IP address
 2. **No Topology Awareness**: No understanding of physical network connections
 3. **No Automatic Neighbor Discovery**: Missing devices that are reachable via LLDP neighbors
 4. **No Visual Network Maps**: No built-in topology visualization
 5. **Limited Relationship Data**: Device relationships not captured or displayed
 ## lldp2map Key Features & Algorithms
 ### 1. Recursive BFS (Breadth-First Search) Discovery
 **Algorithm** (from `cmd/root.go:138-215`):
 ```go
 queue := []queueItem{{host: seedHost, depth: 0}}
 visited := map[string]bool{}
 for len(queue) > 0 {
    item := queue[0]
    queue = queue[1:]
    if visited[item.host] {
        continue
    }
    visited[item.host] = true
    // Walk LLDP neighbors
    info, err := lldp.Walk(client)
    // Add discovered neighbors to queue
    for _, neighbor := range info.Neighbors {
        for _, ip := range neighbor.MgmtAddrs {
            if !visited[ipStr] && item.depth < maxHops {
                queue = append(queue, queueItem{
                    host: ipStr,
                    depth: item.depth + 1
                })
            }
        }
    }
 }
 ```
 **Benefits**:
 - Automatically discovers entire network from a single seed device
 - Configurable depth limit prevents runaway discovery
 - Handles network cycles via visited tracking
 - Scales to large networks efficiently
 ### 2. Comprehensive LLDP MIB Walking
 **OIDs Used** (from `internal/lldp/walker.go:13-45`):
 ```
 Local System:
 - 1.0.8802.1.1.2.1.3.3.0         lldpLocSysName (device hostname)
 - 1.0.8802.1.1.2.1.3.7.1.4       lldpLocPortDesc (local port names)
 Remote Neighbors:
 - 1.0.8802.1.1.2.1.4.1.1.7       lldpRemPortId (remote port ID)
 - 1.0.8802.1.1.2.1.4.1.1.8       lldpRemPortDesc (remote port name)
 - 1.0.8802.1.1.2.1.4.1.1.9       lldpRemSysName (neighbor hostname)
 - 1.0.8802.1.1.2.1.4.2.1.3       lldpRemManAddrIfId (management IPs)
 - 1.0.8802.1.1.2.1.4.1.1.4       lldpRemChassisIdSubtype
 - 1.0.8802.1.1.2.1.4.1.1.5       lldpRemChassisId
 Interface Addresses:
 - 1.3.6.1.2.1.4.34.1.3           ipAddressIfIndex (IPv4+IPv6, RFC 4293)
 - 1.3.6.1.2.1.4.20.1.1           ipAdEntAddr (IPv4 fallback, RFC 1213)
 ARP Resolution:
 - 1.3.6.1.2.1.4.22.1.2           ipNetToPhysAddr (MAC→IP resolution)
 ```
 **Smart Fallback Strategy**:
 1. Try modern management address table (`lldpRemManAddrIfId`)
 2. Fall back to chassis ID with network address type
 3. For MAC chassis IDs, resolve via device ARP table
 4. This ensures maximum compatibility across device vendors
 ### 3. Advanced OID Index Parsing
 **Example: Management Address Index** (`internal/lldp/walker.go:242-284`):
 ```
 Index format: timeMark.portNum.remIndex.addrSubtype.addrLen.addr[bytes]
 - addrSubtype: 1=IPv4 (4 bytes), 2=IPv6 (16 bytes)
 - Handles both IPv4 and IPv6 natively
 - Extracts neighbor identification across multiple MIB entries
 ```
 **Parsing robustness**:
 - Handles malformed OID responses gracefully
 - Filters out loopback and link-local addresses
 - Deduplicates addresses across multiple discovery paths
 ### 4. Topology Graph Structure
 **Data Model** (`internal/graph/topology.go`):
 ```go
 type Node struct {
    Name  string      // Device hostname
    IP    string      // Management IP for SNMP
    Addrs []string    // All interface IPs (optional)
 }
 type Edge struct {
    From     string   // Source device name
    To       string   // Destination device name
    FromPort string   // Local port (eth0, ge-0/0/1, etc)
    ToPort   string   // Remote port
 }
 ```
 **Edge Deduplication**:
 - Automatically removes A→B when B→A exists
 - Prevents duplicate links in visual diagrams
 - Maintains bidirectional link information
 ## Recommended Implementation for Towerops
 ### Phase 1: Core LLDP Discovery (Elixir Migration)
 **New Elixir Modules**:
 ```elixir
 # lib/towerops/snmp/lldp.ex
 defmodule Towerops.Snmp.Lldp do
  @moduledoc """
  LLDP topology discovery via SNMP.
  Walks LLDP-MIB tables to discover network neighbors.
  """
  @oid_loc_sys_name "1.0.8802.1.1.2.1.3.3.0"
  @oid_rem_sys_name "1.0.8802.1.1.2.1.4.1.1.9"
  @oid_rem_man_addr "1.0.8802.1.1.2.1.4.2.1.3"
  def discover_neighbors(device_id) do
    # Walk LLDP tables
    # Parse neighbor data
    # Return list of %{hostname, ports, management_ips}
  end
 end
 # lib/towerops/topology/discovery.ex
 defmodule Towerops.Topology.Discovery do
  @moduledoc """
  Recursive network topology discovery using LLDP.
  """
  def discover_from_seed(device_id, opts \\\\ []) do
    max_depth = Keyword.get(opts, :max_depth, 10)
    # BFS queue implementation
    # Track visited devices
    # Build topology graph
  end
 end
 # lib/towerops/topology/neighbor.ex (new schema)
 defmodule Towerops.Topology.Neighbor do
  @moduledoc """
  Represents an LLDP neighbor relationship between devices.
  """
  schema "device_neighbors" do
    belongs_to :device, Device
    belongs_to :neighbor_device, Device
    field :local_port, :string
    field :remote_port, :string
    field :discovered_at, :utc_datetime
    field :last_seen_at, :utc_datetime
    timestamps()
  end
 end
 ```
 **Database Schema**:
 ```sql
 CREATE TABLE device_neighbors (
  id UUID PRIMARY KEY,
  device_id UUID NOT NULL REFERENCES devices(id) ON DELETE CASCADE,
  neighbor_device_id UUID REFERENCES devices(id) ON DELETE SET NULL,
  local_port TEXT NOT NULL,
  remote_port TEXT,
  discovered_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  last_seen_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  inserted_at TIMESTAMPTZ NOT NULL,
  updated_at TIMESTAMPTZ NOT NULL,
  UNIQUE(device_id, local_port, neighbor_device_id)
 );
 CREATE INDEX idx_neighbors_device ON device_neighbors(device_id);
 CREATE INDEX idx_neighbors_neighbor ON device_neighbors(neighbor_device_id);
 ```
 ### Phase 2: Topology Worker (Background Discovery)
 **New Oban Worker**:
 ```elixir
 defmodule Towerops.Workers.TopologyDiscoveryWorker do
  use Oban.Worker, queue: :discovery, max_attempts: 3
  @impl Oban.Worker
  def perform(%Oban.Job{args: %{"seed_device_id" => device_id}}) do
    # Run discovery from seed device
    # Update device_neighbors table
    # Create new devices for unknown neighbors
    # Broadcast topology update to LiveView
    :ok
  end
 end
 ```
 **Trigger Options**:
 1. **Manual**: User clicks "Discover Topology" button
 2. **Scheduled**: Nightly cron job for each site
 3. **Event-Driven**: When new device added with LLDP enabled
 ### Phase 3: Topology Visualization (LiveView Component)
 **New LiveView**:
 ```elixir
 defmodule ToweropsWeb.TopologyLive.Show do
  use ToweropsWeb, :live_view
  def mount(%{"site_id" => site_id}, _session, socket) do
    neighbors = Topology.list_site_neighbors(site_id)
    {:ok,
     socket
     |> assign(:site_id, site_id)
     |> assign(:neighbors, neighbors)
     |> assign(:layout_format, "force_directed")}
  end
  def handle_event("run_discovery", %{"device_id" => device_id}, socket) do
    TopologyDiscoveryWorker.enqueue(%{seed_device_id: device_id})
    {:noreply, put_flash(socket, :info, "Topology discovery started")}
  end
 end
 ```
 **Visualization Options**:
 1. **SVG Force-Directed Graph**: Use D3.js or Cytoscape.js via Alpine.js hook
 2. **Interactive Canvas**: vis-network or sigma.js
 3. **Export to Draw.io**: Generate XML for external editing
 4. **Export to PNG/PDF**: Server-side Graphviz rendering
 ### Phase 4: Device Auto-Discovery Enhancement
 **Current Flow**:
 ```
 User manually adds device → SNMP discovery runs → Sensors created
 ```
 **Enhanced Flow with LLDP**:
 ```
 User adds seed device →
  SNMP discovery runs →
  LLDP neighbor discovery runs →
  New devices auto-created (pending approval) →
  Recursive SNMP discovery on approved neighbors
 ```
 **UI Changes**:
 ```elixir
 # New "Pending Devices" tab on Devices page
 def handle_event("approve_discovered_device", %{"id" => id}, socket) do
  case Devices.approve_discovered_device(id) do
    {:ok, device} ->
      # Enable SNMP polling
      # Run discovery on new device
      # Continue topology discovery if depth allows
    {:error, _} ->
      # Show error
  end
 end
 ```
 ## Benefits for Towerops
 ### For Network Operators
 1. **Faster Onboarding**: Add one device, discover entire network
 2. **Topology Awareness**: Visual understanding of network structure
 3. **Change Detection**: Alert when topology changes (new/missing links)
 4. **Documentation**: Auto-generated network diagrams for customers
 5. **Troubleshooting**: See physical path between devices
 ### For Developers
 1. **Path Analysis**: Calculate shortest path between devices
 2. **Impact Analysis**: "If this device fails, what's affected?"
 3. **Capacity Planning**: Identify bottleneck links
 4. **Configuration Validation**: Verify redundancy in topology
 ### Technical Advantages
 1. **Standards-Based**: LLDP is IEEE 802.1AB standard, vendor-neutral
 2. **Low Overhead**: Single SNMP walk per device
 3. **Incremental Updates**: Only re-scan changed devices
 4. **Scalable**: BFS with depth limit prevents resource exhaustion
 ## Implementation Complexity
 **Estimated Effort**:
 - Phase 1 (Core Discovery): 2-3 days
 - Phase 2 (Background Worker): 1 day
 - Phase 3 (Visualization): 3-5 days
 - Phase 4 (Auto-Discovery): 2 days
 - **Total**: 8-11 days
 **Dependencies**:
 - No new external libraries needed (use existing SnmpKit)
 - Graphviz optional (only for export features)
 - D3.js or Cytoscape.js for visualization (already using Alpine.js)
 ## Risks & Mitigation
 **Risks**:
 1. **LLDP Not Enabled**: Some devices don't have LLDP enabled
   - *Mitigation*: Provide instructions to enable, support manual links
 2. **Management Address Missing**: Neighbor has no mgmt IP
   - *Mitigation*: Use chassis ID fallback like lldp2map does
 3. **Large Network Scan**: Could overwhelm SNMP on many devices
   - *Mitigation*: Rate limiting, depth limits, async background jobs
 4. **Stale Data**: Topology changes not reflected in real-time
   - *Mitigation*: Periodic re-scan, "last_seen_at" timestamps
 ## Comparison: lldp2map vs Towerops Integration
 | Feature | lldp2map (Go CLI) | Towerops Enhancement |
 |---------|-------------------|----------------------|
 | Discovery | One-time, manual | Background cron + manual trigger |
 | Storage | In-memory graph | PostgreSQL with TimescaleDB |
 | Visualization | Static PNG/PDF | Interactive LiveView with real-time updates |
 | Integration | Standalone | Integrated with existing devices, sites, alerts |
 | Multi-tenancy | None | Organization/site scoped |
 | Authentication | CLI flags | Inherited from device SNMP credentials |
 | History | None | Track topology changes over time |
 | Alerting | None | Alert on topology changes |
 ## Recommended Next Steps
 1. **Prototype** (1 day):
   - Create basic `Lldp.discover_neighbors/1` function
   - Test on production devices to verify LLDP support
   - Validate OID parsing logic
 2. **Schema & Migration** (0.5 days):
   - Add `device_neighbors` table
   - Create Ecto schema and context functions
 3. **Worker Implementation** (1 day):
   - Build `TopologyDiscoveryWorker`
   - Add manual trigger UI
 4. **Visualization** (3-5 days):
   - Choose visualization library (recommend Cytoscape.js)
   - Build interactive topology view
   - Add export options
 5. **Auto-Discovery** (2 days):
   - Pending devices workflow
   - Approval UI and logic
 ## Code Quality Notes
 **lldp2map Strengths to Adopt**:
 1. ✅ **Comprehensive OID coverage** - handles multiple MIB table formats
 2. ✅ **Smart fallbacks** - gracefully degrades when data missing
 3. ✅ **Clean separation** - SNMP, LLDP, Graph, Render all separate
 4. ✅ **Robust parsing** - handles malformed/unexpected SNMP responses
 5. ✅ **IPv6 support** - first-class support, not an afterthought
 **Additional Improvements for Towerops**:
 1. **Historical tracking** - store topology snapshots for trend analysis
 2. **Change alerting** - notify when new neighbors appear/disappear
 3. **Multi-site correlation** - show connections between sites
 4. **Performance metrics** - link utilization on topology edges
 5. **Agent integration** - prefer local agent polling for LLDP data
 ## Conclusion
 lldp2map demonstrates a **production-ready approach to network topology discovery** that would significantly enhance Towerops' value proposition. The recursive BFS algorithm, comprehensive LLDP MIB coverage, and smart fallback mechanisms provide a solid foundation.
 By adapting this approach to Elixir/Phoenix with background workers, persistent storage, and interactive visualization, Towerops can offer **automated network discovery and topology mapping** as a key differentiator from manual monitoring tools.
 **Recommendation**: Implement Phases 1-2 (8-11 days) to provide immediate value, then iterate on visualization and auto-discovery based on customer feedback.