# Automatic Network Topology Inference

**Date**: 2026-02-12
**Status**: Draft

## Goal

Replace the current on-demand, LLDP-only topology builder with a persistent topology model that automatically infers device relationships, roles, and link types from polling data. The topology updates incrementally as data arrives, supports both managed and discovered devices, and provides the foundation for future AI analysis and network optimization features.

## Target Users

ISP/WISP operators managing mixed-vendor networks with hundreds to thousands of devices. These operators add core infrastructure (routers, switches, APs, backhaul radios) as managed devices but typically do not add every CPE/subscriber module.

## Data Model

### Migration 1: Add device role fields to `devices`

```elixir
alter table(:devices) do
  add :device_role, :string  # enum stored as string
  add :device_role_source, :string, default: "inferred"
end
```

**`device_role` values**: `core_router`, `distribution_switch`, `access_switch`, `access_point`, `cpe`, `backhaul_radio`, `ups`, `server`, `firewall`, `unknown`

**`device_role_source` values**: `inferred` (auto-detected, can be overwritten by inference), `manual` (user-set, never overwritten by inference)

### Migration 2: `device_links` table

Persistent record of a connection between two devices.

```elixir
create table(:device_links, primary_key: false) do
  add :id, :binary_id, primary_key: true
  add :source_device_id, references(:devices, type: :binary_id, on_delete: :delete_all), null: false
  add :target_device_id, references(:devices, type: :binary_id, on_delete: :delete_all), null: true
  add :source_interface_id, references(:snmp_interfaces, type: :binary_id, on_delete: :nilify_all)
  add :target_interface_id, references(:snmp_interfaces, type: :binary_id, on_delete: :nilify_all)
  add :link_type, :string, null: false
  add :confidence, :float, null: false, default: 0.5
  add :metadata, :map, default: %{}
  add :discovered_remote_name, :string
  add :discovered_remote_ip, :string
  add :discovered_remote_mac, :string
  add :last_confirmed_at, :utc_datetime, null: false
  timestamps(type: :utc_datetime)
end

create index(:device_links, [:source_device_id])
create index(:device_links, [:target_device_id])
create unique_index(:device_links, [:source_device_id, :source_interface_id, :discovered_remote_mac],
  name: :device_links_source_interface_remote_mac)
```

**`link_type` values**: `lldp`, `cdp`, `mac_match`, `wireless_association`, `arp_inference`, `manual`

**`target_device_id`**: Nullable. When the remote end is a discovered (unmanaged) device, this is nil and the `discovered_remote_*` fields identify it. When the remote end is a managed device, this FK is populated.

### Migration 3: `device_link_evidence` table

Records why a link is believed to exist. Enables explainability and re-evaluation.

```elixir
create table(:device_link_evidence, primary_key: false) do
  add :id, :binary_id, primary_key: true
  add :device_link_id, references(:device_links, type: :binary_id, on_delete: :delete_all), null: false
  add :evidence_type, :string, null: false
  add :evidence_data, :map, default: %{}
  add :observed_at, :utc_datetime, null: false
  timestamps(type: :utc_datetime)
end

create index(:device_link_evidence, [:device_link_id])
create index(:device_link_evidence, [:observed_at])
```

**`evidence_type` values**: `lldp_neighbor`, `cdp_neighbor`, `mac_on_interface`, `arp_entry`, `wireless_registration`, `subnet_match`

## Topology Inference Engine

### Module: `Towerops.Topology`

Primary public function: `process_device(device)` — called after each poll cycle.

### Pipeline

```
1. Collect evidence from this device's latest SNMP data
2. Match evidence to known managed devices (by MAC, IP, hostname)
3. Upsert device_links with merged confidence scores
4. Record evidence in device_link_evidence
5. Expire links with no recent evidence
6. Re-evaluate device_role if device_role_source == :inferred
7. Broadcast change on PubSub if anything changed
```

### Evidence Collectors

Each collector examines one data source and returns link candidates:

| Collector | Source | Base Confidence | Notes |
|-----------|--------|-----------------|-------|
| `collect_lldp_evidence/1` | `snmp_neighbors` (protocol=lldp) | 0.95 | Highest — explicit L2 protocol |
| `collect_cdp_evidence/1` | `snmp_neighbors` (protocol=cdp) | 0.95 | Same quality as LLDP |
| `collect_mac_evidence/1` | `snmp_mac_addresses` cross-referenced with device interface MACs | 0.7 | Good but can be indirect through unmanaged switches |
| `collect_arp_evidence/1` | `snmp_arp_entries` cross-referenced with device IPs | 0.6 | Strong for /30-/31 point-to-point, weaker for shared subnets |
| `collect_wireless_evidence/1` | Vendor-specific registration tables (phase 2) | 0.9 | Very reliable when available |

### Confidence Merging

When multiple evidence types support the same link, confidence is combined:

```
merged = 1 - ((1 - a) * (1 - b))
```

Examples:
- LLDP (0.95) alone → 0.95
- LLDP (0.95) + MAC match (0.7) → 0.985
- MAC match (0.7) + ARP (0.6) → 0.88

### Device Matching

Evidence must be matched to devices. Matching priority:

1. **MAC address** → `snmp_interfaces.if_phys_address` on any managed device
2. **IP address** → `devices.ip_address` on any managed device
3. **Hostname** → `devices.name` (case-insensitive) on any managed device

If no managed device matches, the link is stored with `target_device_id: nil` and the `discovered_remote_*` fields populated. This creates a discovered (unmanaged) node in the topology.

### Device Role Inference

Rules evaluated in priority order. Only applied when `device_role_source == :inferred`.

| Priority | Condition | Role |
|----------|-----------|------|
| 1 | LLDP capabilities include "router" + device has 10+ interfaces | `core_router` |
| 2 | LLDP capabilities include "router" | `distribution_switch` |
| 3 | LLDP capabilities include "wlan-ap" | `access_point` |
| 4 | Device has outbound wireless links to many devices (>2) | `access_point` |
| 5 | Device connects to exactly one AP via wireless | `cpe` |
| 6 | Device has wireless backhaul interface + no wireless clients | `backhaul_radio` |
| 7 | LLDP capabilities include "bridge" or device has 10+ switchports | `access_switch` |
| 8 | Known UPS vendor (APC, CyberPower, Eaton, Tripp Lite) | `ups` |
| 9 | Known firewall vendor (Fortinet, Palo Alto, pfSense) | `firewall` |
| 10 | Known server platform (Linux, Windows, VMware) | `server` |
| 11 | Fallback | `unknown` |

Vendor detection uses `snmp_devices.manufacturer` (already collected).

## Integration Points

### Polling Integration

`DevicePollerWorker` calls `Topology.process_device(device)` after writing SNMP data. This is a function call, not a new worker. Topology inference runs in the same Oban job, adding minimal overhead per poll.

### Stale Link Cleanup

Added to the existing `NeighborCleanupWorker` (hourly Oban cron):

- Links with `last_confirmed_at` older than 24 hours → set confidence to 0.1 (stale)
- Links with `last_confirmed_at` older than 72 hours → delete link and evidence

### PubSub

After `process_device/1` detects a change (new link, removed link, role change, confidence change >0.1), broadcast:

```elixir
Phoenix.PubSub.broadcast(Towerops.PubSub, "topology:#{org_id}", {:topology_updated, org_id})
```

The `NetworkMapLive` already subscribes to this topic and has a handler — it just never received messages before. Now it does.

## UI Changes

### Network Map

**"Added Devices" tab (default):**
- Only managed devices from `devices` table
- Only links where both `source_device_id` and `target_device_id` are non-nil (both sides managed)
- Role-based node shapes/icons instead of generic circles
- Link confidence indicated by edge style: solid (>0.9), dashed (0.5-0.9), dotted (<0.5)
- Link type shown on hover (LLDP, MAC match, etc.)
- Clean operational view — no clutter from subscriber devices

**"All Devices" tab:**
- Everything from Added tab, plus discovered nodes from links with `target_device_id: nil`
- Discovered nodes rendered with muted colors and dashed borders
- AP nodes show discovered CPEs clustered around them
- When an AP has >10 discovered devices, collapse into a count badge ("47 clients"). Click to expand.
- Discovered nodes show info popover on click (MAC, IP, signal if available, "Add device" action)

**Node visual mapping by role:**

| Role | Shape | Color accent |
|------|-------|-------------|
| `core_router` | Rounded rectangle | Blue |
| `distribution_switch` | Rectangle | Teal |
| `access_switch` | Rectangle (smaller) | Teal (lighter) |
| `access_point` | Triangle | Green |
| `cpe` | Small circle | Gray |
| `backhaul_radio` | Diamond | Orange |
| `ups` | Square | Yellow |
| `firewall` | Hexagon | Red |
| `server` | Rectangle | Purple |
| `unknown` | Circle | Gray |

### Device Detail Page

**On AP devices (`device_role == :access_point`):**

New "Connected Devices" section in overview tab. Compact table:

| Name | IP | Signal | Link Type | Managed |
|------|-----|--------|-----------|---------|
| CPE-Customer-Smith | 10.0.5.47 | -62 dBm | Wireless | No |
| BACKHAUL-TOWER2 | 10.0.1.2 | -44 dBm | LLDP | Yes (link) |

Discovered devices show connection data but no status. Managed devices link to their detail page.

**On CPE devices (`device_role == :cpe`):**

New "Uplink" section in overview tab showing the AP this CPE connects to, with signal/link type and a link to the AP's detail page.

**On all devices:**

The existing Neighbors tab is enhanced to show persistent links from `device_links` instead of raw `snmp_neighbors` data. Shows link type, confidence, evidence count, and last confirmed time. Links to the remote device's detail page when managed.

### Device Edit Page

New `device_role` dropdown with all role options. When the user manually selects a role, `device_role_source` is set to `manual`, preventing future inference from overriding it. A "Reset to auto-detect" option sets it back to `inferred`.

## Phase 2: Wireless Registration Tables

Not in initial scope but designed for. After the core topology model is working:

1. Add vendor-specific wireless registration table polling to `DevicePollerWorker`
2. New schema `snmp_wireless_registrations` (MAC, signal, CCQ, TX/RX rate, uptime, last seen)
3. New evidence collector `collect_wireless_evidence/1` feeds into the same topology pipeline
4. Signal/CCQ data stored in `device_links.metadata` for display

Vendor coverage priority: MikroTik (registration-table), Ubiquiti (dot11StaTable), Cambium (PMP/ePMP registration).

## Testing Strategy

- Unit tests for each evidence collector with mock SNMP data
- Unit tests for confidence merging
- Unit tests for device role inference rules
- Integration test: poll a device, verify links created
- Integration test: remove a neighbor, verify link expires after cleanup
- Integration test: manual role override preserved through re-inference
- LiveView test: map renders with role-based styling
- LiveView test: AP detail page shows connected devices section
