graham/network
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
network/CLAUDE.md
Graham McIntire ab6ac75391
routers update
2026-05-10 14:48:52 -05:00

13 KiB
Raw Blame History

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

A small fleet of CLIs for documenting, auditing, and operating the vntx WISP — a multi-tower MikroTik backbone with cnMaestro and UISP managing the radios/CPE behind it. Source of truth for network documentation is NetBox at https://netbox.vntx.net/ (API token in env var NETBOX_KEY); source of truth for the routers is whatever the live .rsc exports say.

Repo Layout

The repo is a collection of independent CLIs that each operate on the same physical fleet but talk to different control planes. Each subproject has its own README/CLAUDE.md with details; only repo-wide context lives here.

Subdir Lang Talks to Purpose
mikrotik-tool/ Go MikroTik API-SSL + SSH Fan-out to every router in routers.yaml: list, export (saves <name>.rsc per router), inventory (rewrites inventory.yaml from live data + SNMP), api/schema (ad-hoc API introspection on one router), script (push a .rsc file via SSH and stream output). Authoritative inventory.yaml, subnets.yaml, radios.yaml, ipv6.md, and mpls.md live here.
mikrotik-tool-rs/ Rust MikroTik API-SSL + system ssh Newer rewrite of a subset of mikrotik-tool (list, export, run <cmd…>) with -r name1,name2 filter to scope a fan-out command. Has its own routers.yaml. See mikrotik-tool-rs/CLAUDE.md.
uisp/ Go (main.go) + Python (uisp.py) UISP API at uisp.vntx.net List/approve/upgrade UISP-managed devices. Auth via UISP_KEY env var. UNKNOWN-model devices are filtered by default (use --include-unknown).
cnmaestro/ Python (uv) cnMaestro Cloud OAuth2 Cleanup offline devices and bulk-upgrade firmware. uv sync + .env with client id/secret. See cnmaestro/README.md.

routers.yaml at the repo root is the config consumed by mikrotik-tool/ (Go); mikrotik-tool-rs/ has its own copy. Keep them aligned when adding/removing routers.

poll_snmp_site.sh is a one-off helper for SNMP-polling a list of IPs at a site (./poll_snmp_site.sh <site> <ip>...). The *.rsc files at the repo root are stale exports — fresh ones come out of mikrotik-tool or mikrotik-tool-rs.

Common Commands

# mikrotik-tool (Go)
cd mikrotik-tool && go build .
./mikrotik-tool list
./mikrotik-tool export                       # writes <name>.rsc per router into cwd
./mikrotik-tool api verona /ip/route/print
./mikrotik-tool schema verona /routing/bgp/connection
./mikrotik-tool script 494 494-v6-apply.rsc
./mikrotik-tool inventory                    # re-renders inventory.yaml in place
go test ./...                                # inventory_test.go

# mikrotik-tool-rs (Rust)
cd mikrotik-tool-rs && cargo build --release
cargo run -- list
cargo run -- -r verona,982 export            # filter to a subset before the verb
cargo run -- run /system identity print
cargo test                                   # full suite
cargo test encode_length                     # single test by name substring
cargo clippy --all-targets && cargo fmt

# uisp (Go binary, also a Python script)
cd uisp && go build .
./uisp list
./uisp approve <device>
./uisp upgrade
# python equivalent:
python3 uisp.py list

# cnmaestro (Python, uv)
cd cnmaestro && uv sync
uv run cnmaestro cleanup --dry-run
uv run cnmaestro upgrade --product ePMP,PMP --dry-run
uv run pytest
uv run pytest tests/test_devices.py::test_specific_thing   # single test

UISP_KEY and NETBOX_KEY need to be exported in the shell. cnmaestro reads cnmaestro/.env.

NetBox

  • URL: https://netbox.vntx.net/, API base https://netbox.vntx.net/api/
  • Auth: Authorization: Token $NETBOX_KEY
  • Convention when creating sites/devices: site slugs are lowercase-hyphenated, status active; routers are Manufacturer=MikroTik / Role=Router / primary IP = loopback /32 (10.254.254.x/32).
  • Prefix roles in use: Infrastructure, Customer, Management, Loopback.

MikroTik Router Access

Read-only fan-out (used by all the above tools): API-SSL on port 8729, user grahamro / password in CLAUDE.md history. The committed routers.yaml actually uses graham (not grahamro) because /export requires the ftp policy in addition to read,api, which the read-only group lacks. New routers: ensure the user's group has read,api,ftp (or extend the read group: /user group set read add-policy=ftp).

Every router's loopback is 10.254.254.x/32; see the Fleet Topology table below for the full mapping.

Network Topology Patterns

Access Point Placement

  • Access points are always placed in the top /24 of the management subnet for each tower
  • Example: For management subnet 10.10.16.0/20, APs are in 10.10.31.0/24 (the last /24 in that range)
  • Formula: For subnet X.Y.Z.0/20, APs are in X.Y.(Z+15).0/24

Ubiquiti MAC Prefixes

Common MAC address prefixes for Ubiquiti devices:

  • 00:04:56 (legacy)
  • 00:27:22 (legacy)
  • 04:18:D6
  • 24:A4:3C
  • 68:72:51
  • 80:2A:A8
  • F0:9F:C2
  • FC:EC:DA

MPLS / LDP

FastTrack is incompatible with MPLS on RouterOS 7

FastTrack bypasses the IP forwarding path that MPLS push/pop runs on, so any flow that gets fasttracked on a router whose path uses an MPLS-enabled interface can break — packets either hit the wrong interface or never get labeled, which presents as black-holing for specific source subnets that weren't fasttracked before. Symptoms: pings/SSH/TCP from one source IP work but the same destination is unreachable from another source on the same router; loopback-sourced traffic works but vlan-interface-sourced doesn't.

Fix: before each action=fasttrack-connection rule in chain=forward, add accept rules that match the MPLS-bound interface(s) so those flows never enter the fasttrack path:

/ip firewall filter
add chain=forward action=accept in-interface=<mpls-iface>  comment="bypass fasttrack for MPLS spine (in)"  place-before=<fasttrack-id>
add chain=forward action=accept out-interface=<mpls-iface> comment="bypass fasttrack for MPLS spine (out)" place-before=<fasttrack-id>

Customer→internet flows continue to fasttrack normally; only flows traversing the MPLS spine bypass it.

LDP doesn't label OSPF Type-5 externals by default

Prefixes redistributed via redistribute=connected (e.g., a /27 customer WAN handoff like 204.110.191.0/27) appear as Type-5 external LSAs and don't get LDP label bindings. Forward path to a labeled destination still works, but the return path is plain IP. If you need labeled bidirectional reach for a redistributed prefix, configure an LDP advertise-filter that explicitly includes it.

MPLS-MTU is the labeled-frame cap, not the IP-payload cap

mpls-mtu=1500 caps the labeled frame at 1500 bytes, which means an inner IP payload is limited to 1496 bytes — so 1500-byte DF customer traffic gets icmp-frag-needed. Use mpls-mtu=1508 for a 1500-byte IP payload + 4-byte label, with 4 bytes of headroom for one more stacked label. The AF11/AF24 radio l2mtu is 2024, so 1508 fits comfortably.

Fleet-wide MPLS topology

LDP runs IPv4-only across every backbone link in the network. Every backbone port has mpls-mtu=1508 set explicitly and a fasttrack-bypass pair (in/out) above the fasttrack-connection rule on both endpoints. Documented in mikrotik-tool/mpls.md.

verona ──AF11── climax ──AF24── core ──AF11── culleoka
                  │              │              │
                  │ AF11         │ AF11         │ AF11 (DOWN: power injector unplugged)
                  │              │              │
                 494           newhope ──AF24── lowry
                                 │
                                 │ 60 GHz
                                 │
                                982

Wait — that diagram's links are: climax↔494 (AF11), core↔newhope (AF11), core↔982 (60 GHz), newhope↔lowry (AF24). The climax↔culleoka direct AF11 is currently down at the radio (physical issue), so culleoka traffic transits via core.

Fleet Topology

Routers and loopbacks

All ROS7 routers run RouterOS 7.21.4 long-term (post-2026-05-08 fleet upgrade). Edge runs ROS 6.49.18 (legacy, no MPLS, ignore for the spine).

Router Loopback (10.254.254.x) Hardware Site name
verona .101 CCR2004-16G-2S+ (arm64) verona
climax .102 CCR2004-16G-2S+ (arm64) climax
culleoka .104 CCR1009-7G-1C-1S+ (tile) culleoka
newhope .108 CCR1009-7G-1C-1S+ (tile) newhope
lowry .109 (tile) lowrycrossing
982 .110 (CCR, tile) 982
494 .111 (CCR, tile) 494
core .253 (CCR, arm64) at 380 core/380
edge .254 (legacy, ROS 6.49.18) edge

Tile-arch boxes can run MPLS but not ZeroTier (no .npk for tile).

Backbone links

Every link below has IPv4 LDP enabled at both ends, mpls-mtu=1508, and fasttrack-bypass rules in both directions on both routers.

Link Type A-side iface B-side iface /29 subnet l2mtu
verona↔climax AF11 verona ether3-climax-11ghz climax ether6-verona-11ghz 10.250.1.24/29 2024
climax↔core AF24 climax ether4-380-airfiber24 core ether5-climax 10.250.1.88/29 2024
climax↔494 AF11 climax ether5-494 494 ether2-climax 10.250.1.64/29 1580
climax↔culleoka AF11 climax ether3-culleoka-11ghz culleoka ether1-climax-11ghz 10.250.1.8/29 2024 (link DOWN)
core↔culleoka AF11 core ether6-culleoka-11ghz culleoka ether6-380-11ghz 10.250.1.48/29 2024
core↔newhope AF11 core ether4-newhope newhope ether2-380 10.250.1.56/29 9000
core↔982 60 GHz core ether1-982-60ghz 982 ether7-380 10.250.1.32/29 9000
newhope↔lowry AF24 newhope ether6-lowrycrossing lowry ether1-newhope 10.250.1.104/29 9000
core↔edge wired core sfp-sfpplus1-edge-preseem + ether3-edge-direct edge ports 204.110.191.x n/a

l2mtu mismatches across the fleet are intentional per platform: AF11 base ports default to 2024 on CCR2004 / 1580 on smaller CCRs; jumbo-capable links (60 GHz, AF24-with-jumbo, fiber) go to 9000. Always raise both sides symmetrically when changing l2mtu — single-side raises usually work because Ethernet receivers accept anything ≤ their cap, but symmetric is the rule.

IGP / routing

  • OSPFv2 area backbone-v2 (id 0.0.0.0) on all spine links, SHA-512 auth with auth-id=1 and a shared key. PTP type. BFD is off on every wireless backbone interface (AF11/AF24/60GHz) — the global timers (/routing/bfd/configuration = 200ms×5 = 1s detection) are too aggressive for AF RF: a single >1s burst tore down OSPF on climax↔494 every ~15–40 min until use-bfd=false was applied to both ends 2026-05-09. Wired fiber links may keep BFD if desired. If sub-second failover on a wireless link is genuinely needed, also loosen the BFD timers (e.g. min-rx/min-tx=500ms, multiplier=3) — do not flip use-bfd=true alone.
  • OSPFv3 area backbone-v3 for IPv6 (some interfaces only).
  • All instances redistribute=connected with passthrough filters (/routing filter rule chain=ospf-out rule="accept;").
  • Verona has a static default to 10.250.1.30 (climax) backing up the OSPF default — keep this; bouncing OSPF on verona doesn't blackhole it.
  • Distance-1 static routes also exist on climax for 204.110.191.0/27 so the home /27 has guaranteed return path even if OSPF redistribution hiccups.

Management subnets per tower

10.10.x.0/20 per site, top /24 reserved for APs (see Access Point Placement section). Authoritative mapping is in mikrotik-tool/inventory.yaml. Quick reference:

  • verona: 10.10.0.0/20
  • altoga (behind verona, no router): 10.10.16.0/20
  • climax: 10.10.48.0/20
  • core/380: 10.10.64.0/20
  • culleoka: 10.10.96.0/20
  • 982: 10.10.128.0/20
  • newhope: 10.10.144.0/20
  • 494: 10.10.160.0/20
  • lowry: 10.10.80.0/20

CGNAT pools: 100.64.x.x/22 per tower (see inventory.yaml / subnets.yaml).

graham's home network gotcha

graham's home connects to verona via vlan9_sfpplus1 carrying 204.110.191.0/27 (home router at .1, verona at .30). This /27 is a subnet of the verona hotspot's covered range (204.110.188.0/22). After any verona reboot, ensure /ip hotspot ip-binding has an entry: address=204.110.191.0/27 type=bypassed comment="graham home /27" — without it, hotspot drops all 204.110.191.x traffic in hs-unauth-to chain with icmp-host-prohibited. Symptom is "I can reach verona but nothing past it" from the home network.

IPv6 plan

Per-tower /44s + central server LAN at 2606:1c80::/64 on edge. Full allocation plan in mikrotik-tool/ipv6.md. NetBox has these as IPAM prefixes.

Claude Assistant Guidelines

  • Any time Claude learns something new, automatically add it to CLAUDE.md

Development Best Practices

  • When making scripts, keep them as generic and reusable as possible