# home-recursive-failover.rsc # Replace stateful netwatch enable/disable failover with stateless recursive # check-gateway failover. Root cause of the "VNTX default stuck disabled" # incident: netwatch down-script persists disabled=yes (survives reboots), # and its probe to 4.2.2.2 is only glued to the VNTX path while the probe-pin # route is active — so the check can die permanently (verona hotspot-binding # loss) or leak out another WAN (TMO netwatch reads "up" today with the TMO # interface physically down, answered via VNTX). # # New model (standard ROS recursive failover): # - probe pin: 4.2.2.x/32 -> WAN gw %interface, scope=10, NO check-gateway. # Stays active whenever the interface has link, so probes always egress # the WAN under test regardless of any default-route state. # - default: 0.0.0.0/0 gw=4.2.2.x target-scope=11 check-gateway=ping. # check-gateway pings 4.2.2.x THROUGH the pin; 2 misses (~20s) -> route # inactive (not disabled), first success -> active again. Nothing latches. # - Detection is 10s-cadence/2-miss instead of netwatch's 2s/1s single # packet — deliberate: the AF11 backbone has documented >1s RF bursts # (same reason BFD was turned off fleet-wide 2026-05-09). /ip route # pins: remove check-gateway (the check belongs on the recursive default) set [find comment="TMO probe pin"] check-gateway=none set [find comment="VNTX probe pin"] check-gateway=none set [find comment="Starlink probe pin"] check-gateway=none # defaults: recurse through the probe IP, carry the check set [find comment="Default via TMO (primary)"] gateway=4.2.2.1 check-gateway=ping set [find comment="Default via VNTX (secondary)"] gateway=4.2.2.2 check-gateway=ping # Starlink default already gateway=4.2.2.3 + check-gateway=ping (correct form) # tmo table default (eweka policy routing) already recurses via 4.2.2.1; # add the check so it goes inactive (falls back to main) when TMO upstream dies set [find comment="tmo table default"] check-gateway=ping # netwatch scripts are the latch — remove them /tool netwatch remove [find comment="TMO path"] remove [find comment="VNTX path"] remove [find comment="Starlink path"]