# High Availability Setup with MikroTik

## Overview
Using MikroTik's built-in features to ensure K3s cluster remains accessible when node1 is down.

## Option 1: MikroTik Load Balancing with PCC (Per Connection Classifier)

### On MikroTik Router:
```mikrotik
# 1. Create address list for K3s nodes
/ip firewall address-list
add address=10.0.101.21 list=k3s-nodes comment="node1"
add address=10.0.101.22 list=k3s-nodes comment="node2"
add address=10.0.101.23 list=k3s-nodes comment="node3"

# 2. Create NAT rules for incoming traffic to distribute to nodes
# For HTTP/HTTPS traffic (ports 80, 443)
/ip firewall nat
add chain=dstnat action=dst-nat protocol=tcp dst-address=204.110.191.2 \
    dst-port=80 to-addresses=10.0.101.21-10.0.101.23 to-ports=80 \
    comment="K3s HTTP load balance"
    
add chain=dstnat action=dst-nat protocol=tcp dst-address=204.110.191.2 \
    dst-port=443 to-addresses=10.0.101.21-10.0.101.23 to-ports=443 \
    comment="K3s HTTPS load balance"

# 3. Add connection marking for load balancing
/ip firewall mangle
add chain=prerouting action=mark-connection new-connection-mark=k3s-conn-1 \
    dst-address=204.110.191.2 dst-port=80,443 protocol=tcp \
    per-connection-classifier=both-addresses:3/0
    
add chain=prerouting action=mark-connection new-connection-mark=k3s-conn-2 \
    dst-address=204.110.191.2 dst-port=80,443 protocol=tcp \
    per-connection-classifier=both-addresses:3/1
    
add chain=prerouting action=mark-connection new-connection-mark=k3s-conn-3 \
    dst-address=204.110.191.2 dst-port=80,443 protocol=tcp \
    per-connection-classifier=both-addresses:3/2
```

## Option 2: MikroTik with VRRP (Virtual Router Redundancy Protocol)

### Prerequisites:
- Assign additional IPs to each node for VRRP
- Configure MikroTik to route to VRRP virtual IP

### On Each K3s Node:
```bash
# Install keepalived
sudo apt-get update && sudo apt-get install -y keepalived

# Create keepalived configuration
sudo tee /etc/keepalived/keepalived.conf << EOF
vrrp_instance VI_1 {
    state BACKUP              # All nodes start as BACKUP
    interface eth0            # Your network interface
    virtual_router_id 51      # Same on all nodes
    priority $(hostname | grep -o '[0-9]*')0   # node1=10, node2=20, node3=30
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass k3scluster
    }
    virtual_ipaddress {
        10.0.101.100/24       # Virtual IP for cluster
    }
}
EOF

# Start keepalived
sudo systemctl enable --now keepalived
```

### On MikroTik:
```mikrotik
# Route public IP traffic to VRRP virtual IP
/ip firewall nat
add chain=dstnat action=dst-nat protocol=tcp dst-address=204.110.191.2 \
    dst-port=80,443 to-addresses=10.0.101.100 \
    comment="K3s VRRP Virtual IP"
```

## Option 3: MikroTik Netwatch with Failover (Simplest)

This monitors nodes and updates NAT dynamically:

```mikrotik
# 1. Create scripts for failover
/system script
add name="k3s-use-node1" source={
    /ip firewall nat set [find comment="K3s HTTP"] to-addresses=10.0.101.21
    /ip firewall nat set [find comment="K3s HTTPS"] to-addresses=10.0.101.21
    :log info "K3s traffic switched to node1"
}

add name="k3s-use-node2" source={
    /ip firewall nat set [find comment="K3s HTTP"] to-addresses=10.0.101.22
    /ip firewall nat set [find comment="K3s HTTPS"] to-addresses=10.0.101.22
    :log info "K3s traffic switched to node2"
}

add name="k3s-use-node3" source={
    /ip firewall nat set [find comment="K3s HTTP"] to-addresses=10.0.101.23
    /ip firewall nat set [find comment="K3s HTTPS"] to-addresses=10.0.101.23
    :log info "K3s traffic switched to node3"
}

# 2. Create initial NAT rules
/ip firewall nat
add chain=dstnat action=dst-nat protocol=tcp dst-address=204.110.191.2 \
    dst-port=80 to-addresses=10.0.101.21 to-ports=80 comment="K3s HTTP"
    
add chain=dstnat action=dst-nat protocol=tcp dst-address=204.110.191.2 \
    dst-port=443 to-addresses=10.0.101.21 to-ports=443 comment="K3s HTTPS"

# 3. Set up Netwatch monitors
/tool netwatch
add host=10.0.101.21 interval=10s timeout=5s \
    down-script={
        :if ([/tool netwatch get [find host=10.0.101.22] status]="up") do={
            /system script run k3s-use-node2
        } else={
            :if ([/tool netwatch get [find host=10.0.101.23] status]="up") do={
                /system script run k3s-use-node3
            }
        }
    } \
    up-script="/system script run k3s-use-node1"

add host=10.0.101.22 interval=10s timeout=5s
add host=10.0.101.23 interval=10s timeout=5s
```

## Recommended: Configure K3s Services for Any Node Access

Make sure your services use NodePort or LoadBalancer type:

```yaml
# Example service configuration
apiVersion: v1
kind: Service
metadata:
  name: myapp
spec:
  type: NodePort
  ports:
    - port: 80
      targetPort: 8080
      nodePort: 32080  # This port will be accessible on ALL nodes
  selector:
    app: myapp
```

Then configure MikroTik to forward to the NodePort on any available node.

## Testing the Failover

1. Verify current routing:
   ```bash
   curl -v http://204.110.191.2
   ```

2. Shut down node1:
   ```bash
   ssh node1 sudo shutdown -h now
   ```

3. Watch MikroTik logs and verify traffic switches:
   ```mikrotik
   /log print follow where topics~"info"
   ```

4. Verify service remains accessible:
   ```bash
   curl -v http://204.110.191.2
   ```