infra/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Repository Overview

This is a multi-environment infrastructure-as-code repository managing:

• Ansible: Configuration management for servers across multiple environments (VNTX, home, app servers)
• OpenTofu: Proxmox/Talos VM provisioning and Porkbun domain NS management
• Home Lab: Server configurations with Tailscale integration

Common Commands

Proxmox VM Provisioning

Infrastructure VMs are provisioned using OpenTofu and configured via Butane/Ignition (Flatcar) or Ansible (traditional Linux):

# Infrastructure VMs are managed by OpenTofu (tofu/proxmox.tf)
cd tofu && tofu plan && tofu apply

# Post-provision configuration via Ansible
cd ansible && source .envrc

# Configure BIND9 nameserver at 204.110.191.222
ansible-playbook playbook.yml --tags provision,bind9

Flatcar Linux VMs (managed by lucidsolns/flatcar-vm/proxmox module):

• No manual template needed — module downloads the Flatcar image automatically
• VM definition: tofu/proxmox.tf (module flatcar_resolver, VMID 110)
• Configuration: tofu/flatcar-resolver.bu (Butane → Ignition)
• Config files: tofu/unbound.conf (rendered from group_vars), tofu/blocklist.rpz
• Static IP: 10.0.0.30/24, 8GB RAM, 2 cores, UEFI boot
• Default user: core (passwordless sudo)
• Unbound runs as a Docker container (mvance/unbound) with config + blocklist mounted
• DoH endpoint on 127.0.0.1:8080 for future Caddy proxy
• To deploy: update flatcar-resolver.bu or config files, then tofu apply
• Proxmox storage local must have snippets and import content types enabled (/etc/pve/storage.cfg)

### Ansible Operations
```bash
# Run main playbook
cd ansible && ./run.sh

# Bootstrap new host
make bootstrap-init HOSTNAME=hostname ANSIBLE_HOST=ip_address

# Run specific playbook
ansible-playbook general.yml

# Run with specific tags
ansible-playbook -t caddy general.yml

# Gather facts from all hosts
make facts

OpenTofu Operations

IMPORTANT: Use OpenTofu (tofu) instead of terraform for all operations.

cd tofu
tofu init
tofu plan
tofu apply

# Format terraform files before committing
tofu fmt

Talos Operations (Home Cluster)

# Change to talos directory
cd /Users/graham/dev/infra/talos

# Check cluster health
talosctl --talosconfig talosconfig health

# View cluster nodes
kubectl get nodes

# View all pods
kubectl get pods -A

# Check talos services
talosctl --talosconfig talosconfig service

# View logs
talosctl --talosconfig talosconfig logs -f kubelet

Architecture

Directory Structure

• ansible/ - Ansible playbooks and roles for server configuration

  • roles/ - Reusable roles (base, caddy, docker, resolvers, etc.)
  • group_vars/ - Group-specific variables
  • host_vars/ - Host-specific variables
  • Main playbooks: playbook.yml, general.yml, bootstrap.yml

• tofu/ - OpenTofu infrastructure management

  • Proxmox/Talos VM provisioning (main.tf, talos_vms.tf)
  • Porkbun domain NS management (dns.tf)
    • Provider: marcfrederick/porkbun ~> 1.3
    • Credentials via PORKBUN_API_KEY and PORKBUN_SECRET_API_KEY env vars
    • Manages NS records for all Porkbun domains (as393837, cloudflare, or porkbun default NS)
  • Backend: Local state (committed to repo)

• home/ - Home lab configurations

  • cluster/ - Kubernetes application deployments
    • argocd/ - ArgoCD install + Application manifests (app-of-apps via root-app.yaml)
    • aprs/ - APRS.me deployment manifests (legacy; live manifests in the aprs.me repo's k8s/)

Key Infrastructure Services

Monitoring: Icinga2 with MariaDB backend DNS: Unbound resolvers, PowerDNS for reverse zones VPN: Tailscale integration across environments Web Proxy: Caddy for reverse proxy and SSL termination

Security Notes

• Ansible user (UID 10001) with passwordless sudo
• SSH keys fetched from GitHub for authentication
• Tailscale requires TAILSCALE_KEY environment variable
• OpenTofu credentials should use environment variables (PORKBUN_API_KEY, PORKBUN_SECRET_API_KEY)

Development Workflow

Adding New Hosts

1. Add host to ansible/hosts inventory
2. Create host_vars file if needed
3. Run bootstrap: make bootstrap-init HOSTNAME=x ANSIBLE_HOST=y
4. Apply configuration: ansible-playbook -l hostname general.yml

DNS Zone Changes (BIND9)

1. Edit zone YAML in ansible/group_vars/bind9_servers/<domain>.yml
2. Run ansible playbook to deploy zone changes

Domain NS Changes (Porkbun Registrar)

1. Edit tofu/dns.tf — update the domain's NS in the local.domains map
2. Run tofu fmt to format
3. Review with tofu plan
4. Apply with tofu apply

Testing

Ansible

# Syntax check
ansible-playbook --syntax-check playbook.yml

# Dry run
ansible-playbook --check playbook.yml

# Test on specific host
ansible-playbook -l hostname playbook.yml

Terraform

tofu validate
tofu plan

Recent Infrastructure Updates

Talos Home Cluster (10.0.19.1-6)

• Purpose: Primary home lab Kubernetes cluster
• Cluster Name: home-cluster
• Kubernetes Version: v1.36.1
• Control Plane Nodes: 3 (10.0.19.1-3)
• Worker Nodes: 3 (10.0.19.4-6)
• API Endpoint: https://10.0.19.10:6443
• Proxmox Cluster: "home" (node1, node2, node3 at 10.0.19.101-103)
• VM Disks: local-lvm on each Proxmox node
• Persistent Storage: TBD (Longhorn was removed — evaluating alternatives)
• CNI: Flannel
• Talos Extensions (workers only):
  • iscsi-tools v0.2.0
  • util-linux-tools 2.41.2
• Installer image: factory.talos.dev/installer/613e1592b2da41ae5e265e8789429f22e121aab91cb4deb6bc3c0b6262961245:v1.13.3
• Control plane nodes use the stock ghcr.io/siderolabs/installer:v1.13.3
• Deployment: Managed via OpenTofu in /tofu/
• Configuration: /talos/ directory
  • talosconfig - Talosctl client config
  • secrets.yaml - Cluster secrets
  • controlplane.yaml / worker.yaml - Base node configs
  • worker.yaml / controlplane.yaml - Base node configs (applied with per-node patches)
  • patches/*.yaml - Per-node patches (hostname, disk overrides)
• GitOps: ArgoCD (replaced Flux on 2026-04-29). See home/cluster/argocd/ and the GitOps section below.
• Applications: deployed via ArgoCD Application manifests in home/cluster/argocd/apps/
  • APRS.me (namespace aprs, https://aprs.me) — sourced from codeberg.org/gmcintire/aprs.me.git ./k8s
  • gridmap (namespace gridmap) — sourced from codeberg.org/gmcintire/gridmap.git ./k8s
  • prop (namespace prop) — sourced from codeberg.org/gmcintire/prop.git ./k8s
  • towerops (namespace towerops) — sourced from codeberg.org/gmcintire/towerops-web.git ./k8s
• Documentation: See /talos/README.md and /cluster/README.md for detailed operations
• Config Notes:
  • install.extraKernelArgs and install.grubUseUKICmdline cannot be used together
  • Worker disks with existing data (e.g. old Ceph LVM) must be wiped before Talos will partition them: talosctl reset --user-disks-to-wipe /dev/sdb --wipe-mode user-disks --reboot

GitOps via ArgoCD (home-cluster)

• Install: home/cluster/argocd/install/ — upstream ArgoCD manifests + a Kustomize patch enabling server.insecure=true (cloudflared terminates TLS) and reposerver.git.modules.enabled=false (the aprs.me repo has a stale submodule pin that breaks fetch otherwise)
• App-of-apps: home/cluster/argocd/root-app.yaml watches home/cluster/argocd/apps/ in this repo and reconciles all child Applications. Add a new YAML to apps/ and push — Argo picks it up.
• Repo creds: forgejo-mcintire-creds Secret in argocd ns is a credential template for https://codeberg.org/gmcintire — username/password copied from flux-system/forgejo-git-credentials.
• UI: argo.w5isp.com via cloudflared tunnel → argocd-server.argocd.svc.cluster.local:80. Initial admin password is in argocd/argocd-initial-admin-secret.
• Image auto-update: Flux's ImagePolicy/ImageUpdateAutomation is gone. Annotations like # {"$imagepolicy": "flux-system:prop"} in app repo manifests are inert. To restore, install argocd-image-updater.

Caddy server: w5isp.w5isp.com (204.110.191.200)

• Purpose: Single-node server running Debian 13
• Features:
  • Web services with automatic SSL
  • Single public IP handles all services

DNS Configuration

• Photos: photos.w5isp.com → 204.110.191.212 (Caddy) → 100.107.11.77:2283
• Mailcow: mcintire.me mail records → mail.w5isp.com (sync.w5isp.com at 204.110.191.216)
  • Full mail setup with MX, SPF, DKIM, DMARC records
  • Autodiscover/Autoconfig for mail clients
  • Backup MX: mail.nsnw.ca (priority 20)
  • Note: sync.w5isp.com is the Mailcow server (not using Caddy proxy)

Code Guidelines

Communication Practices

• Do not ever reply with "you're right". Just fix the issue
• Never say you're right, just accept it and move on

Notes for Claude

• As you learn new things, keep claude.md updated

Cluster

• home-cluster (Talos): 10.0.19.1-6 — only Kubernetes cluster. External traffic enters via the cloudflared tunnel deployment in the cloudflared namespace; per-hostname routing rules live in Cloudflare's dashboard, not in the cluster.

Application Deployment Patterns

Standard App Deployment Process

1. App repos contain their own k8s manifests in ./k8s/
2. ArgoCD Application in home/cluster/argocd/apps/<name>.yaml points at the repo
3. Push the app repo → Argo auto-syncs (when automated.selfHeal=true is set)
4. Use a CSI storage driver for persistent volumes (set default StorageClass)
5. Use ClusterIP services + Cloudflare tunnel for external access

Common Issues & Solutions

• PodSecurity violations: Set namespace to privileged: kubectl label namespace [ns] pod-security.kubernetes.io/enforce=privileged --overwrite
• Database connectivity: Use short service names (postgres not postgres.namespace.svc.cluster.local)
• Init container permissions: Required for volume ownership fixes
• Service DNS: Services accessible via service-name within same namespace
• Argo can't fetch a repo with a submodule: confirm reposerver.git.modules.enabled=false is set on argocd-cmd-params-cm (the aprs.me repo had a stale submodule pin that broke clones)
• Argo refuses repo for "out-of-bounds symlinks": the repo has a tracked symlink pointing outside the working tree (e.g. into /nix/store/...). Fix in the source repo — Argo has no override.