infra/findings.md

Infra Code Audit Findings

Generated: 2026-06-04 Scope: tofu/ (OpenTofu) and talos/ (Talos Kubernetes cluster)

Severity Legend

• 🔴 CRITICAL — Security vulnerability or data exposure
• 🟠 HIGH — Configuration that will break or is actively broken
• 🟡 MEDIUM — Suboptimal config, drift, or potential future issue
• 🔵 LOW — Cleanup / best practice / suggestion

---

🔴 Security Issues

S-1: talos/kubeconfig committed to git

• File: talos/kubeconfig
• Severity: 🔴 CRITICAL
• Issue: Full admin kubeconfig for the home-cluster is tracked in git. Grants cluster admin access.
• Fix: Add to .gitignore, remove from git history (git rm --cached talos/kubeconfig), rotate certs if exposed publicly.

S-2: talos/secrets.yaml.backup tracked in git

• File: talos/secrets.yaml.backup
• Severity: 🔴 CRITICAL
• Issue: Backup of cluster secrets (contains CA keys, bootstrap tokens, service account keys) is tracked in git. The main secrets.yaml is correctly gitignored, but the backup isn't — .gitignore only has *.bak not *.backup.
• Fix: Add *.backup to .gitignore, git rm --cached talos/secrets.yaml.backup.

S-3: talos/talosconfig.backup tracked in git

• File: talos/talosconfig.backup
• Severity: 🔴 CRITICAL
• Issue: Backup of talosctl admin config is tracked in git (same .backup gitignore gap).
• Fix: Add *.backup to .gitignore, git rm --cached talos/talosconfig.backup.

S-4: Tailscale pre-auth keys in plaintext in CP configs

• Files: talos/configs/talos-cp1.yaml, talos-cp2.yaml, talos-cp3.yaml (TS_AUTHKEY line)
• Severity: 🔴 CRITICAL
• Issue: Tailscale pre-authentication keys are stored in plaintext in three config files. Also exposed in talos/configs/talos-worker2.yaml. These keys are reusable until revoked and grant Tailscale network access.
• Fix: Use sops or environment variable injection at apply time. Rotate the exposed keys in Tailscale admin console.

S-5: tofu/terraform.tfstate committed to git

• File: tofu/terraform.tfstate
• Severity: 🔴 CRITICAL
• Issue: Terraform state file (5,081 lines) committed to repo. Contains Cloudflare account IDs, zone IDs, internal K8s service DNS names (*.svc.cluster.local), tunnel ingress rules, and DKIM keys. OpenTofu has native state encryption (v1.7+) which is not being used.
• Fix: Add *.tfstate* to .gitignore, migrate to remote encrypted state backend, or use OpenTofu state encryption.

S-6: Private CA keys and cluster secrets exposed in per-node configs

• Files: All talos/configs/talos-*.yaml — every per-node config contains raw PKI private keys (machine CA, cluster CA, aggregator CA, service account, etcd CA)
• Severity: 🔴 CRITICAL
• Issue: Per-node configs contain the full cluster PKI private keys in base64. These are NOT in .gitignore. Anyone with repo access can decrypt and use them to impersonate cluster components.
• Fix: Either gitignore the per-node configs, or strip keys and inject at apply time via talosctl apply-config --patch.

---

🟠 Configuration Issues & Likely Bugs

C-1: Installer version drift — CPs on v1.12.4, some workers on v1.12.6

• Files:
  • talos/controlplane.yaml:80 → v1.12.4
  • talos/configs/talos-cp{1,2,3}.yaml:80 → v1.12.4
  • talos/configs/talos-worker{1,2,3}.yaml:78 → v1.12.4
  • talos/configs/talos-worker{4,5}.yaml:78 → v1.12.6
  • talos/CLAUDE.md:9 → "Talos Version: v1.12.6"
• Severity: 🟠 HIGH
• Issue: Configs reference a mix of v1.12.4 and v1.12.6 installer images. CLAUDE.md documents v1.12.6 as the cluster version. Per-node CP configs and workers 1-3 are on v1.12.4. Mixed versions across control plane and workers can cause compatibility issues.
• Fix: Update all references to v1.12.6 (whatever the current running version is).

C-2: grubUseUKICmdline + extraKernelArgs conflict on CPs

• Files:
  • talos/configs/talos-cp1.yaml:84-85 (and cp2, cp3)
  • talos/CLAUDE.md documents this explicitly: "install.extraKernelArgs and install.grubUseUKICmdline cannot be used together"
• Severity: 🟠 HIGH
• Issue: All three control plane per-node configs set BOTH grubUseUKICmdline: true AND extraKernelArgs: ["console=tty0"]. Talos v1.12 validates these as mutually exclusive and will reject the config. Since these are Proxmox VMs booting with UEFI (OVMF), they likely use systemd-boot which ignores extraKernelArgs anyway.
• Fix: Remove extraKernelArgs and bake kernel args into the Image Factory schematic if needed.

C-3: Tailscale extension won't work on control plane nodes

• Files: talos/configs/talos-cp{1,2,3}.yaml
  • install.image: ghcr.io/siderolabs/installer:v1.12.4 (stock installer)
  • install.extensions: [ghcr.io/siderolabs/tailscale:1.82.1]
  • ExtensionServiceConfig: configured for tailscale with TS_AUTHKEY
• Severity: 🟠 HIGH
• Issue: CP configs specify install.extensions but use the stock installer image (ghcr.io/siderolabs/installer). Extensions require a custom/factory installer image (like the worker configs use: factory.talos.dev/installer/<hash>). The stock installer does NOT support extensions. Result: tailscale extension binary is never installed, ExtensionServiceConfig for tailscale silently fails.
• Fix: Either switch CP nodes to use the factory installer image (bundling tailscale into the schematic), or remove the tailscale extension config from CP nodes.

C-4: Longhorn disk configs missing on workers 2 and 3 ✅ RESOLVED

• Files: talos/configs/talos-worker{1,2,3,4,5}.yaml
• Severity: 🟠 HIGH → ✅
• Issue: Longhorn has been removed from the cluster. All Longhorn disk configs (extraMounts, disks, installer image comments) removed from worker configs and documentation.

C-5: All per-node worker configs lack /dev/sdb disk config ✅ RESOLVED

• Same as C-4 — all Longhorn disk references removed.

C-6: Suspect firewall expression (not http.user_agent ne "")

• File: tofu/cloudflare.tf:437
• Severity: 🟠 HIGH
• Issue: The expression (not http.user_agent ne "") uses confusing double negation. Cloudflare's WAF expression language may not handle this as intended — Cloudflare normalizes empty User-Agents by injecting defaults, making this condition effectively a no-op.
• Fix: Use (not http.user_agent exists) to block requests with missing User-Agent, or remove the condition entirely.

---

🟡 Configuration Drift & Suboptimal Settings

D-1: Base configs (controlplane.yaml, worker.yaml) are stale templates

• Files: talos/controlplane.yaml, talos/worker.yaml
• Severity: 🟡 MEDIUM
• Issue: These are the original talosctl gen config output with auto-generated hostnames. They are NOT applied to any node (per-node configs in configs/ are used instead). Yet they contain the same cluster secrets and must be kept in sync with any changes. They serve no deployment purpose and add maintenance burden.
• Fix: Either delete them (rely solely on per-node configs) or clearly mark them as templates with a big warning comment.

D-2: controlplane.yaml has no DNS nameservers configured

• File: talos/controlplane.yaml (network: {})
• Severity: 🟡 MEDIUM
• Issue: Base controlplane config has empty network config. worker.yaml has three explicit nameservers (9.9.9.9, 149.112.112.112, 1.1.1.1). CP nodes rely entirely on DHCP DNS with no fallback.
• Fix: Add the same nameservers to the CP template for consistency.

D-3: No nameservers in per-node configs

• Files: All talos/configs/talos-*.yaml
• Severity: 🟡 MEDIUM
• Issue: While worker.yaml template has nameservers, none of the per-node configs carry them. All nodes rely on DHCP DNS only.
• Fix: Add nameservers to per-node configs for DNS resilience.

D-4: Hostname auto-generation in base configs (potential footgun)

• Files: talos/controlplane.yaml (HostnameConfig: auto: stable), talos/worker.yaml (same)
• Severity: 🟡 MEDIUM
• Issue: The base configs use auto: stable which generates hostnames from machine IDs. If someone applies the base config to a new node thinking it's a template, they'll get an auto-generated hostname instead of a predictable one.
• Fix: Comment out the auto line and add a big WARNING comment.

D-5: Per-node worker configs lack time servers

• Files: talos/configs/talos-worker{1,2,3}.yaml
• Severity: 🟡 LOW
• Issue: Per-node CP configs have time.servers: [0/1/2.us.pool.ntp.org] but worker per-node configs do not. Workers fall back to default NTP.
• Fix: Add NTP servers to worker configs for consistency.

D-6: Kubernetes discovery registry disabled

• Files: All configs — cluster.discovery.registries.kubernetes.disabled: true
• Severity: 🟡 MEDIUM
• Issue: Kubernetes discovery is disabled, and the external service registry is empty {}. If the Talos discovery service is unavailable, new nodes can't discover existing cluster members and won't join.
• Fix: Enable Kubernetes discovery (disabled: false) so nodes can discover each other via the API server.

D-7: talos/CLAUDE.md and talos/README.md doc drift

• Files: talos/README.md vs talos/CLAUDE.md
• Severity: 🟡 MEDIUM
• Issue: README.md says 3 workers (10.0.15.4-6), CLAUDE.md says 4 workers (10.0.15.4-7 including bare-metal talos4). README.md says "Proxmox Ceph storage" for VMs but Ceph is NOT used — it's local-lvm. README.md upgrade example references v1.9.5 (far behind).
• Fix: Align README.md with actual cluster state. Remove Ceph references.

D-8: tofu/ provider versions behind

• Files: tofu/.terraform.lock.hcl
• Severity: 🟡 LOW
• Issue: Cloudflare provider pinned at v4.52.5 (v5.x available — breaking changes to review). Porkbun at v1.3.1 (v1.3.3 latest).
• Fix: Review changelogs and upgrade if safe.

D-9: Towerops tunnel uses .svc.cluster.local DNS

• File: tofu/cloudflare.tf:574-580
• Severity: 🟡 MEDIUM
• Issue: Tunnel ingress rules point to http://towerops.towerops.svc.cluster.local:4000 etc. cloudflared running on a VM at 204.110.191.200 must be able to resolve these cluster-internal DNS names, which requires special DNS forwarding setup.
• Fix: Verify DNS resolution works, or document the setup.

D-10: PodSecurity admission uses deprecated v1alpha1 API version

• Files: All CP configs — admissionControl[0].configuration.apiVersion: pod-security.admission.config.k8s.io/v1alpha1
• Severity: 🟡 LOW
• Issue: The v1alpha1 API version for PodSecurityConfiguration was deprecated in Kubernetes 1.33+. The cluster runs v1.35.0. Should use v1 or v1beta1.
• Fix: Update to pod-security.admission.config.k8s.io/v1.

D-11: machine.time, machine.disks, machine.network.nameservers use legacy fields

• Files: Various in talos/
• Severity: 🟡 LOW
• Issue: Talos v1.12 deprecates these fields in favor of multi-doc configs (TimeSyncConfig, UserVolumeConfig, ResolverConfig). Currently functional but will be removed in a future Talos version.
• Fix: Migrate to multi-doc configs when convenient.

---

🔵 Cleanup & Suggestions

E-1: Stale backup files committed to git

• Files:
  • tofu/terraform.tfstate.*.backup (3 files)
  • tofu/terraform.tfstate.backup
  • talos/configs/talos-worker1.yaml.bak
  • talos/secrets.yaml.backup
  • talos/talosconfig.backup
• Severity: 🔵 LOW
• Issue: Multiple stale backup files clutter the repo. .gitignore in talos/ has *.bak but not *.backup.
• Fix: Add *.backup, *.bak, *.tfstate* to gitignore both places. Clean up tracked files.

E-2: tofu/tfplan committed

• File: tofu/tfplan
• Severity: 🔵 LOW
• Issue: Binary plan file committed. Should be ephemeral/gitignored.
• Fix: Add tfplan to gitignore.

E-3: No .gitignore in tofu/

• File: (missing) tofu/.gitignore
• Severity: 🔵 LOW
• Issue: The tofu directory has no .gitignore. State files, .terraform/ provider cache, plan files, and IDE settings are tracked.
• Fix: Create tofu/.gitignore with .terraform/, *.tfstate*, tfplan, .envrc, .claude/.

E-4: tofu/.claude/ IDE settings tracked

• File: tofu/.claude/settings.local.json
• Severity: 🔵 LOW
• Issue: IDE/editor configuration committed to repo.
• Fix: Add .claude/ to gitignore.

E-5: Orphaned/unused worker configs (workers 4 and 5)

• Files: talos/configs/talos-worker4.yaml, talos/configs/talos-worker5.yaml
• Severity: 🔵 LOW
• Issue: Worker4 config has hostname: talos4 (filename doesn't match hostname). Worker5 hostname talos5 with no documented existence. Worker4 config uses bare-metal install disk pattern (/dev/sdb). Worker5 uses /dev/nvme0n1. No VMs or nodes documented for either.
• Fix: Either document the purpose of these nodes or remove the orphaned configs.

E-6: towerops-dev tunnel documented as "down"

• File: tofu/cloudflare.tf (comment around line 556)
• Severity: 🔵 LOW
• Issue: dev tunnel marked as "down" in comments but still has import + config resource. If the tunnel was deleted from Cloudflare, tofu apply will fail.
• Fix: Either remove the tunnel config or fix the tunnel and update the comment.

E-7: talos/.gitignore doesn't cover secrets effectively

• File: talos/.gitignore
• Severity: 🔵 LOW
• Issue: Currently only ignores secrets.yaml, talosconfig, *.tmp, *.bak. Missing: *.backup, kubeconfig, configs/*.yaml (which contain full cluster PKI).
• Fix: Add *.backup, kubeconfig to .gitignore. Consider gitignoring per-node configs and maintaining them outside the repo.

E-8: disableManifestsDirectory: true comment is misleading

• Files: All configs
• Severity: 🔵 LOW
• Issue: Comment says "configures the kubelet to get static pod manifests from the /etc/kubernetes/manifests directory" — but setting true DISABLES that directory. The comment describes the inverse behavior.
• Fix: Fix the comment to say "Disable static pod manifest directory".

E-9: talos/README.md has stale upgrade example

• File: talos/README.md line 47, 112
• Severity: 🔵 LOW
• Issue: Upgrade example shows v1.9.5 (cluster is on v1.12.x). Also mentions tofu deployment workflow referencing terraform/talos_vms.tf which was removed.
• Fix: Update examples to match current versions and directory structure.

E-10: tofu/ uses OpenTofu-specific features (worth documenting)

• File: tofu/dns.tf, tofu/cloudflare.tf
• Severity: 🔵 LOW (INFO)
• Issue: Uses import with for_each which is an OpenTofu v1.8+ feature with no Terraform equivalent. This config can't go back to Terraform.
• Fix: Document this constraint to prevent future migration attempts.

---

Summary

Severity	Count
🔴 CRITICAL	6
🟠 HIGH	4
🟡 MEDIUM	11
🔵 LOW	10
Total	31

Top 5 priorities:

1. Remove sensitive files from git (kubeconfig, secrets backup, talosconfig backup, per-node configs with PKI) — S-1, S-2, S-3, S-6
2. Rotate exposed Tailscale pre-auth keys — S-4
3. Fix CP configs: either remove tailscale extension or switch to factory installer — C-3
4. Fix grubUseUKICmdline + extraKernelArgs conflict on CPs — C-2
5. Resolve installer version drift across Talos configs (v1.12.4 → v1.12.6) — C-1

---

✅ Fixes Applied (2026-06-04)

Security

• S-1, S-2, S-3: Removed talos/kubeconfig, talos/secrets.yaml.backup, talos/talosconfig.backup from git tracking via git rm --cached
• S-5: Created tofu/.gitignore, removed tofu/terraform.tfstate from tracking
• Updated talos/.gitignore with *.backup, kubeconfig, secrets.yaml.backup, talosconfig.backup

Talos Config Fixes (background task)

• C-1: Installer version drift resolved (all configs → v1.12.6)
• C-2: grubUseUKICmdline + extraKernelArgs conflict resolved on CPs
• C-3: Tailscale extension removed from stock installer CP configs
• D-2/D-3: Nameservers added to controlplane.yaml and per-node configs
• D-4: HostnameConfig auto: stable removed from base templates
• D-5: Time servers added to all per-node worker configs
• D-7: README.md updated (worker count 4, local-lvm, versions corrected)
• E-5: Orphaned worker4/5 configs documented
• Root CLAUDE.md: Longhorn removed, worker count corrected, Ceph→local-lvm fixed

Longhorn Removal

• talos/worker.yaml: Removed active extraMounts and disks blocks
• talos/configs/talos-worker*.yaml: Removed commented-out Longhorn blocks
• All documentation cleaned up (CLAUDE.md, README.md, root CLAUDE.md)

Provider Upgrades

• Porkbun: v1.3.1 → v1.3.3 (via tofu init -upgrade)
• Cloudflare: v4.52.5 → v5.19.1 (full v5 migration):
  • main.tf: Bumped constraint to ~> 5.0
  • cloudflare_zone: Renamed zone → name, account_id → account = { id = ... }
  • cloudflare_record → cloudflare_dns_record (50+ resources + imports)
  • Replaced 4x cloudflare_zone_settings_override with 24x cloudflare_zone_setting
  • Converted ALL cloudflare_ruleset blocks → attribute syntax (rules = [...])
  • Converted DNS data {} blocks → attribute syntax (data = {})
  • Added ttl = 1 to proxied records (required in v5)
  • Tunnel config blocks → attribute syntax (config = { ingress = [...] })

Other Fixes

• C-6: Fixed suspect firewall expression (not http.user_agent ne "") → (not http.request.user_agent exists)
• E-1/2/3: Created tofu/.gitignore, cleaned up backup files and tfplan
• Root CLAUDE.md: Updated Talos notes (storage, worker count, Longhorn references)