diff --git a/CLAUDE.md b/CLAUDE.md index d5ed89a..2feeb21 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -76,19 +76,25 @@ kubectl get pods -n aprs kubectl get svc -n aprs ``` -### Talos Operations +### Talos Operations (Home Cluster) ```bash -# Apply configuration to nodes (from cluster directory) -talosctl apply-config --talosconfig=./talosconfig -e 10.0.101.21 --nodes 10.0.101.21 --file controlplane.yaml --config-patch @node1.patch.yaml --config-patch @tailscale.patch.yaml +# Change to talos directory +cd /Users/graham/dev/infra/talos -# Check node status -talosctl --talosconfig=./talosconfig -e 10.0.101.21 get nodes +# Check cluster health +talosctl --talosconfig talosconfig health -# Bootstrap cluster (only needed once) -talosctl --talosconfig=./talosconfig -e 10.0.101.21 bootstrap +# View cluster nodes +kubectl get nodes -# Get kubeconfig -talosctl --talosconfig=./talosconfig -e 10.0.101.21 kubeconfig +# View all pods +kubectl get pods -A + +# Check talos services +talosctl --talosconfig talosconfig service + +# View logs +talosctl --talosconfig talosconfig logs -f kubelet ``` @@ -162,6 +168,23 @@ tofu plan ## Recent Infrastructure Updates +### Talos Home Cluster (10.0.15.1-6) +- **Purpose**: Primary home lab Kubernetes cluster +- **Cluster Name**: home-cluster +- **Kubernetes Version**: v1.35.0 +- **Control Plane Nodes**: 3 (10.0.15.1-3) +- **Worker Nodes**: 3 (10.0.15.4-6) +- **API Endpoint**: https://10.0.15.1:6443 +- **Proxmox Cluster**: "home" (node1, node2, node3 at 10.0.15.101-103) +- **Storage**: Proxmox Ceph for VM disks +- **CNI**: Flannel +- **Deployment**: Managed via Terraform in `/terraform/` +- **Configuration**: `/talos/` directory + - `talosconfig` - Talosctl client config + - `secrets.yaml` - Cluster secrets + - `controlplane.yaml` / `worker.yaml` - Node configs +- **Documentation**: See `/talos/README.md` for detailed operations + ### K3s Cluster at 204.110.191.2 - **Purpose**: Home lab Kubernetes cluster - **Node1 IPs**: @@ -210,7 +233,8 @@ tofu plan - As you learn new things, keep claude.md updated ## Clusters -- k3s cluster at 204.110.191.2 +- **Talos Home Cluster**: 10.0.15.1-6 (primary home lab cluster) +- **K3s Cluster**: 204.110.191.2 (legacy, see above for details) ## Application Deployment Patterns diff --git a/talos/README.md b/talos/README.md new file mode 100644 index 0000000..fe27474 --- /dev/null +++ b/talos/README.md @@ -0,0 +1,146 @@ +# Talos Home Cluster + +## Cluster Information + +- **Cluster Name**: home-cluster +- **Kubernetes Version**: v1.35.0 +- **Control Plane Nodes**: 3 (10.0.15.1-3) +- **Worker Nodes**: 3 (10.0.15.4-6) +- **API Endpoint**: https://10.0.15.1:6443 + +## Node Details + +| Node Name | Role | IP | Proxmox Node | VMID | +|-----------|------|-----|--------------|------| +| talos-cp1 | Control Plane | 10.0.15.1 | node1 | 200 | +| talos-cp2 | Control Plane | 10.0.15.2 | node2 | 201 | +| talos-cp3 | Control Plane | 10.0.15.3 | node3 | 202 | +| talos-worker1 | Worker | 10.0.15.4 | node1 | 210 | +| talos-worker2 | Worker | 10.0.15.5 | node2 | 211 | +| talos-worker3 | Worker | 10.0.15.6 | node3 | 212 | + +## Configuration Files + +- `secrets.yaml` - Cluster secrets (keep secure!) +- `controlplane.yaml` - Control plane node configuration +- `worker.yaml` - Worker node configuration +- `talosconfig` - Talosctl client configuration + +## Using the Cluster + +### Talosctl Commands + +```bash +# Check cluster health +talosctl --talosconfig talosconfig health + +# Get node list +talosctl --talosconfig talosconfig get members + +# View logs from a node +talosctl --talosconfig talosconfig logs -f kubelet + +# Check service status +talosctl --talosconfig talosconfig service + +# Upgrade Talos (example) +talosctl --talosconfig talosconfig upgrade --image ghcr.io/siderolabs/installer:v1.9.5 +``` + +### Kubectl Commands + +The kubeconfig has been merged into your default kubectl config: + +```bash +# View nodes +kubectl get nodes + +# View all pods +kubectl get pods -A + +# View cluster info +kubectl cluster-info +``` + +## Storage Configuration + +This cluster uses Proxmox Ceph storage for VM disks. For Kubernetes persistent storage, you'll need to deploy a storage solution: + +### Option 1: Ceph CSI (Direct Proxmox Ceph) +See CLAUDE.md for Ceph CSI setup instructions to use Proxmox Ceph directly. + +### Option 2: Longhorn +```bash +kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/v1.7.2/deploy/longhorn.yaml +``` + +### Option 3: Rook-Ceph +Deploy a separate Ceph cluster inside Kubernetes using Rook. + +## Networking + +- **CNI**: Flannel (deployed by default) +- **Service CIDR**: Default Kubernetes range +- **Pod CIDR**: Default Kubernetes range + +## Maintenance + +### Adding a Node + +1. Clone VM from template in Terraform +2. Apply appropriate config: + ```bash + talosctl --talosconfig talosconfig apply-config --insecure --nodes --file controlplane.yaml + ``` + +### Removing a Node + +```bash +# Drain node first +kubectl drain --ignore-daemonsets --delete-emptydir-data + +# Remove from cluster +kubectl delete node + +# Reset Talos on the node +talosctl --talosconfig talosconfig reset --nodes +``` + +### Upgrading Kubernetes + +```bash +talosctl --talosconfig talosconfig upgrade-k8s --to 1.36.0 +``` + +## Troubleshooting + +### Node not joining cluster + +```bash +# Check node logs +talosctl --talosconfig talosconfig logs -f kubelet --nodes + +# Check etcd members +talosctl --talosconfig talosconfig etcd members +``` + +### Reset a Node + +```bash +talosctl --talosconfig talosconfig reset --nodes --graceful=false +``` + +### Reboot a Node + +```bash +talosctl --talosconfig talosconfig reboot --nodes +``` + +## Backup + +Important files to backup: +- `secrets.yaml` - Required to add nodes or recover cluster +- `talosconfig` - Required for talosctl access +- Kubeconfig is stored in `~/.kube/config` + +Store these securely in a password manager or encrypted backup. diff --git a/talos/controlplane.yaml b/talos/controlplane.yaml new file mode 100644 index 0000000..d5c0a16 --- /dev/null +++ b/talos/controlplane.yaml @@ -0,0 +1,399 @@ +version: v1alpha1 # Indicates the schema used to decode the contents. +debug: false # Enable verbose logging to the console. +persist: true +# Provides machine specific configuration options. +machine: + type: controlplane # Defines the role of the machine within the cluster. + token: 6mvqo9.nn5sbv4fhyifbqjk # The `token` is used by a machine to join the PKI of the cluster. + # The root certificate authority of the PKI. + ca: + crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJQakNCOGFBREFnRUNBaEJxdlBudGE0a2JsOVJoRXAyVmQzWjNNQVVHQXl0bGNEQVFNUTR3REFZRFZRUUsKRXdWMFlXeHZjekFlRncweU5qQXhNVEV4TnpJd01qbGFGdzB6TmpBeE1Ea3hOekl3TWpsYU1CQXhEakFNQmdOVgpCQW9UQlhSaGJHOXpNQ293QlFZREsyVndBeUVBWmlCdVZydmc5YzdocG1PNGpUTS9KZ05udEVEcE9CV2M5QzRNCkc2b0ZSTE9qWVRCZk1BNEdBMVVkRHdFQi93UUVBd0lDaERBZEJnTlZIU1VFRmpBVUJnZ3JCZ0VGQlFjREFRWUkKS3dZQkJRVUhBd0l3RHdZRFZSMFRBUUgvQkFVd0F3RUIvekFkQmdOVkhRNEVGZ1FVVkxWa3g4TE8zM2FNNXRNUApOazJnUGpOYUIwZ3dCUVlESzJWd0EwRUFIa1JQY0lBWnFLYUtDY0NTaXpiN3R6WXJ1U3JGUjNhM0RuN3JReTNOCmlpYkUxczVpRUMyMGdrdFFXSkJXSkxwRVpQa1F3Yml4aXB2ZTVUZE5EVFEyQ1E9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg== + key: LS0tLS1CRUdJTiBFRDI1NTE5IFBSSVZBVEUgS0VZLS0tLS0KTUM0Q0FRQXdCUVlESzJWd0JDSUVJRFl1WkVJcC9ONEhLNG9RcndlcnlvZUVYNkt4WFFzWUcyeDRXV3ZxTVM4eAotLS0tLUVORCBFRDI1NTE5IFBSSVZBVEUgS0VZLS0tLS0K + # Extra certificate subject alternative names for the machine's certificate. + certSANs: [] + # # Uncomment this to enable SANs. + # - 10.0.0.10 + # - 172.16.0.10 + # - 192.168.0.10 + + # Used to provide additional options to the kubelet. + kubelet: + image: ghcr.io/siderolabs/kubelet:v1.35.0 # The `image` field is an optional reference to an alternative kubelet image. + defaultRuntimeSeccompProfileEnabled: true # Enable container runtime default Seccomp profile. + disableManifestsDirectory: true # The `disableManifestsDirectory` field configures the kubelet to get static pod manifests from the /etc/kubernetes/manifests directory. + + # # The `ClusterDNS` field is an optional reference to an alternative kubelet clusterDNS ip list. + # clusterDNS: + # - 10.96.0.10 + # - 169.254.2.53 + + # # The `extraArgs` field is used to provide additional flags to the kubelet. + # extraArgs: + # key: value + + # # The `extraMounts` field is used to add additional mounts to the kubelet container. + # extraMounts: + # - destination: /var/lib/example # Destination is the absolute path where the mount will be placed in the container. + # type: bind # Type specifies the mount kind. + # source: /var/lib/example # Source specifies the source path of the mount. + # # Options are fstab style mount options. + # options: + # - bind + # - rshared + # - rw + + # # The `extraConfig` field is used to provide kubelet configuration overrides. + # extraConfig: + # serverTLSBootstrap: true + + # # The `KubeletCredentialProviderConfig` field is used to provide kubelet credential configuration. + # credentialProviderConfig: + # apiVersion: kubelet.config.k8s.io/v1 + # kind: CredentialProviderConfig + # providers: + # - apiVersion: credentialprovider.kubelet.k8s.io/v1 + # defaultCacheDuration: 12h + # matchImages: + # - '*.dkr.ecr.*.amazonaws.com' + # - '*.dkr.ecr.*.amazonaws.com.cn' + # - '*.dkr.ecr-fips.*.amazonaws.com' + # - '*.dkr.ecr.us-iso-east-1.c2s.ic.gov' + # - '*.dkr.ecr.us-isob-east-1.sc2s.sgov.gov' + # name: ecr-credential-provider + + # # The `nodeIP` field is used to configure `--node-ip` flag for the kubelet. + # nodeIP: + # # The `validSubnets` field configures the networks to pick kubelet node IP from. + # validSubnets: + # - 10.0.0.0/8 + # - '!10.0.0.3/32' + # - fdc7::/16 + # Provides machine specific network configuration options. + network: {} + # # Configures KubeSpan feature. + # kubespan: + # enabled: true # Enable the KubeSpan feature. + + # Used to provide instructions for installations. + install: + disk: /dev/sda # The disk used for installations. + image: ghcr.io/siderolabs/installer:v1.12.1 # Allows for supplying the image used to perform the installation. + wipe: false # Indicates if the installation disk should be wiped at installation time. + grubUseUKICmdline: true # Indicates if legacy GRUB bootloader should use kernel cmdline from the UKI instead of building it on the host. + + # # Look up disk using disk attributes like model, size, serial and others. + # diskSelector: + # size: 4GB # Disk size. + # model: WDC* # Disk model `/sys/block//device/model`. + # busPath: /pci0000:00/0000:00:17.0/ata1/host0/target0:0:0/0:0:0:0 # Disk bus path. + # Features describe individual Talos features that can be switched on or off. + features: + diskQuotaSupport: true # Enable XFS project quota support for EPHEMERAL partition and user disks. + # KubePrism - local proxy/load balancer on defined port that will distribute + kubePrism: + enabled: true # Enable KubePrism support - will start local load balancing proxy. + port: 7445 # KubePrism port. + # Configures host DNS caching resolver. + hostDNS: + enabled: true # Enable host DNS caching resolver. + forwardKubeDNSToHost: true # Use the host DNS resolver as upstream for Kubernetes CoreDNS pods. + + # # Configure Talos API access from Kubernetes pods. + # kubernetesTalosAPIAccess: + # enabled: true # Enable Talos API access from Kubernetes pods. + # # The list of Talos API roles which can be granted for access from Kubernetes pods. + # allowedRoles: + # - os:reader + # # The list of Kubernetes namespaces Talos API access is available from. + # allowedKubernetesNamespaces: + # - kube-system + # Configures the node labels for the machine. + nodeLabels: + node.kubernetes.io/exclude-from-external-load-balancers: "" + + # # Provides machine specific control plane configuration options. + + # # ControlPlane definition example. + # controlPlane: + # # Controller manager machine specific configuration options. + # controllerManager: + # disabled: false # Disable kube-controller-manager on the node. + # # Scheduler machine specific configuration options. + # scheduler: + # disabled: true # Disable kube-scheduler on the node. + + # # Used to provide static pod definitions to be run by the kubelet directly bypassing the kube-apiserver. + + # # nginx static pod. + # pods: + # - apiVersion: v1 + # kind: pod + # metadata: + # name: nginx + # spec: + # containers: + # - image: nginx + # name: nginx + + # # Allows the addition of user specified files. + + # # MachineFiles usage example. + # files: + # - content: '...' # The contents of the file. + # permissions: 0o666 # The file's permissions in octal. + # path: /tmp/file.txt # The path of the file. + # op: append # The operation to use + + # # The `env` field allows for the addition of environment variables. + + # # Environment variables definition examples. + # env: + # GRPC_GO_LOG_SEVERITY_LEVEL: info + # GRPC_GO_LOG_VERBOSITY_LEVEL: "99" + # https_proxy: http://SERVER:PORT/ + # env: + # GRPC_GO_LOG_SEVERITY_LEVEL: error + # https_proxy: https://USERNAME:PASSWORD@SERVER:PORT/ + # env: + # https_proxy: http://DOMAIN\USERNAME:PASSWORD@SERVER:PORT/ + + # # Used to configure the machine's sysctls. + + # # MachineSysctls usage example. + # sysctls: + # kernel.domainname: talos.dev + # net.ipv4.ip_forward: "0" + # net/ipv6/conf/eth0.100/disable_ipv6: "1" + + # # Used to configure the machine's sysfs. + + # # MachineSysfs usage example. + # sysfs: + # devices.system.cpu.cpu0.cpufreq.scaling_governor: performance + + # # Configures the udev system. + # udev: + # # List of udev rules to apply to the udev system + # rules: + # - SUBSYSTEM=="drm", KERNEL=="renderD*", GROUP="44", MODE="0660" + + # # Configures the logging system. + # logging: + # # Logging destination. + # destinations: + # - endpoint: tcp://1.2.3.4:12345 # Where to send logs. Supported protocols are "tcp" and "udp". + # format: json_lines # Logs format. + + # # Configures the kernel. + # kernel: + # # Kernel modules to load. + # modules: + # - name: btrfs # Module name. + + # # Configures the seccomp profiles for the machine. + # seccompProfiles: + # - name: audit.json # The `name` field is used to provide the file name of the seccomp profile. + # # The `value` field is used to provide the seccomp profile. + # value: + # defaultAction: SCMP_ACT_LOG + + # # Override (patch) settings in the default OCI runtime spec for CRI containers. + + # # override default open file limit + # baseRuntimeSpecOverrides: + # process: + # rlimits: + # - hard: 1024 + # soft: 1024 + # type: RLIMIT_NOFILE + + # # Configures the node annotations for the machine. + + # # node annotations example. + # nodeAnnotations: + # customer.io/rack: r13a25 + + # # Configures the node taints for the machine. Effect is optional. + + # # node taints example. + # nodeTaints: + # exampleTaint: exampleTaintValue:NoSchedule +# Provides cluster specific configuration options. +cluster: + id: ajFbAfx06GXDzj6aZKEZ7M47wpc_0y4n9f4_NjIpz-c= # Globally unique identifier for this cluster (base64 encoded random 32 bytes). + secret: eZtUgfkU6glNR7S7O6n5uQBHb/m0pkb3QD9OlSo9cDM= # Shared secret of cluster (base64 encoded random 32 bytes). + # Provides control plane specific configuration options. + controlPlane: + endpoint: https://10.0.15.1:6443 # Endpoint is the canonical controlplane endpoint, which can be an IP address or a DNS hostname. + clusterName: home-cluster # Configures the cluster's name. + # Provides cluster specific network configuration options. + network: + dnsDomain: cluster.local # The domain used by Kubernetes DNS. + # The pod subnet CIDR. + podSubnets: + - 10.244.0.0/16 + # The service subnet CIDR. + serviceSubnets: + - 10.96.0.0/12 + + # # The CNI used. + # cni: + # name: custom # Name of CNI to use. + # # URLs containing manifests to apply for the CNI. + # urls: + # - https://docs.projectcalico.org/archive/v3.20/manifests/canal.yaml + token: fkth11.nmhzlprifn9pe6jl # The [bootstrap token](https://kubernetes.io/docs/reference/access-authn-authz/bootstrap-tokens/) used to join the cluster. + secretboxEncryptionSecret: 83iFqNeoP3fTSwstgBRWidjicy//f71Ui83VVcqI6So= # A key used for the [encryption of secret data at rest](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/). + # The base64 encoded root certificate authority used by Kubernetes. + ca: + crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJpRENDQVMrZ0F3SUJBZ0lRSFp3TUhKd0ZyblVLTDMrNjltQ2JnVEFLQmdncWhrak9QUVFEQWpBVk1STXcKRVFZRFZRUUtFd3ByZFdKbGNtNWxkR1Z6TUI0WERUSTJNREV4TVRFM01qQXlPVm9YRFRNMk1ERXdPVEUzTWpBeQpPVm93RlRFVE1CRUdBMVVFQ2hNS2EzVmlaWEp1WlhSbGN6QlpNQk1HQnlxR1NNNDlBZ0VHQ0NxR1NNNDlBd0VICkEwSUFCSWxHL0ppOWdEOTh2WEJKZjVWQ05VbkhuTGlIN1F5UWdMbHR0TFgrUElUeWFUaXVzSW5RRGMvUzZ4aysKTFVkSjhrN0t3d0NYUEZ0SDZTTWM0VE12MTZpallUQmZNQTRHQTFVZER3RUIvd1FFQXdJQ2hEQWRCZ05WSFNVRQpGakFVQmdnckJnRUZCUWNEQVFZSUt3WUJCUVVIQXdJd0R3WURWUjBUQVFIL0JBVXdBd0VCL3pBZEJnTlZIUTRFCkZnUVVyRU5ZMllVN3pWb1hoZ1BCb2Rqa1Z2Q2czWWd3Q2dZSUtvWkl6ajBFQXdJRFJ3QXdSQUlnV2xxb0ZXTzYKa3RnNEt3NVBtUkhUeVNuUi90cmQzTmQyc0x6WHdCRmFZcUlDSUNobFpCWDBHRGpQTVFjM0xQcG1oR2xpMVpCQgpzWkdyY0M4VVpqSjNqR0t4Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K + key: LS0tLS1CRUdJTiBFQyBQUklWQVRFIEtFWS0tLS0tCk1IY0NBUUVFSUlQYklVSWtZWnRVaDI4Z2dqZERsbmp3bUVUWDJ0eEtpV2I2ek1xMTA2aDRvQW9HQ0NxR1NNNDkKQXdFSG9VUURRZ0FFaVViOG1MMkFQM3k5Y0VsL2xVSTFTY2VjdUlmdERKQ0F1VzIwdGY0OGhQSnBPSzZ3aWRBTgp6OUxyR1Q0dFIwbnlUc3JEQUpjOFcwZnBJeHpoTXkvWHFBPT0KLS0tLS1FTkQgRUMgUFJJVkFURSBLRVktLS0tLQo= + # The base64 encoded aggregator certificate authority used by Kubernetes for front-proxy certificate generation. + aggregatorCA: + crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJYekNDQVFXZ0F3SUJBZ0lRREFKZDVPVGdJbnFnZ0NnMUUrNmFaVEFLQmdncWhrak9QUVFEQWpBQU1CNFgKRFRJMk1ERXhNVEUzTWpBeU9Wb1hEVE0yTURFd09URTNNakF5T1Zvd0FEQlpNQk1HQnlxR1NNNDlBZ0VHQ0NxRwpTTTQ5QXdFSEEwSUFCTi9oNms5MEMzT0VLOWlLY3daZEFvWFVXNDBXY0JFWnNEYnRxamMwWkI5cUxseHpLMkdYCnZZZHVsODJ4c1JiTjlXVjFoS085NjZCc2w5TXF2ZDAvOXJHallUQmZNQTRHQTFVZER3RUIvd1FFQXdJQ2hEQWQKQmdOVkhTVUVGakFVQmdnckJnRUZCUWNEQVFZSUt3WUJCUVVIQXdJd0R3WURWUjBUQVFIL0JBVXdBd0VCL3pBZApCZ05WSFE0RUZnUVVpTGtETGhmcVQ5OUJ4VVBUVFBOa2tyUG1xRkV3Q2dZSUtvWkl6ajBFQXdJRFNBQXdSUUloCkFQOXJxVHhXeEFWRXA2MElnNkgxSDJNaVpxc20waXlTSWFGNktHSDhGRDB5QWlCWkhWRml1SzFvKzZUWEt6QzkKSnNjait3emhuSTZWYUNQU2pEUUt3NGFrYkE9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg== + key: LS0tLS1CRUdJTiBFQyBQUklWQVRFIEtFWS0tLS0tCk1IY0NBUUVFSUpTYXNCall3ZDAzcjZxRUFJaXNCTU9lWm5Dd3BRcm54cnE5T1JNZnFkT01vQW9HQ0NxR1NNNDkKQXdFSG9VUURRZ0FFMytIcVQzUUxjNFFyMklwekJsMENoZFJialJad0VSbXdOdTJxTnpSa0gyb3VYSE1yWVplOQpoMjZYemJHeEZzMzFaWFdFbzczcm9HeVgweXE5M1QvMnNRPT0KLS0tLS1FTkQgRUMgUFJJVkFURSBLRVktLS0tLQo= + # The base64 encoded private key for service account token generation. + serviceAccount: + key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlKS2dJQkFBS0NBZ0VBOXRQbWRWMDFYTWNXenAzc3RXS3p3NHFDcW9kZDVjaXVQanF2MXQ5VkcxbHRQNEJTCmlVS3U3VERTS3R4Zm1qQ3pMLzVvVHVwMUVPbDRHckVOK0RZNDhvZjN1MU1VdDdjWE9lelF5SXBLaitDMUJyRTEKQ29pNTFOUHdXNGpUaHRPTExXUmRrU2doUCtPaDM0alpzelBvczVvS1E4aFJtK1hZNzRjbytPZ3lOL2NORXQzNQowa3h1aVEyUzZPam51Z0tIZk9zOFhRL2d0cDloSitUYW0zdkpTSHdOUFpOVkFHb3dYSmQrRU5PeEhMWjlLSDIrCmNOcmpNZGUvbmZza3lvV2h1NXZFZnQxTmV0ZkEwWXZSb1hzZERqdS82YXJ5NW92MWNoMDJYVENhc0tLN05RajAKT1Y2REVqZm5PTlNRRndXRUpTVzY4RVlWY1AxSCtoY3E4VkozODM3b0JyWVpUWUNFZllMaC8rNnpZY3RjSGdLTAo5VFhvdkxkTlZLaDlpZ0Q5NVlsT3lsTmpCVlVDNm45Q0RPQ3BZWDUzaXdiTGl1VFVsRUNwK3g0eCtrcVlPUFNDCmFrWVhwZXZ2R1F0STFUTDFLK2lDamNXUm9EaVZoblFWREhsV243R3pqeFNKMEJWd25ydFk3aWlXR1JmemxMTVUKQzUrVVY3TGxSaytwMFlDMG85dmczbXZQNkJRTm5NQUtETGc5MG5PSjVzZWFkR21DQVdyVnI2TjhhaDR2ZmpGdQprUkVVa29vcjZmYXlERmxpN1pQWGYzaXVLY0FiNXhXb3JoTXVvR0tzMUFRZHdLZFdUZi9janVKOVJYR2lLSXJ2ClJWQWQ4TW82MGhKd25SRElCK1M0cUNOWXlvYXQxd2x4NldCcjQyRUZ2ekFxdk41Z0thOVBQR09KOEJjQ0F3RUEKQVFLQ0FnQml4dXc4WFovRVVEdzFWWWZzRXVManh5T0JnUWkzczNiZm1uYlNJN0FZY05KQ2ptVDVYMEZIMUxqVwp4RGhTS1ZrY1JOQ1RmRXA3Q1NPeWN0YW5WWE8ydzJrRmd6c1FSbW82R1NaNGlyZEtHdC8zSEtXZ1lKVVZzZmRICmVsdEVZcFlaQlVoWWxkbXdpclUyRXJza2R3cVNEWHc4SWdkZXZ6M3dwWU5sRU84LzdGTkVFS2N3WEx3RGh2WWEKMVZJZTQzSTdNSnlJbFprVkhIVXlGT2pUUGxlUDlXMExEbEgvQjMveEhnTXFVcjFYUTBTVkZyVkd6cUlMU1U3Ngp0dzNES25WTjhnbFJ4OWlBNTZUWGR2L2VncmxlLzlST2U4WVhNejEzZkc5TUhlNVo3OVhCWktMdjduMi9wYldYCnphRVlSWEhidDl5MjJTVXRnWDd6ZTZBSTBUc0U0clIrY0RCZndwcm9ZSHpSRUdlb2t4a0JucHRSNG1YNkNyYm8KYUxZdTJiTWpzd0dCclE5VEJiTk9vNjVkOExRV3JlaGNFY0ZSbWxXN2JCT2M5N3FVbUIvakZGSXhQMUk5NjY5WgpveXpuZ2tzcHZISHZXR2NVTTRsMVkxYkhXNCtOUFpaUk9uOFc4dGJKT2hTNjZ2YTdFQ3pnb2QzRC9aSmFmdlViCkRjTlNCdVltZTZsUisvU1Z6SEcvOTROQVZJdmdPdVVxVWxqbmlpNnRtUFBqUEduNXlNTmZ4QUN4aEJlQmZJNCsKbitqc2V4UGtHamhrSjdPVDVKQjYxQWFId28wdjJ3Y212a0I1YkE3cEo3K0k4OUp1bE9VM3V2eXQ2OUQ2b1k0agpEQk00ZVVVd3ArZ3c1ZHVMYWpES2w0cno5SHNQQllPSUpDNE9uUE5wdXRlb2tzWWEyUUtDQVFFQS82YlpGYkI3CkZ2dGdxYkozV0ZEL1BjcjFwbnljakI4ZFZObVNBN0FhVHZaZll5THovbHlJM0VUQkJUbkhWUk9YVy9qa2VYcGcKa1FpRGRLbzlZS2g3Y1EwdTBwVnVDeGNOdXRkRmIwRkROMERvWjNuRlJlSkpEcmh2c2gyRDkvMGdaTmw2Tkg0dApPMUxWUDBGUStpeTZ5TGlUci9XelVZYVc1OHVNcmw2cFUxRStQTjJPQ1l4bnBmVk0vbFJQbHdJUFF5ZUYrc3krCkZibjBnVTArOEJHY0R6aDJZZldBZ1J4OUt5RStNcWtROVdIQzY2MXZkUGFPdm5ROTY3Y2xTRE5kU3FiRFRuU3EKSlExbVdLT0c3RzJXRGtDcmw0RElZU0tQQUZCeWowbGcxeCtjQ1JoN1hZOUtDVnZxMmFkcXJOZ1A4UHZmblFpdwpGcThuSWE1UnNhM3Y5UUtDQVFFQTl5bjVuNW5rQUd4Z2JlSEF4MnFMbWFBNHRvUGVHUkxHeUdCWU9Wa3RtSFN5Ck4rS2tvamw4ZmEyN0ppTVVNWHRONzhsSzdNM3lGQldSWENocUgvaURZYmtVZFNLUk4yTFJBRWtudTkxTWVVTWkKV1pmMVo5QzNpL2JlUmV4aEl0aGFNQkk2YldQeTMzTExYREdsZXlmTnp6N21nNFVOQm5relp6aUMwRjN0NUlMcgp5WG1VdDZxUjNPQlBIcUhDZXhTeERqQVk4ZDBXTjhQejk0YUJkNW1pelBEQ1FwYkFuTmZ4NFVGU0Z5N2hRRGVrCkFQUm5MY0tacXZibGJzT3JlUUo3TUJXV1h6U3VwYnUxL2krYTNwZWNwMTRmek9HMC9GaDh0c1JQTUtHNFNDcDYKQ2lOVm55T210MGRHaWxaOXV4emRVZVF6RXBSZk93YkF6UlNqYW95VVd3S0NBUUVBMDBLQWovRFh0dVVpT3ZsVQpzREcyeXN5R2RQN3lxc1FuaSsrRFB2THV1cVBUcld1NzcwMXVkMytGZGhrS0JONzhyQzhnTnRhUEtDcTkzdEd6CmlNV1J4SmxkUVAzbEF0c1pLSFl1VmNLN2xkUlllUEhpdUY1SGV4ZWtqYTdnV0RUZjZSYmlhckIrRmxEY0VicWEKL1RHclpLVkt2aFN2QzBiS08yQTByLzlmczQ4bWZFdWx6WFF1VHJGSDM3K3lTWnFWSEN4MEJnM2RZcnpaTldTdApCdHUyZDhsejRuMXFwTWJ6RklvVENQQTNLRitTSVFtZ1dpcEpTVTlydi9jSG93TjNLM0psWU5iOXAwTE1lYUVnCnBRM2wrMzhyalRZSjRHTWJndW4zMFRxVkFSQ0k4K2M4UXROZkhoN01FdkxMTDJ3WXVFNmJ1ZnZzNzhIcjkyUkMKOUM1RlpRS0NBUUVBb2pxaTVlY1VpRkthV0lVOUM2Y2JTaEtvSnVRVyswNGZ1NVhCVXFMK0VudEdIdmhjSno2awpQN213aFMreEtXbE5sbjRBOTcyVmp5azQ1OHgvRzh6NkwwSlQ4eTVsTW4xaHdWWkRCa1FTMWVVQ3Z0dkc4QVZHCjlFOXRSV2dKdWQxWUZRVmg1RHNlUnh3Sjd0R0hWSGFqMTh2cGNabjdHcXlpMkxWMjUxMC9zQ1ZlcmpkbkxjWi8KUndMVDY3OHc3Nmg5TGxyVGhLMGcwQ21HRUlud29KZnZDNlZiWEdtbXMwdnFMM2U0Y2c4TE9pTkxPblV3NUJBKwozTUNCaDFwWFNtNGVmVWt1RWY5NlFyNGFqbVNndWttNVFRRyt6Yk80SDRuVkVnRjRHT3VORVpFUDJXZW40M1I3CkVRRnZnNDJScVNjZy9ZZFRIbmJVcXVHYjJ3QVk0dTZ5MFFLQ0FRRUFpQm0zbkUxTGg2VFFtZjlZQVVnV1pNeVkKQWlVSHNFUnRTUGdCdWgrZzUxY1krMmp2ckEvbVZ3bG5PQlphWmoreU52Tk5ESktoWUk3TDlFb2krbTlZM1duUAp4OGFlRndhNFBRMzhUU09wM0hSYXhtcUd2Ylo0aktqcVVSc3R3b1B2UzkyK2JGZUxyaFArREh2dmhOMW41TDNkCjZtdFQycjVpSkZWWEc1RVZWMGpUdkVMUVBDT013c0NVL2hyOVdCY0hpeFdIdzBGOGdZaEo2VjJ6WjB2ZEhTUHYKMTNhUWFHZlh4M0o2NEN4dy9CWmJPaUdnYUNOOEtBKzJhYmhLY2M1UTR5L21QRUN1WXRlVHJPMFZURU9GOHczMgpta29SOUUxWHI2RnowK1Z1b0puKzBXSWNkN0l5dS8ybHZzWjVPc2ltMVRkUS9VaFAvVjlsMHpweGhOYlJUQT09Ci0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg== + # API server specific configuration options. + apiServer: + image: registry.k8s.io/kube-apiserver:v1.35.0 # The container image used in the API server manifest. + # Configure the API server admission plugins. + admissionControl: + - name: PodSecurity # Name is the name of the admission controller. + # Configuration is an embedded configuration object to be used as the plugin's + configuration: + apiVersion: pod-security.admission.config.k8s.io/v1alpha1 + defaults: + audit: restricted + audit-version: latest + enforce: baseline + enforce-version: latest + warn: restricted + warn-version: latest + exemptions: + namespaces: + - kube-system + runtimeClasses: [] + usernames: [] + kind: PodSecurityConfiguration + # Configure the API server audit policy. + auditPolicy: + apiVersion: audit.k8s.io/v1 + kind: Policy + rules: + - level: Metadata + + # # Configure the API server authorization config. Node and RBAC authorizers are always added irrespective of the configuration. + # authorizationConfig: + # - type: Webhook # Type is the name of the authorizer. Allowed values are `Node`, `RBAC`, and `Webhook`. + # name: webhook # Name is used to describe the authorizer. + # # webhook is the configuration for the webhook authorizer. + # webhook: + # connectionInfo: + # type: InClusterConfig + # failurePolicy: Deny + # matchConditionSubjectAccessReviewVersion: v1 + # matchConditions: + # - expression: has(request.resourceAttributes) + # - expression: '!(\''system:serviceaccounts:kube-system\'' in request.groups)' + # subjectAccessReviewVersion: v1 + # timeout: 3s + # - type: Webhook # Type is the name of the authorizer. Allowed values are `Node`, `RBAC`, and `Webhook`. + # name: in-cluster-authorizer # Name is used to describe the authorizer. + # # webhook is the configuration for the webhook authorizer. + # webhook: + # connectionInfo: + # type: InClusterConfig + # failurePolicy: NoOpinion + # matchConditionSubjectAccessReviewVersion: v1 + # subjectAccessReviewVersion: v1 + # timeout: 3s + # Controller manager server specific configuration options. + controllerManager: + image: registry.k8s.io/kube-controller-manager:v1.35.0 # The container image used in the controller manager manifest. + # Kube-proxy server-specific configuration options + proxy: + image: registry.k8s.io/kube-proxy:v1.35.0 # The container image used in the kube-proxy manifest. + + # # Disable kube-proxy deployment on cluster bootstrap. + # disabled: false + # Scheduler server specific configuration options. + scheduler: + image: registry.k8s.io/kube-scheduler:v1.35.0 # The container image used in the scheduler manifest. + # Configures cluster member discovery. + discovery: + enabled: true # Enable the cluster membership discovery feature. + # Configure registries used for cluster member discovery. + registries: + # Kubernetes registry uses Kubernetes API server to discover cluster members and stores additional information + kubernetes: + disabled: true # Disable Kubernetes discovery registry. + # Service registry is using an external service to push and pull information about cluster members. + service: {} + # # External service endpoint. + # endpoint: https://discovery.talos.dev/ + # Etcd specific configuration options. + etcd: + # The `ca` is the root certificate authority of the PKI. + ca: + crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJmRENDQVNPZ0F3SUJBZ0lRRFV2eGpQbDViSFRSR05KN1VSQ0JzVEFLQmdncWhrak9QUVFEQWpBUE1RMHcKQ3dZRFZRUUtFd1JsZEdOa01CNFhEVEkyTURFeE1URTNNakF5T1ZvWERUTTJNREV3T1RFM01qQXlPVm93RHpFTgpNQXNHQTFVRUNoTUVaWFJqWkRCWk1CTUdCeXFHU000OUFnRUdDQ3FHU000OUF3RUhBMElBQkpWSlhnSGRMbTJFCjFSNmJyZjJ2aG91blBndlpzeVlsZFA5TlMzT0FXaGk1dUczc2JBTi96ZnF1VEtpOUFjQXpuOWNnbHY5MkI3MXkKQXNtMktPcUJXYTJqWVRCZk1BNEdBMVVkRHdFQi93UUVBd0lDaERBZEJnTlZIU1VFRmpBVUJnZ3JCZ0VGQlFjRApBUVlJS3dZQkJRVUhBd0l3RHdZRFZSMFRBUUgvQkFVd0F3RUIvekFkQmdOVkhRNEVGZ1FVOGpCellJc3RITjA2CjFNSDBsNjFRaXlibnFIa3dDZ1lJS29aSXpqMEVBd0lEUndBd1JBSWdWbDZ0SWw5MlN0YUtKWDhkVnpzKzJiWUgKQi9sTnlPbkx1eGNZTWsvL1lGTUNJQXZWT1ZMckQ4Q3hCMlgyMDREVVpOL0t6KzVZcHFtRy9yUTFiRnBDSkF1NwotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg== + key: LS0tLS1CRUdJTiBFQyBQUklWQVRFIEtFWS0tLS0tCk1IY0NBUUVFSUVKaktuempHY1ViakNRekpkamF0cFNXUVlacnI2MHg4b1NNeEV3eUhkNUFvQW9HQ0NxR1NNNDkKQXdFSG9VUURRZ0FFbFVsZUFkMHViWVRWSHB1dC9hK0dpNmMrQzltekppVjAvMDFMYzRCYUdMbTRiZXhzQTMvTgorcTVNcUwwQndET2YxeUNXLzNZSHZYSUN5YllvNm9GWnJRPT0KLS0tLS1FTkQgRUMgUFJJVkFURSBLRVktLS0tLQo= + + # # The container image used to create the etcd service. + # image: registry.k8s.io/etcd:v3.6.7 + + # # The `advertisedSubnets` field configures the networks to pick etcd advertised IP from. + # advertisedSubnets: + # - 10.0.0.0/8 + # A list of urls that point to additional manifests. + extraManifests: [] + # - https://www.example.com/manifest1.yaml + # - https://www.example.com/manifest2.yaml + + # A list of inline Kubernetes manifests. + inlineManifests: [] + # - name: namespace-ci # Name of the manifest. + # contents: |- # Manifest contents as a string. + # apiVersion: v1 + # kind: Namespace + # metadata: + # name: ci + + + # # A key used for the [encryption of secret data at rest](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/). + + # # Decryption secret example (do not use in production!). + # aescbcEncryptionSecret: z01mye6j16bspJYtTB/5SFX8j7Ph4JXxM2Xuu4vsBPM= + + # # Core DNS specific configuration options. + # coreDNS: + # image: registry.k8s.io/coredns/coredns:v1.13.2 # The `image` field is an override to the default coredns image. + + # # External cloud provider configuration. + # externalCloudProvider: + # enabled: true # Enable external cloud provider. + # # A list of urls that point to additional manifests for an external cloud provider. + # manifests: + # - https://raw.githubusercontent.com/kubernetes/cloud-provider-aws/v1.20.0-alpha.0/manifests/rbac.yaml + # - https://raw.githubusercontent.com/kubernetes/cloud-provider-aws/v1.20.0-alpha.0/manifests/aws-cloud-controller-manager-daemonset.yaml + + # # A map of key value pairs that will be added while fetching the extraManifests. + # extraManifestHeaders: + # Token: "1234567" + # X-ExtraInfo: info + + # # Settings for admin kubeconfig generation. + # adminKubeconfig: + # certLifetime: 1h0m0s # Admin kubeconfig certificate lifetime (default is 1 year). + + # # Allows running workload on control-plane nodes. + # allowSchedulingOnControlPlanes: true +--- +apiVersion: v1alpha1 +kind: HostnameConfig +auto: stable # A method to automatically generate a hostname for the machine. + +# # A static hostname to set for the machine. +# hostname: controlplane1 +# hostname: controlplane1.example.org diff --git a/talos/worker.yaml b/talos/worker.yaml new file mode 100644 index 0000000..063a401 --- /dev/null +++ b/talos/worker.yaml @@ -0,0 +1,433 @@ +version: v1alpha1 # Indicates the schema used to decode the contents. +debug: false # Enable verbose logging to the console. +persist: true +# Provides machine specific configuration options. +machine: + type: worker # Defines the role of the machine within the cluster. + token: 6mvqo9.nn5sbv4fhyifbqjk # The `token` is used by a machine to join the PKI of the cluster. + # The root certificate authority of the PKI. + ca: + crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJQakNCOGFBREFnRUNBaEJxdlBudGE0a2JsOVJoRXAyVmQzWjNNQVVHQXl0bGNEQVFNUTR3REFZRFZRUUsKRXdWMFlXeHZjekFlRncweU5qQXhNVEV4TnpJd01qbGFGdzB6TmpBeE1Ea3hOekl3TWpsYU1CQXhEakFNQmdOVgpCQW9UQlhSaGJHOXpNQ293QlFZREsyVndBeUVBWmlCdVZydmc5YzdocG1PNGpUTS9KZ05udEVEcE9CV2M5QzRNCkc2b0ZSTE9qWVRCZk1BNEdBMVVkRHdFQi93UUVBd0lDaERBZEJnTlZIU1VFRmpBVUJnZ3JCZ0VGQlFjREFRWUkKS3dZQkJRVUhBd0l3RHdZRFZSMFRBUUgvQkFVd0F3RUIvekFkQmdOVkhRNEVGZ1FVVkxWa3g4TE8zM2FNNXRNUApOazJnUGpOYUIwZ3dCUVlESzJWd0EwRUFIa1JQY0lBWnFLYUtDY0NTaXpiN3R6WXJ1U3JGUjNhM0RuN3JReTNOCmlpYkUxczVpRUMyMGdrdFFXSkJXSkxwRVpQa1F3Yml4aXB2ZTVUZE5EVFEyQ1E9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg== + key: "" + # Extra certificate subject alternative names for the machine's certificate. + certSANs: [] + # # Uncomment this to enable SANs. + # - 10.0.0.10 + # - 172.16.0.10 + # - 192.168.0.10 + + # Used to provide additional options to the kubelet. + kubelet: + image: ghcr.io/siderolabs/kubelet:v1.35.0 # The `image` field is an optional reference to an alternative kubelet image. + defaultRuntimeSeccompProfileEnabled: true # Enable container runtime default Seccomp profile. + disableManifestsDirectory: true # The `disableManifestsDirectory` field configures the kubelet to get static pod manifests from the /etc/kubernetes/manifests directory. + + # # The `ClusterDNS` field is an optional reference to an alternative kubelet clusterDNS ip list. + # clusterDNS: + # - 10.96.0.10 + # - 169.254.2.53 + + # # The `extraArgs` field is used to provide additional flags to the kubelet. + # extraArgs: + # key: value + + # # The `extraMounts` field is used to add additional mounts to the kubelet container. + # extraMounts: + # - destination: /var/lib/example # Destination is the absolute path where the mount will be placed in the container. + # type: bind # Type specifies the mount kind. + # source: /var/lib/example # Source specifies the source path of the mount. + # # Options are fstab style mount options. + # options: + # - bind + # - rshared + # - rw + + # # The `extraConfig` field is used to provide kubelet configuration overrides. + # extraConfig: + # serverTLSBootstrap: true + + # # The `KubeletCredentialProviderConfig` field is used to provide kubelet credential configuration. + # credentialProviderConfig: + # apiVersion: kubelet.config.k8s.io/v1 + # kind: CredentialProviderConfig + # providers: + # - apiVersion: credentialprovider.kubelet.k8s.io/v1 + # defaultCacheDuration: 12h + # matchImages: + # - '*.dkr.ecr.*.amazonaws.com' + # - '*.dkr.ecr.*.amazonaws.com.cn' + # - '*.dkr.ecr-fips.*.amazonaws.com' + # - '*.dkr.ecr.us-iso-east-1.c2s.ic.gov' + # - '*.dkr.ecr.us-isob-east-1.sc2s.sgov.gov' + # name: ecr-credential-provider + + # # The `nodeIP` field is used to configure `--node-ip` flag for the kubelet. + # nodeIP: + # # The `validSubnets` field configures the networks to pick kubelet node IP from. + # validSubnets: + # - 10.0.0.0/8 + # - '!10.0.0.3/32' + # - fdc7::/16 + # Provides machine specific network configuration options. + network: {} + # # Configures KubeSpan feature. + # kubespan: + # enabled: true # Enable the KubeSpan feature. + + # Used to provide instructions for installations. + install: + disk: /dev/sda # The disk used for installations. + image: ghcr.io/siderolabs/installer:v1.12.1 # Allows for supplying the image used to perform the installation. + wipe: false # Indicates if the installation disk should be wiped at installation time. + grubUseUKICmdline: true # Indicates if legacy GRUB bootloader should use kernel cmdline from the UKI instead of building it on the host. + + # # Look up disk using disk attributes like model, size, serial and others. + # diskSelector: + # size: 4GB # Disk size. + # model: WDC* # Disk model `/sys/block//device/model`. + # busPath: /pci0000:00/0000:00:17.0/ata1/host0/target0:0:0/0:0:0:0 # Disk bus path. + registries: {} + # Features describe individual Talos features that can be switched on or off. + features: + diskQuotaSupport: true # Enable XFS project quota support for EPHEMERAL partition and user disks. + # KubePrism - local proxy/load balancer on defined port that will distribute + kubePrism: + enabled: true # Enable KubePrism support - will start local load balancing proxy. + port: 7445 # KubePrism port. + # Configures host DNS caching resolver. + hostDNS: + enabled: true # Enable host DNS caching resolver. + forwardKubeDNSToHost: true # Use the host DNS resolver as upstream for Kubernetes CoreDNS pods. + + # # Configure Talos API access from Kubernetes pods. + # kubernetesTalosAPIAccess: + # enabled: true # Enable Talos API access from Kubernetes pods. + # # The list of Talos API roles which can be granted for access from Kubernetes pods. + # allowedRoles: + # - os:reader + # # The list of Kubernetes namespaces Talos API access is available from. + # allowedKubernetesNamespaces: + # - kube-system + + # # Provides machine specific control plane configuration options. + + # # ControlPlane definition example. + # controlPlane: + # # Controller manager machine specific configuration options. + # controllerManager: + # disabled: false # Disable kube-controller-manager on the node. + # # Scheduler machine specific configuration options. + # scheduler: + # disabled: true # Disable kube-scheduler on the node. + + # # Used to provide static pod definitions to be run by the kubelet directly bypassing the kube-apiserver. + + # # nginx static pod. + # pods: + # - apiVersion: v1 + # kind: pod + # metadata: + # name: nginx + # spec: + # containers: + # - image: nginx + # name: nginx + + # # Allows the addition of user specified files. + + # # MachineFiles usage example. + # files: + # - content: '...' # The contents of the file. + # permissions: 0o666 # The file's permissions in octal. + # path: /tmp/file.txt # The path of the file. + # op: append # The operation to use + + # # The `env` field allows for the addition of environment variables. + + # # Environment variables definition examples. + # env: + # GRPC_GO_LOG_SEVERITY_LEVEL: info + # GRPC_GO_LOG_VERBOSITY_LEVEL: "99" + # https_proxy: http://SERVER:PORT/ + # env: + # GRPC_GO_LOG_SEVERITY_LEVEL: error + # https_proxy: https://USERNAME:PASSWORD@SERVER:PORT/ + # env: + # https_proxy: http://DOMAIN\USERNAME:PASSWORD@SERVER:PORT/ + + # # Used to configure the machine's sysctls. + + # # MachineSysctls usage example. + # sysctls: + # kernel.domainname: talos.dev + # net.ipv4.ip_forward: "0" + # net/ipv6/conf/eth0.100/disable_ipv6: "1" + + # # Used to configure the machine's sysfs. + + # # MachineSysfs usage example. + # sysfs: + # devices.system.cpu.cpu0.cpufreq.scaling_governor: performance + + # # Configures the udev system. + # udev: + # # List of udev rules to apply to the udev system + # rules: + # - SUBSYSTEM=="drm", KERNEL=="renderD*", GROUP="44", MODE="0660" + + # # Configures the logging system. + # logging: + # # Logging destination. + # destinations: + # - endpoint: tcp://1.2.3.4:12345 # Where to send logs. Supported protocols are "tcp" and "udp". + # format: json_lines # Logs format. + + # # Configures the kernel. + # kernel: + # # Kernel modules to load. + # modules: + # - name: btrfs # Module name. + + # # Configures the seccomp profiles for the machine. + # seccompProfiles: + # - name: audit.json # The `name` field is used to provide the file name of the seccomp profile. + # # The `value` field is used to provide the seccomp profile. + # value: + # defaultAction: SCMP_ACT_LOG + + # # Override (patch) settings in the default OCI runtime spec for CRI containers. + + # # override default open file limit + # baseRuntimeSpecOverrides: + # process: + # rlimits: + # - hard: 1024 + # soft: 1024 + # type: RLIMIT_NOFILE + + # # Configures the node labels for the machine. + + # # node labels example. + # nodeLabels: + # exampleLabel: exampleLabelValue + + # # Configures the node annotations for the machine. + + # # node annotations example. + # nodeAnnotations: + # customer.io/rack: r13a25 + + # # Configures the node taints for the machine. Effect is optional. + + # # node taints example. + # nodeTaints: + # exampleTaint: exampleTaintValue:NoSchedule +# Provides cluster specific configuration options. +cluster: + id: ajFbAfx06GXDzj6aZKEZ7M47wpc_0y4n9f4_NjIpz-c= # Globally unique identifier for this cluster (base64 encoded random 32 bytes). + secret: eZtUgfkU6glNR7S7O6n5uQBHb/m0pkb3QD9OlSo9cDM= # Shared secret of cluster (base64 encoded random 32 bytes). + # Provides control plane specific configuration options. + controlPlane: + endpoint: https://10.0.15.1:6443 # Endpoint is the canonical controlplane endpoint, which can be an IP address or a DNS hostname. + clusterName: home-cluster # Configures the cluster's name. + # Provides cluster specific network configuration options. + network: + dnsDomain: cluster.local # The domain used by Kubernetes DNS. + # The pod subnet CIDR. + podSubnets: + - 10.244.0.0/16 + # The service subnet CIDR. + serviceSubnets: + - 10.96.0.0/12 + + # # The CNI used. + # cni: + # name: custom # Name of CNI to use. + # # URLs containing manifests to apply for the CNI. + # urls: + # - https://docs.projectcalico.org/archive/v3.20/manifests/canal.yaml + token: fkth11.nmhzlprifn9pe6jl # The [bootstrap token](https://kubernetes.io/docs/reference/access-authn-authz/bootstrap-tokens/) used to join the cluster. + # The base64 encoded root certificate authority used by Kubernetes. + ca: + crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJpRENDQVMrZ0F3SUJBZ0lRSFp3TUhKd0ZyblVLTDMrNjltQ2JnVEFLQmdncWhrak9QUVFEQWpBVk1STXcKRVFZRFZRUUtFd3ByZFdKbGNtNWxkR1Z6TUI0WERUSTJNREV4TVRFM01qQXlPVm9YRFRNMk1ERXdPVEUzTWpBeQpPVm93RlRFVE1CRUdBMVVFQ2hNS2EzVmlaWEp1WlhSbGN6QlpNQk1HQnlxR1NNNDlBZ0VHQ0NxR1NNNDlBd0VICkEwSUFCSWxHL0ppOWdEOTh2WEJKZjVWQ05VbkhuTGlIN1F5UWdMbHR0TFgrUElUeWFUaXVzSW5RRGMvUzZ4aysKTFVkSjhrN0t3d0NYUEZ0SDZTTWM0VE12MTZpallUQmZNQTRHQTFVZER3RUIvd1FFQXdJQ2hEQWRCZ05WSFNVRQpGakFVQmdnckJnRUZCUWNEQVFZSUt3WUJCUVVIQXdJd0R3WURWUjBUQVFIL0JBVXdBd0VCL3pBZEJnTlZIUTRFCkZnUVVyRU5ZMllVN3pWb1hoZ1BCb2Rqa1Z2Q2czWWd3Q2dZSUtvWkl6ajBFQXdJRFJ3QXdSQUlnV2xxb0ZXTzYKa3RnNEt3NVBtUkhUeVNuUi90cmQzTmQyc0x6WHdCRmFZcUlDSUNobFpCWDBHRGpQTVFjM0xQcG1oR2xpMVpCQgpzWkdyY0M4VVpqSjNqR0t4Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K + key: "" + # Configures cluster member discovery. + discovery: + enabled: true # Enable the cluster membership discovery feature. + # Configure registries used for cluster member discovery. + registries: + # Kubernetes registry uses Kubernetes API server to discover cluster members and stores additional information + kubernetes: + disabled: true # Disable Kubernetes discovery registry. + # Service registry is using an external service to push and pull information about cluster members. + service: {} + # # External service endpoint. + # endpoint: https://discovery.talos.dev/ + + # # A key used for the [encryption of secret data at rest](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/). + + # # Decryption secret example (do not use in production!). + # aescbcEncryptionSecret: z01mye6j16bspJYtTB/5SFX8j7Ph4JXxM2Xuu4vsBPM= + + # # A key used for the [encryption of secret data at rest](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/). + + # # Decryption secret example (do not use in production!). + # secretboxEncryptionSecret: z01mye6j16bspJYtTB/5SFX8j7Ph4JXxM2Xuu4vsBPM= + + # # The base64 encoded aggregator certificate authority used by Kubernetes for front-proxy certificate generation. + + # # AggregatorCA example. + # aggregatorCA: + # crt: LS0tIEVYQU1QTEUgQ0VSVElGSUNBVEUgLS0t + # key: LS0tIEVYQU1QTEUgS0VZIC0tLQ== + + # # The base64 encoded private key for service account token generation. + + # # AggregatorCA example. + # serviceAccount: + # key: LS0tIEVYQU1QTEUgS0VZIC0tLQ== + + # # API server specific configuration options. + # apiServer: + # image: registry.k8s.io/kube-apiserver:v1.35.0 # The container image used in the API server manifest. + # # Extra arguments to supply to the API server. + # extraArgs: + # feature-gates: ServerSideApply=true + # http2-max-streams-per-connection: "32" + # # Extra certificate subject alternative names for the API server's certificate. + # certSANs: + # - 1.2.3.4 + # - 4.5.6.7 + # # Configure the API server admission plugins. + # admissionControl: + # - name: PodSecurity # Name is the name of the admission controller. + # # Configuration is an embedded configuration object to be used as the plugin's + # configuration: + # apiVersion: pod-security.admission.config.k8s.io/v1alpha1 + # defaults: + # audit: restricted + # audit-version: latest + # enforce: baseline + # enforce-version: latest + # warn: restricted + # warn-version: latest + # exemptions: + # namespaces: + # - kube-system + # runtimeClasses: [] + # usernames: [] + # kind: PodSecurityConfiguration + # # Configure the API server audit policy. + # auditPolicy: + # apiVersion: audit.k8s.io/v1 + # kind: Policy + # rules: + # - level: Metadata + # # Configure the API server authorization config. Node and RBAC authorizers are always added irrespective of the configuration. + # authorizationConfig: + # - type: Webhook # Type is the name of the authorizer. Allowed values are `Node`, `RBAC`, and `Webhook`. + # name: webhook # Name is used to describe the authorizer. + # # webhook is the configuration for the webhook authorizer. + # webhook: + # connectionInfo: + # type: InClusterConfig + # failurePolicy: Deny + # matchConditionSubjectAccessReviewVersion: v1 + # matchConditions: + # - expression: has(request.resourceAttributes) + # - expression: '!(\''system:serviceaccounts:kube-system\'' in request.groups)' + # subjectAccessReviewVersion: v1 + # timeout: 3s + # - type: Webhook # Type is the name of the authorizer. Allowed values are `Node`, `RBAC`, and `Webhook`. + # name: in-cluster-authorizer # Name is used to describe the authorizer. + # # webhook is the configuration for the webhook authorizer. + # webhook: + # connectionInfo: + # type: InClusterConfig + # failurePolicy: NoOpinion + # matchConditionSubjectAccessReviewVersion: v1 + # subjectAccessReviewVersion: v1 + # timeout: 3s + + # # Controller manager server specific configuration options. + # controllerManager: + # image: registry.k8s.io/kube-controller-manager:v1.35.0 # The container image used in the controller manager manifest. + # # Extra arguments to supply to the controller manager. + # extraArgs: + # feature-gates: ServerSideApply=true + + # # Kube-proxy server-specific configuration options + # proxy: + # disabled: false # Disable kube-proxy deployment on cluster bootstrap. + # image: registry.k8s.io/kube-proxy:v1.35.0 # The container image used in the kube-proxy manifest. + # mode: ipvs # proxy mode of kube-proxy. + # # Extra arguments to supply to kube-proxy. + # extraArgs: + # proxy-mode: iptables + + # # Scheduler server specific configuration options. + # scheduler: + # image: registry.k8s.io/kube-scheduler:v1.35.0 # The container image used in the scheduler manifest. + # # Extra arguments to supply to the scheduler. + # extraArgs: + # feature-gates: AllBeta=true + + # # Etcd specific configuration options. + # etcd: + # image: registry.k8s.io/etcd:v3.6.7 # The container image used to create the etcd service. + # # The `ca` is the root certificate authority of the PKI. + # ca: + # crt: LS0tIEVYQU1QTEUgQ0VSVElGSUNBVEUgLS0t + # key: LS0tIEVYQU1QTEUgS0VZIC0tLQ== + # # Extra arguments to supply to etcd. + # extraArgs: + # election-timeout: "5000" + # # The `advertisedSubnets` field configures the networks to pick etcd advertised IP from. + # advertisedSubnets: + # - 10.0.0.0/8 + + # # Core DNS specific configuration options. + # coreDNS: + # image: registry.k8s.io/coredns/coredns:v1.13.2 # The `image` field is an override to the default coredns image. + + # # External cloud provider configuration. + # externalCloudProvider: + # enabled: true # Enable external cloud provider. + # # A list of urls that point to additional manifests for an external cloud provider. + # manifests: + # - https://raw.githubusercontent.com/kubernetes/cloud-provider-aws/v1.20.0-alpha.0/manifests/rbac.yaml + # - https://raw.githubusercontent.com/kubernetes/cloud-provider-aws/v1.20.0-alpha.0/manifests/aws-cloud-controller-manager-daemonset.yaml + + # # A list of urls that point to additional manifests. + # extraManifests: + # - https://www.example.com/manifest1.yaml + # - https://www.example.com/manifest2.yaml + + # # A map of key value pairs that will be added while fetching the extraManifests. + # extraManifestHeaders: + # Token: "1234567" + # X-ExtraInfo: info + + # # A list of inline Kubernetes manifests. + # inlineManifests: + # - name: namespace-ci # Name of the manifest. + # contents: |- # Manifest contents as a string. + # apiVersion: v1 + # kind: Namespace + # metadata: + # name: ci + + # # Settings for admin kubeconfig generation. + # adminKubeconfig: + # certLifetime: 1h0m0s # Admin kubeconfig certificate lifetime (default is 1 year). + + # # Allows running workload on control-plane nodes. + # allowSchedulingOnControlPlanes: true +--- +apiVersion: v1alpha1 +kind: HostnameConfig +auto: stable # A method to automatically generate a hostname for the machine. + +# # A static hostname to set for the machine. +# hostname: controlplane1 +# hostname: controlplane1.example.org diff --git a/terraform/.terraform.lock.hcl b/terraform/.terraform.lock.hcl index 1e0570c..1fcf83a 100644 --- a/terraform/.terraform.lock.hcl +++ b/terraform/.terraform.lock.hcl @@ -1,6 +1,29 @@ # This file is maintained automatically by "tofu init". # Manual edits may be lost in future updates. +provider "registry.opentofu.org/bpg/proxmox" { + version = "0.92.0" + constraints = "~> 0.72" + hashes = [ + "h1:7mILKxL+QB3lvtrz3AVAkX747jWZsuY/E+iRT2n/TIg=", + "zh:00831b5e56fc83e59435788b6130b6749abe28142b12c7da4ef885a86e449caa", + "zh:05bfe22f2b89bd047facb5702910fd381481e225c25b32590ec5e15e28c0150a", + "zh:1168580e56852affb2da446143c59a4b0dde277ea50dd1e8abb2dcd6a76f30f9", + "zh:2bc7b6c8209b02862f334d4c61a7c21e243255486bb0cf153347325e85bc9d45", + "zh:3d0aff79fc962153c49615511e468b167a36ffd41b1b6bf4b1c4161b23e4b348", + "zh:42a62840361b09b6497279b5ea23a82fe1a19d8d5121eacbdec0c35e9e283b9f", + "zh:4bc9d0ee6cba0be3f997d3469aa82aad21307327d594e51e96ecbe959e119622", + "zh:4db1d852bf19f88b090465c127dbb22dac94d5f549bc047c241343c993ec63f5", + "zh:6da6eb72dafdcd9892e45687b8f28e32b6ad99acd18f924226506d7197f63f56", + "zh:71296a009690ceceab56b24799566d7f81bc68c3610d9b55da3ed20b0cc8e49e", + "zh:785d1b8215a0e778d6d3b25f6761138721227df10c1e9e388ba0e2058808922d", + "zh:bdd56e875bd3816c5aca249ec131e8ac456e001b8ed3a7ba1a9e2009edeaec3a", + "zh:c2569eb65424fe87f3f7097df218ba65c93d50fb37d158cbaccd891552dbae48", + "zh:c7f2e8672c5c58d2a7dfcd86ccda03aaeba626d9874ac70366df0b4bd9ede2ac", + "zh:f26e0763dbe6a6b2195c94b44696f2110f7f55433dc142839be16b9697fa5597", + ] +} + provider "registry.opentofu.org/cloudflare/cloudflare" { version = "5.15.0" constraints = "~> 5.0" diff --git a/terraform/TALOS_DEPLOYMENT.md b/terraform/TALOS_DEPLOYMENT.md new file mode 100644 index 0000000..fb5ea71 --- /dev/null +++ b/terraform/TALOS_DEPLOYMENT.md @@ -0,0 +1,123 @@ +# Talos Cluster Deployment Guide + +## Prerequisites + +1. Talos template VM created (VMID 9001 on node1) +2. Proxmox "home" cluster: node1, node2, node3 (10.0.15.101-103) +3. SSH access as root (passwordless) to all Proxmox nodes +4. Environment variables loaded: `source .envrc` + +## Configuration + +The deployment creates: +- **3 Control Plane nodes**: talos-cp1, talos-cp2, talos-cp3 (VMIDs 200-202) +- **3 Worker nodes**: talos-worker1, talos-worker2, talos-worker3 (VMIDs 210-212) + +Nodes are distributed across the cluster: +- node1: talos-cp1, talos-worker1 +- node2: talos-cp2, talos-worker2 +- node3: talos-cp3, talos-worker3 + +## Deployment Process + +Since VMs get DHCP IPs initially, deploy **one node at a time** to configure static DHCP leases: + +### Initialize Terraform + +```bash +cd /Users/graham/dev/infra/terraform +source .envrc +tofu init +``` + +### Deploy Nodes One at a Time + +```bash +# Deploy first control plane node +./deploy_talos_node.sh talos_cp1 +# Note the MAC address, configure DHCP static lease, verify IP + +# Deploy remaining control plane nodes +./deploy_talos_node.sh talos_cp2 +./deploy_talos_node.sh talos_cp3 + +# Deploy worker nodes +./deploy_talos_node.sh talos_worker1 +./deploy_talos_node.sh talos_worker2 +./deploy_talos_node.sh talos_worker3 +``` + +### Manual Deployment (Alternative) + +```bash +# Deploy specific node +tofu apply -target="proxmox_virtual_environment_vm.talos_cp1" + +# Check MAC address +tofu output talos_cp1_mac + +# Continue with next node... +``` + +### View All MAC Addresses + +```bash +tofu output | grep mac +``` + +## Node Specifications + +### Control Plane +- **CPU**: 2 cores (from `var.control_plane_cores`) +- **Memory**: 8192 MB (from `var.control_plane_memory`) +- **Disk**: Inherited from template (~32GB) + +### Workers +- **CPU**: 4 cores (from `var.worker_cores`) +- **Memory**: 8192 MB (from `var.worker_memory`) +- **Disk**: Inherited from template (~32GB) + +## Post-Deployment + +After all VMs are created with static IPs: + +1. Generate Talos configuration in `/talos/` directory +2. Apply Talos configuration to nodes +3. Bootstrap the Kubernetes cluster +4. Deploy Ceph CSI driver (see CLAUDE.md) + +## Troubleshooting + +### Check VM Status +```bash +ssh root@10.0.15.101 "qm list" +ssh root@10.0.15.102 "qm list" +ssh root@10.0.15.103 "qm list" +``` + +### Start VM Manually +```bash +ssh root@10.0.15.101 "qm start 200" +``` + +### Remove and Recreate Node +```bash +tofu destroy -target="proxmox_virtual_environment_vm.talos_cp1" +tofu apply -target="proxmox_virtual_environment_vm.talos_cp1" +``` + +## Variables + +Edit `variables.tf` to customize: +- `proxmox_home_nodes`: Node names +- `control_plane_cores/memory`: Control plane resources +- `worker_cores/memory`: Worker resources +- `talos_template_vmid`: Template VMID +- `talos_storage`: Ceph pool name + +## Files + +- `talos_vms.tf`: VM resource definitions +- `talos_outputs.tf`: MAC addresses and VMID outputs +- `deploy_talos_node.sh`: Helper script for one-at-a-time deployment +- `TALOS_DEPLOYMENT.md`: This file diff --git a/terraform/deploy_talos_node.sh b/terraform/deploy_talos_node.sh new file mode 100755 index 0000000..3ba8d32 --- /dev/null +++ b/terraform/deploy_talos_node.sh @@ -0,0 +1,44 @@ +#!/bin/bash +# Script to deploy Talos nodes one at a time for DHCP static lease configuration + +set -e + +NODE=$1 + +if [ -z "$NODE" ]; then + echo "Usage: $0 " + echo "" + echo "Available nodes:" + echo " talos_cp1 - Control Plane 1 (node1)" + echo " talos_cp2 - Control Plane 2 (node2)" + echo " talos_cp3 - Control Plane 3 (node3)" + echo " talos_worker1 - Worker 1 (node1)" + echo " talos_worker2 - Worker 2 (node2)" + echo " talos_worker3 - Worker 3 (node3)" + echo "" + echo "Example: $0 talos_cp1" + exit 1 +fi + +# Source environment variables +if [ -f .envrc ]; then + source .envrc +fi + +echo "=== Deploying $NODE ===" +echo "" + +# Apply only the targeted node +tofu apply -target="proxmox_virtual_environment_vm.$NODE" + +echo "" +echo "=== VM Created - MAC Address ===" +tofu output "${NODE}_mac" + +echo "" +echo "Next steps:" +echo "1. Note the MAC address above" +echo "2. Configure DHCP static lease for this MAC" +echo "3. Start the VM if not auto-started" +echo "4. Verify the VM gets the correct IP" +echo "5. Run this script again with the next node" diff --git a/terraform/main.tf b/terraform/main.tf index d5e36fa..f850de7 100644 --- a/terraform/main.tf +++ b/terraform/main.tf @@ -20,6 +20,10 @@ terraform { source = "hashicorp/null" version = "~> 3.0" } + proxmox = { + source = "bpg/proxmox" + version = "~> 0.72" + } } } @@ -45,3 +49,12 @@ provider "cloudflare" { # API token is read from CLOUDFLARE_API_TOKEN environment variable # No explicit api_token parameter needed - provider automatically reads the env var } + +provider "proxmox" { + endpoint = var.proxmox_home_api_url + api_token = "${var.proxmox_home_api_token}=${var.proxmox_home_api_secret}" + insecure = var.proxmox_home_tls_insecure + ssh { + agent = true + } +} diff --git a/terraform/talos_outputs.tf b/terraform/talos_outputs.tf new file mode 100644 index 0000000..4ae6c72 --- /dev/null +++ b/terraform/talos_outputs.tf @@ -0,0 +1,63 @@ +# Outputs for Talos VMs + +# Control Plane Outputs +output "talos_cp1_mac" { + description = "MAC address of talos-cp1" + value = try(proxmox_virtual_environment_vm.talos_cp1.network_device[0].mac_address, "Not yet created") +} + +output "talos_cp1_vmid" { + description = "VMID of talos-cp1" + value = try(proxmox_virtual_environment_vm.talos_cp1.vm_id, "Not yet created") +} + +output "talos_cp2_mac" { + description = "MAC address of talos-cp2" + value = try(proxmox_virtual_environment_vm.talos_cp2.network_device[0].mac_address, "Not yet created") +} + +output "talos_cp2_vmid" { + description = "VMID of talos-cp2" + value = try(proxmox_virtual_environment_vm.talos_cp2.vm_id, "Not yet created") +} + +output "talos_cp3_mac" { + description = "MAC address of talos-cp3" + value = try(proxmox_virtual_environment_vm.talos_cp3.network_device[0].mac_address, "Not yet created") +} + +output "talos_cp3_vmid" { + description = "VMID of talos-cp3" + value = try(proxmox_virtual_environment_vm.talos_cp3.vm_id, "Not yet created") +} + +# Worker Outputs +output "talos_worker1_mac" { + description = "MAC address of talos-worker1" + value = try(proxmox_virtual_environment_vm.talos_worker1.network_device[0].mac_address, "Not yet created") +} + +output "talos_worker1_vmid" { + description = "VMID of talos-worker1" + value = try(proxmox_virtual_environment_vm.talos_worker1.vm_id, "Not yet created") +} + +output "talos_worker2_mac" { + description = "MAC address of talos-worker2" + value = try(proxmox_virtual_environment_vm.talos_worker2.network_device[0].mac_address, "Not yet created") +} + +output "talos_worker2_vmid" { + description = "VMID of talos-worker2" + value = try(proxmox_virtual_environment_vm.talos_worker2.vm_id, "Not yet created") +} + +output "talos_worker3_mac" { + description = "MAC address of talos-worker3" + value = try(proxmox_virtual_environment_vm.talos_worker3.network_device[0].mac_address, "Not yet created") +} + +output "talos_worker3_vmid" { + description = "VMID of talos-worker3" + value = try(proxmox_virtual_environment_vm.talos_worker3.vm_id, "Not yet created") +} diff --git a/terraform/talos_vms.tf b/terraform/talos_vms.tf new file mode 100644 index 0000000..6f341f7 --- /dev/null +++ b/terraform/talos_vms.tf @@ -0,0 +1,191 @@ +# Talos Control Plane Nodes +resource "proxmox_virtual_environment_vm" "talos_cp1" { + name = "talos-cp1" + description = "Talos Control Plane 1" + node_name = var.proxmox_home_nodes[0] # node1 + vm_id = 200 + + clone { + vm_id = var.talos_template_vmid + full = true + } + + cpu { + cores = var.control_plane_cores + type = "host" + } + + memory { + dedicated = var.control_plane_memory + } + + network_device { + bridge = var.network_bridge + } + + lifecycle { + ignore_changes = [ + network_device, + ] + } +} + +resource "proxmox_virtual_environment_vm" "talos_cp2" { + name = "talos-cp2" + description = "Talos Control Plane 2" + node_name = var.proxmox_home_nodes[1] # node2 + vm_id = 201 + + clone { + vm_id = var.talos_template_vmid + node_name = var.talos_template_node + full = true + } + + cpu { + cores = var.control_plane_cores + type = "host" + } + + memory { + dedicated = var.control_plane_memory + } + + network_device { + bridge = var.network_bridge + } + + lifecycle { + ignore_changes = [ + network_device, + ] + } +} + +resource "proxmox_virtual_environment_vm" "talos_cp3" { + name = "talos-cp3" + description = "Talos Control Plane 3" + node_name = var.proxmox_home_nodes[2] # node3 + vm_id = 202 + + clone { + vm_id = var.talos_template_vmid + node_name = var.talos_template_node + full = true + } + + cpu { + cores = var.control_plane_cores + type = "host" + } + + memory { + dedicated = var.control_plane_memory + } + + network_device { + bridge = var.network_bridge + } + + lifecycle { + ignore_changes = [ + network_device, + ] + } +} + +# Talos Worker Nodes +resource "proxmox_virtual_environment_vm" "talos_worker1" { + name = "talos-worker1" + description = "Talos Worker 1" + node_name = var.proxmox_home_nodes[0] # node1 + vm_id = 210 + + clone { + vm_id = var.talos_template_vmid + full = true + } + + cpu { + cores = var.worker_cores + type = "host" + } + + memory { + dedicated = var.worker_memory + } + + network_device { + bridge = var.network_bridge + } + + lifecycle { + ignore_changes = [ + network_device, + ] + } +} + +resource "proxmox_virtual_environment_vm" "talos_worker2" { + name = "talos-worker2" + description = "Talos Worker 2" + node_name = var.proxmox_home_nodes[1] # node2 + vm_id = 211 + + clone { + vm_id = var.talos_template_vmid + node_name = var.talos_template_node + full = true + } + + cpu { + cores = var.worker_cores + type = "host" + } + + memory { + dedicated = var.worker_memory + } + + network_device { + bridge = var.network_bridge + } + + lifecycle { + ignore_changes = [ + network_device, + ] + } +} + +resource "proxmox_virtual_environment_vm" "talos_worker3" { + name = "talos-worker3" + description = "Talos Worker 3" + node_name = var.proxmox_home_nodes[2] # node3 + vm_id = 212 + + clone { + vm_id = var.talos_template_vmid + node_name = var.talos_template_node + full = true + } + + cpu { + cores = var.worker_cores + type = "host" + } + + memory { + dedicated = var.worker_memory + } + + network_device { + bridge = var.network_bridge + } + + lifecycle { + ignore_changes = [ + network_device, + ] + } +} diff --git a/terraform/talosconfig b/terraform/talosconfig new file mode 100644 index 0000000..e35e5db --- /dev/null +++ b/terraform/talosconfig @@ -0,0 +1,2 @@ +context: "" +contexts: {} diff --git a/terraform/terraform.tfvars b/terraform/terraform.tfvars index 3155aa2..a91163a 100644 --- a/terraform/terraform.tfvars +++ b/terraform/terraform.tfvars @@ -1,7 +1,3 @@ -# Proxmox tokens -token_secret = "1cdd961f-5ee7-4110-8806-e72357a2226f" -token_id = "terraform@pve!terraform-token" - # DNS Provider secrets dnsimple_token = "dnsimple_a_sRmKiuHrxR2UiMHf0NREqAtgfELyRVAI" porkbun_api_key = "pk1_5f321ff82ce0c085e67f9569db48976d838f302bc15b88292373e4b6b9891f17" diff --git a/terraform/variables.tf b/terraform/variables.tf index f27201b..cbad05d 100644 --- a/terraform/variables.tf +++ b/terraform/variables.tf @@ -6,38 +6,6 @@ variable "vultr_api_key" { default = "" } - -# Proxmox Variables -variable "proxmox_api_url" { - description = "Proxmox API URL" - type = string - default = "https://10.0.16.231:8006" -} - -variable "proxmox_user" { - description = "Proxmox user for authentication" - type = string - default = "root@pam" -} - -variable "proxmox_password" { - description = "Proxmox password" - type = string - sensitive = true -} - -variable "proxmox_tls_insecure" { - description = "Allow insecure TLS connections to Proxmox" - type = bool - default = true -} - -variable "proxmox_nodes" { - description = "List of Proxmox nodes to distribute VMs across" - type = list(string) - default = ["vm2-380"] -} - variable "network_bridge" { description = "Network bridge to use for VMs" type = string @@ -130,31 +98,51 @@ variable "porkbun_secret_api_key" { sensitive = true } -variable "proxmox_host" { - description = "Proxmox host for VM deployment" +# Home Proxmox Cluster Variables +variable "proxmox_home_api_url" { + description = "Home Proxmox cluster API URL" type = string - default = "vm2-380" + default = "https://10.0.15.101:8006" } -variable "nic_name" { - description = "Network interface name" - type = string - default = "vmbr0" -} - -variable "api_url" { - description = "Proxmox API URL (legacy)" - type = string - default = "https://10.0.0.2:8006/api2/json" -} - -variable "token_secret" { - description = "Proxmox token secret" +variable "proxmox_home_api_token" { + description = "Home Proxmox API token ID" type = string sensitive = true } -variable "token_id" { - description = "Proxmox token ID" +variable "proxmox_home_api_secret" { + description = "Home Proxmox API token secret" type = string + sensitive = true +} + +variable "proxmox_home_tls_insecure" { + description = "Allow insecure TLS connections to home Proxmox" + type = bool + default = true +} + +variable "proxmox_home_nodes" { + description = "List of home Proxmox nodes" + type = list(string) + default = ["node1", "node2", "node3"] +} + +variable "talos_template_vmid" { + description = "VMID of Talos template" + type = number + default = 9001 +} + +variable "talos_template_node" { + description = "Proxmox node where Talos template is stored" + type = string + default = "node1" +} + +variable "talos_storage" { + description = "Ceph storage pool for Talos VMs" + type = string + default = "ceph" }