Cluster-side:
- Deploy kube-state-metrics (kube-system) for cluster-state series.
- Deploy node-exporter DaemonSet on all 7 nodes via the new monitoring ns
(privileged PSS, hostNetwork, read-only host mounts).
- Both managed by ArgoCD apps under home/cluster/argocd/apps/.
Prometheus-side:
- Replace direct pod-IP scraping (which can't reach the 10.244.0.0/16 pod
network from outside the cluster) with API-server-proxy scraping for
both Services and Pods. Now one target per pod, all healthy.
- Extend external Prometheus RBAC with services/proxy + pods/proxy.
Grafana-side:
- Add file-based dashboard provisioner; ship 5 standard dashboards
(Node Exporter Full, k8s cluster-via-prometheus, k8s all-in-one,
k8s monitoring, Prometheus 2.0 overview).
prom.w5isp.com:3000 now shows live data for all 7 cluster nodes plus
the prom host itself.
Provisions prom.w5isp.com (10.0.15.31) as the home-cluster monitoring host.
Prometheus 3.6.0 scrapes the Talos cluster API/nodes/cAdvisor via an external
SA token; Grafana ships with the Prometheus datasource pre-provisioned.
host_vars opens 9090/9093/9100/3000 to 10.0.0.0/16 plus SSH from the home LAN
so the firewall role doesn't lock out admin access.
The towerops-web repo got renamed to codeberg.org/gmcintire/towerops
and its container image follows the same rename
(codeberg.org/gmcintire/towerops). Application source repo and
image-updater registry list updated.
The prop and prop-grid-rs container images now live on
codeberg.org/gmcintire/* (see prop repo commit 8a9c0ba6 for the
workflow flip). Application source repo and image-updater registry
list follow.
Cluster-side state still required for this to roll out, applied
out-of-band via kubectl:
- Secret prop/forgejo-registry — dockerconfigjson with codeberg.org
auth (also keeps git.mcintire.me auth during the transition).
- Secret argocd/forgejo-registry — same shape, used by the
argocd-image-updater registry config.
- Secret argocd/codeberg-credentials-gmcintire (label
argocd.argoproj.io/secret-type=repo-creds) — credentials template
matching url=https://codeberg.org/gmcintire so future codeberg
repos under that owner pick up auth automatically.
- ConfigMap argocd/argocd-image-updater-config — registries.conf
extended with codeberg.org entry referencing the new pull secret.
annotates the 4 Applications so image-updater scans git.mcintire.me
on a 2min loop and writes new tags to spec.source.kustomize.images
(no git commits, no CI sed-and-bump).
Configure passthrough NVMe drives (/dev/sdb) on each worker with
XFS partition mounted at /var/mnt/longhorn. Switch worker installer
image to Talos Image Factory build with iscsi-tools and util-linux-tools
extensions required by Longhorn. Remove conflicting grubUseUKICmdline
setting.
- Upgrade all nodes from v1.12.1/v1.12.2 to v1.12.4 installer image
- Set explicit nameservers (9.9.9.9, 149.112.112.112, 1.1.1.1) on
workers to replace unreliable DHCP-provided resolver at 10.0.15.254
- Fix talos-worker2 missing ExtensionServiceConfig for ext-tailscale
which was causing node reboots every ~35 minutes
- Update tailscale.patch.yaml with working auth key
- Remove proxmox/talos/null providers and variables
- Remove HTTP backend, use local state
- Remove skookum.me, towerops.net, vntx.net, as393837.net from DNS
- Switch aprs.me NS to Cloudflare
- Rename terraform/ to tofu/
- Add marcfrederick/porkbun provider for registrar NS management
- Add dns.tf with all 14 Porkbun domains and import blocks
- Update gridmap.org BIND9 zone NS to Cloudflare
- Update CLAUDE.md to reflect new directory structure
