Switch deployment pipeline from GitLab Agent kubectl to FluxCD image
automation. CI now only builds and pushes tagged images to Forgejo
registry; FluxCD detects new tags and updates the deployment manifest.
Fix Ecto.MultipleResultsError on /dashboard when a device has multiple
Preseem access point records by using limit(1) instead of Repo.get_by.
Removing all Valkey (Redis) resources from Kubernetes due to instability
caused by Flannel CNI networking issues. Redis will now run on Proxmox
hosts for better stability and performance.
Changes:
- Delete Valkey StatefulSet (master + 2 replicas)
- Delete Valkey Sentinel StatefulSet (3 instances)
- Delete Valkey services (headless and sentinel)
- Delete Valkey ConfigMap
- Remove Valkey resources from kustomization.yaml
- Update deployment to use towerops-redis secret for connection
Next Steps:
- Set up Redis Sentinel on 3 Proxmox hosts/LXC containers
- Create towerops-redis secret with REDIS_HOST and REDIS_PORT
- Test failover and application connectivity
Benefits:
- Not affected by K8s networking issues (Flannel failures)
- More stable (no restarts from node issues)
- Better performance (no K8s overhead)
- Independent lifecycle from K8s cluster
Prevents race condition where towerops pods try to start before Flannel
CNI is fully initialized during node restarts.
Same fix as applied to Valkey StatefulSet (commit 30a0b9a). Both
critical infrastructure components now have proper priority classes to
ensure CNI readiness before pod scheduling.
Benefits:
- Eliminates FailedCreatePodSandBox errors during node restarts
- Reduces unnecessary pod restarts from 24+ to 0
- Ensures stable startup even during Flannel restarts
- Combined with Redis health checks for complete resilience
Verified working:
- Deployment rolled out successfully with 0 issues
- Priority class applied: system-cluster-critical
- Redis health check confirmed: 'Redis is available, starting Exq'
🤖 Generated with Claude Code
- Remove PVC volume mount from k8s/deployment.yaml
- Delete k8s/mib-pvc.yaml (no longer needed)
- Update .gitignore to commit MIB files to git
- Add mix import_mibs task to copy MIBs from LibreNMS
- Add all MIB files from priv/mibs/ to git
This fixes the multi-attach volume error in Kubernetes where new pods
couldn't start because the RWO (ReadWriteOnce) PVC was attached to the
old pod. MIBs are now part of the Docker image and can be used by all
pods simultaneously.
- Add timezone to socket assigns in mount_current_scope
- Detect user timezone from browser using Intl.DateTimeFormat
- Auto-save detected timezone to user profile on first connection
- Update all templates to use TimeHelpers with timezone parameter
- Update agent helper functions to accept timezone parameter
- Templates updated:
- Alert list (triggered, acknowledged, resolved)
- Device list (last checked)
- Agent list and show (created, last seen, updated)
- Admin dashboard (impersonation logs)
- Admin org/user lists (created/joined dates)
- Main dashboard (alert times)
Changed Valkey service from headless to ClusterIP and updated
REDIS_HOST to use full FQDN for more reliable DNS resolution.
Changes:
- valkey-service.yaml: Changed from headless (clusterIP: None) to ClusterIP
- deployment.yaml: Changed REDIS_HOST from "valkey" to "valkey.towerops.svc.cluster.local"
This ensures stable ClusterIP and explicit DNS resolution for Redis connections.
Previously Valkey ran as a sidecar container in the main deployment,
causing it to restart every time the web app deployed. This resulted
in connection errors and cache loss during deployments.
Changes:
- Created valkey-statefulset.yaml with StatefulSet resource (1 replica)
- Created valkey-service.yaml with headless service
- Removed Valkey sidecar container from deployment.yaml
- Updated REDIS_HOST from 127.0.0.1 to valkey-0.valkey.towerops.svc.cluster.local
- Added new resources to kustomization.yaml
Benefits:
- Valkey persists across web app deployments
- No connection errors during rolling updates
- Cache data preserved
- Independent scaling and resource management
With Horde's distributed supervision, devices will automatically
distribute ~25% per pod across all 4 replicas.
Benefits:
- 4x horizontal scaling for device monitoring and SNMP polling
- Better fault tolerance (can lose 3 pods and still operate)
- Automatic load balancing via consistent hashing
- Each device still polls from exactly ONE pod (no duplicates)
Changes:
- Added startupProbe to towerops container to allow up to 70s for startup
(gives Valkey sidecar time to be ready)
- Changed REDIS_HOST from 'localhost' to '127.0.0.1' for explicit loopback
- Removed initialDelaySeconds from liveness/readiness probes (handled by startup)
- Added comment clarifying Redis connects to Valkey sidecar
This ensures the app waits for Valkey to be ready before accepting traffic.
