Graham McIntire 91f23416e4 feat(insights): SectorOverload + CpeRealign recommendation rules

Two new multi-source rules using data we already collect, both wired
into the existing hourly RecommendationsRunWorker.

SectorOverload — fires when a Preseem-monitored AP has <25% free
airtime AND active subscribers. Critical urgency when free airtime
drops below 15%, OR when free airtime <25% AND QoE <50. Reuses the
airtime/subscriber/QoE data already pulled by PreseemSyncWorker so
no new collection is needed. UI: orange evidence card with free
airtime, subscriber count, QoE score, and AP model.

CpeRealign — fires when a wireless client has BOTH signal_strength
<=-78 dBm AND SNR <=18 dB seen in the last 2 hours. This is the
classic alignment / obstruction signature, distinct from the
existing single-metric wireless_signal_weak / wireless_snr_low
alerts produced by WirelessInsightWorker. One insight per (AP, CPE)
pair via a metadata.dedup_key. Critical when signal <=-88 OR SNR
<=10. UI: rose evidence card with signal, SNR, TX/RX rate, distance,
hostname.

Insights.insert_insight_if_new/1 dedup logic now prefers a
metadata.dedup_key over device_id when explicitly set, allowing
per-CPE insights without collapsing multiple CPEs on the same AP
into a single insight. Existing rules without dedup_key are
unaffected.

Insight.@valid_types extended with sector_overload and cpe_realign.

LLM enrichment automatically applies to both new types via the
existing Phase 1 worker — no extra wiring.

Also adds k8s/secrets.yaml to .gitignore so operators can drop a
local Secret manifest with real values, kubectl apply manually, and
never accidentally commit it. Documented in k8s/README.md.

2026-05-09 17:07:15 -05:00

2.4 KiB

Raw Blame History

Kubernetes Deployment

Secrets Management

Secrets are managed directly in the cluster and must be created before deploying the application.

Required secrets in the towerops namespace:

gitlab-registry - Docker registry credentials for pulling images
towerops-secrets - Application secrets (RELEASE_COOKIE, SECRET_KEY_BASE)
towerops-db - Database connection credentials
towerops-aws - AWS credentials (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION)

Optional secrets:

towerops-llm - DeepSeek API credentials for LLM-powered insight enrichment. Optional — when missing, insights still display without an AI summary.

Local secrets workflow

The file k8s/secrets.yaml is gitignored. Use it to keep one-or-more Secret manifests with real values for hand-application against the cluster. Bootstrap from any *.example.yaml template:

cp k8s/towerops-llm-secret.example.yaml k8s/secrets.yaml
# edit k8s/secrets.yaml — fill in DEEPSEEK_API_KEY, etc.
kubectl apply -f k8s/secrets.yaml
kubectl rollout restart deployment/towerops -n towerops

k8s/secrets.yaml may contain multiple ----separated documents if you need more than one secret. It is excluded from git via .gitignore.

For local development, the project root .envrc is used by direnv.

Deployment Timestamp

The application footer displays the deployment timestamp to track when the current version was deployed. This is automatically set by GitLab CI during deployment:

# GitLab CI sets this during deploy
- kubectl set env deployment/towerops DEPLOY_TIMESTAMP=$(date -u +"%Y-%m-%dT%H:%M:%SZ") -n towerops

All pods in the deployment share the same timestamp (when the deployment was initiated), regardless of when individual pods were created. This is displayed in the footer as "Last deployed X ago · YYYY-MM-DD HH:MM:SS UTC".

For manual deployments without GitLab CI, set the timestamp:

kubectl set env deployment/towerops DEPLOY_TIMESTAMP=$(date -u +"%Y-%m-%dT%H:%M:%SZ") -n towerops

Deploying

Apply all resources using kustomize:

kubectl apply -k k8s/

Or individually:

kubectl apply -f k8s/namespace.yaml
kubectl apply -f k8s/secret.yaml
kubectl apply -f k8s/deployment.yaml
kubectl apply -f k8s/service.yaml
kubectl apply -f k8s/service-headless.yaml
kubectl apply -f k8s/certificate.yaml
kubectl apply -f k8s/ingressroute.yaml

2.4 KiB Raw Blame History