graham/towerops
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
2220 commits 1 branch 0 tags 50 MiB
Commit graph

487 commits

Author SHA1 Message Date
Graham McIntire
c9cb95129c fix/production-workflow-conflict (#208) 
Co-authored-by: Forgejo Actions <actions@git.mcintire.me>
Reviewed-on: graham/towerops-web#208
 2026-03-28 12:42:23 -05:00
Graham McIntire
fc03ed7b0e Disable SSL for database connections on private network (#97) 
Every timeout stack trace pointed at ssl_gen_statem.call/2 — the OTP 27
SSL gen_statem hanging during recv. PG is on private network (10.0.15.20)
and SSL verify was already disabled (verify_none), providing no security
benefit while adding CPU overhead and the gen_statem layer that caused
all connection timeouts.

Removed DATABASE_SSL and DATABASE_SSL_VERIFY env vars from both the
init container and main container. Also disabled SSL on the PG server
(ssl=off in postgresql.conf, hostssl→host in pg_hba.conf).

Reviewed-on: graham/towerops-web#97
 2026-03-20 12:48:05 -05:00
Graham McIntire
8a54f9a866 Increase CPU limits to fix SSL connection timeouts (#96) 
Pods were running at 500m/500m (100% of limit) with 44-47% of scheduling
periods being CPU-throttled. SSL decryption is CPU-intensive — when the
BEAM gets throttled mid ssl_gen_statem:call/2, the SSL recv stalls until
the cgroup allows CPU again, causing the 15s DBConnection timeout.

This was the actual root cause of "client timed out because it queued
and checked out the connection for longer than 15000ms" errors. Previous
fixes (TCP keepalive, pool tuning, oban pruner) were treating symptoms.

- CPU limit: 500m → 2000m (4 cores burst capacity)
- CPU request: 100m → 500m (guarantee adequate baseline)

Reviewed-on: graham/towerops-web#96
 2026-03-20 12:24:06 -05:00
Graham McIntire
071729f65c Reduce pool size and deployment surge to prevent PG connection exhaustion (#94) 
PG max_connections=100 (97 usable). With pool_size=15 and maxSurge=100%,
rolling deploys briefly run 4 pods × 15 = 60 pool connections plus Oban
notifiers, exceeding the limit and causing too_many_connections errors
that cascade into ssl recv: closed timeouts.

- pool_size: 15 → 10 (steady state: 2×10=20, deploy: 3×10=30)
- maxSurge: 100% → 1 (max 3 pods during rollout instead of 4)

Reviewed-on: graham/towerops-web#94
 2026-03-20 10:39:52 -05:00
Graham McIntire
b03f280b65 fix-ci-docker (#59) 
Co-authored-by: Forgejo Actions <actions@git.mcintire.me>
Reviewed-on: graham/towerops-web#59
 2026-03-17 13:13:09 -05:00
Graham McIntire
06c5d14455 ui/remove-device-type-column (#49) 
Co-authored-by: Forgejo Actions <actions@git.mcintire.me>
Reviewed-on: graham/towerops-web#49
 2026-03-16 16:38:00 -05:00
Graham McIntire
8abf01ccb1 fix-mobile-qr-device-info-width (#43) 
Co-authored-by: Forgejo Actions <actions@git.mcintire.me>
Reviewed-on: graham/towerops-web#43
 2026-03-16 14:02:31 -05:00
Forgejo Actions
011d82ce26 chore: update production image to production-1773007128-e91d74d [skip ci] 2026-03-08 22:04:51 +00:00
CI
bf7004c770 chore: update towerops-web image to git.mcintire.me/graham/towerops-web:main-1772816933-2a61626 [skip ci] 2026-03-06 17:16:12 +00:00
Graham McIntire
d0e7eb4308
feat: add Stripe meter ID configuration 
- Add stripe_meter_id to dev, runtime, and test configs
- Add STRIPE_METER_ID to k8s deployment secret references
- Use real Stripe meter and price IDs from Stripe setup
- Meter: mtr_test_61UHPU067veEZ7bhl41S77kvnTfgyODY
- Price: price_1T81XBS77kvnTfgyPlw1jF8N ($1/device/month)
- Include setup script for Stripe billing configuration
 2026-03-06 11:08:06 -06:00
Graham McIntire
20f05584dc
fix: use k8s secret for Stripe env vars instead of hardcoded values 
- Changed from hardcoded placeholder values to secretKeyRef
- References towerops-billing secret (with optional: true)
- Won't crash if secret doesn't exist yet
- User can create secret later with real Stripe keys
 2026-03-06 11:08:06 -06:00
Graham McIntire
3523676359
feat: configure Stripe environment variables for dev and k8s 
- Add Stripe test API keys to dev.exs for local development
- Add placeholder Stripe env vars to k8s deployment to prevent startup crashes
- Includes secret_key, webhook_secret, and price_id configuration
 2026-03-06 11:08:05 -06:00
CI
1c081623aa chore: update towerops-web image to git.mcintire.me/graham/towerops-web:main-1772813291-f832130 [skip ci] 2026-03-06 16:14:54 +00:00
CI
43532007b8 chore: update towerops-web image to git.mcintire.me/graham/towerops-web:main-1772807346-b5e8e00 [skip ci] 2026-03-06 14:32:10 +00:00
CI
bc7922f476 chore: update towerops-web image to git.mcintire.me/graham/towerops-web:main-1772806694-acf7e55 [skip ci] 2026-03-06 14:21:09 +00:00
CI
a083c7069c chore: update towerops-web image to git.mcintire.me/graham/towerops-web:main-1772806349-1e072c7 [skip ci] 2026-03-06 14:15:35 +00:00
CI
e47b08c44f chore: update towerops-web image to git.mcintire.me/graham/towerops-web:main-1772805892-52329cf [skip ci] 2026-03-06 14:11:25 +00:00
CI
1038cc3d42 chore: update towerops-web image to git.mcintire.me/graham/towerops-web:main-1772805491-0c7d195 [skip ci] 2026-03-06 14:01:06 +00:00
CI
4ebe215a90 chore: update towerops-web image to git.mcintire.me/graham/towerops-web:main-1772804771-b5cf562 [skip ci] 2026-03-06 13:49:06 +00:00
CI
be9ebff63e chore: update towerops-web image to git.mcintire.me/graham/towerops-web:main-1772804218-9e525aa [skip ci] 2026-03-06 13:39:48 +00:00
CI
3ed732fecc chore: update towerops-web image to git.mcintire.me/graham/towerops-web:main-1772803826-1f49e1f [skip ci] 2026-03-06 13:33:19 +00:00
CI
d3c4a9cc32 chore: update towerops-web image to git.mcintire.me/graham/towerops-web:main-1772760484-7a13368 [skip ci] 2026-03-06 01:41:10 +00:00
CI
0bd82678ef chore: update towerops-web image to git.mcintire.me/graham/towerops-web:main-1772759087-9300415 [skip ci] 2026-03-06 01:15:38 +00:00
CI
964d9dab76 chore: update towerops-web image to git.mcintire.me/graham/towerops-web:main-1772736989-9e6d006 [skip ci] 2026-03-05 18:59:57 +00:00
Graham McIntire
ab4948467a
feat: run database migrations on every deploy 
- Add init container to run migrations before app starts
- Uses /app/bin/migrate script (runs Towerops.Release.migrate())
- Migrations run once per deployment, before any pods start
- Update CI/CD to set both migrate and towerops container images
- Prevents issues like missing severity column in production
 2026-03-05 12:39:01 -06:00
CI
7abd57a2f3 chore: update towerops-web image to git.mcintire.me/graham/towerops-web:main-1772735406-025fa88 [skip ci] 2026-03-05 18:33:40 +00:00
CI
a87b729abd chore: update towerops-web image to git.mcintire.me/graham/towerops-web:main-1772722775-cf412e2 [skip ci] 2026-03-05 15:03:24 +00:00
CI
8ead55d2bf chore: update towerops-web image to git.mcintire.me/graham/towerops-web:main-1772720897-c231cbd [skip ci] 2026-03-05 14:32:07 +00:00
CI
61cad41f5c chore: update towerops-web image to git.mcintire.me/graham/towerops-web:main-1772719169-53ab811 [skip ci] 2026-03-05 14:01:21 +00:00
CI
be1add2e1a chore: update towerops-web image to git.mcintire.me/graham/towerops-web:main-1772668279-5e1f8d3 [skip ci] 2026-03-04 23:55:04 +00:00
CI
1b11bb25fc chore: update towerops-web image to git.mcintire.me/graham/towerops-web:main-1772667234-f1b6151 [skip ci] 2026-03-04 23:34:19 +00:00
FluxCD
65b41f82fe chore: update towerops image to git.mcintire.me/graham/towerops-web:main-1772667132-044e21a [skip ci] 2026-03-04 23:33:03 +00:00
FluxCD
33cbd99cb7 chore: update towerops image to git.mcintire.me/graham/towerops-web:main-1772666722-10df6d8 [skip ci] 2026-03-04 23:32:06 +00:00
FluxCD
9e28d37256 chore: update towerops image to git.mcintire.me/graham/towerops-web:main-1772666343-10df6d8 [skip ci] 2026-03-04 23:26:03 +00:00
FluxCD
18c610ca50 chore: update towerops image to git.mcintire.me/graham/towerops-web:main-1772578710-f761290 [skip ci] 2026-03-03 23:05:29 +00:00
FluxCD
19dd54114a chore: update towerops image to git.mcintire.me/graham/towerops-web:main-1772578563-f761290 [skip ci] 2026-03-03 22:58:30 +00:00
FluxCD
8882012a65 chore: update towerops image to git.mcintire.me/graham/towerops-web:main-1771432526-f0fea0d [skip ci] 2026-02-18 16:39:38 +00:00
FluxCD
cc26b05a9b chore: update towerops image to git.mcintire.me/graham/towerops-web:main-1771431639-998bdf3 [skip ci] 2026-02-18 16:26:36 +00:00
FluxCD
90c24cce49 chore: update towerops image to git.mcintire.me/graham/towerops-web:main-1771347031-2929848 [skip ci] 2026-02-17 16:54:13 +00:00
FluxCD
621afeb5c3 chore: update towerops image to git.mcintire.me/graham/towerops-web:main-1771346675-edeeacf [skip ci] 2026-02-17 16:48:12 +00:00
FluxCD
1c745a7e1e chore: update towerops image to git.mcintire.me/graham/towerops-web:main-1771346299-b7e2fb4 [skip ci] 2026-02-17 16:42:13 +00:00
FluxCD
f5f98ce00b chore: update towerops image to git.mcintire.me/graham/towerops-web:main-1771337653-c92dfff [skip ci] 2026-02-17 14:19:34 +00:00
FluxCD
c6cc803425 chore: update towerops image to git.mcintire.me/graham/towerops-web:main-1771336126-1712e7a [skip ci] 2026-02-17 13:52:27 +00:00
FluxCD
9897af57df chore: update towerops image to git.mcintire.me/graham/towerops-web:main-1771335850-8f850f9 [skip ci] 2026-02-17 13:48:27 +00:00
FluxCD
327c981422 chore: update towerops image to git.mcintire.me/graham/towerops-web:main-1771333965-78aa415 [skip ci] 2026-02-17 13:16:21 +00:00
FluxCD
ee478a4817 chore: update towerops image to git.mcintire.me/graham/towerops-web:main-1771282252-7160bf9 [skip ci] 2026-02-16 22:54:51 +00:00
FluxCD
0b03efe7b6 chore: update towerops image to git.mcintire.me/graham/towerops-web:main-1771281501-de986bd [skip ci] 2026-02-16 22:42:48 +00:00
FluxCD
09c56e3e9b chore: update towerops image to git.mcintire.me/graham/towerops-web:main-1771279924-179f049 [skip ci] 2026-02-16 22:14:45 +00:00
FluxCD
72e0c0bf22 chore: update towerops image to git.mcintire.me/graham/towerops-web:main-1771279533-e09b306 [skip ci] 2026-02-16 22:09:42 +00:00
FluxCD
0cc519b6bf chore: update towerops image to git.mcintire.me/graham/towerops-web:main-1771279255-d233f55 [skip ci] 2026-02-16 22:03:41 +00:00