134 lines
3.1 KiB
Markdown
134 lines
3.1 KiB
Markdown
# Deployment Monitoring Scripts
|
|
|
|
## monitor-deploy.sh
|
|
|
|
Real-time monitoring of GitLab CI/CD pipelines and Kubernetes deployments.
|
|
|
|
### Features
|
|
|
|
- Auto-detects project (main app vs agent) from git remote
|
|
- Shows live pipeline progress with job statuses
|
|
- Color-coded output with icons
|
|
- Auto-refreshes every 5 seconds
|
|
- Shows failed job logs
|
|
- For main app: automatically monitors Kubernetes rollout after successful build
|
|
|
|
### Setup
|
|
|
|
1. **Get GitLab Personal Access Token**
|
|
|
|
Visit: https://gitlab.com/-/profile/personal_access_tokens
|
|
|
|
Create a token with scope: `read_api`
|
|
|
|
2. **Export token**
|
|
|
|
```bash
|
|
export GITLAB_ACCESS_TOKEN=<your-token>
|
|
```
|
|
|
|
Add to your `~/.zshrc` or `~/.bashrc`:
|
|
```bash
|
|
export GITLAB_ACCESS_TOKEN=glpat-xxxxxxxxxxxxxxxxxxxx
|
|
```
|
|
|
|
3. **Install dependencies**
|
|
|
|
```bash
|
|
brew install jq
|
|
```
|
|
|
|
### Usage
|
|
|
|
**From main towerops directory:**
|
|
```bash
|
|
# Auto-detect project and monitor
|
|
./scripts/monitor-deploy.sh
|
|
|
|
# Or explicitly specify
|
|
./scripts/monitor-deploy.sh app
|
|
```
|
|
|
|
**From towerops-agent directory:**
|
|
```bash
|
|
# Auto-detect and monitor agent
|
|
../scripts/monitor-deploy.sh
|
|
|
|
# Or explicitly specify
|
|
../scripts/monitor-deploy.sh agent
|
|
```
|
|
|
|
**After git push:**
|
|
```bash
|
|
git push && ./scripts/monitor-deploy.sh
|
|
```
|
|
|
|
### What it shows
|
|
|
|
**Pipeline Status:**
|
|
- ✓ Success (green)
|
|
- ✗ Failed (red)
|
|
- → Running (blue)
|
|
- ⏳ Pending (yellow)
|
|
|
|
**For each job:**
|
|
- Stage (test, build, release)
|
|
- Status icon
|
|
- Job name
|
|
- Duration
|
|
|
|
**For main app on main branch:**
|
|
- Triggers Kubernetes rollout restart
|
|
- Monitors deployment progress
|
|
- Shows pod status
|
|
- Displays current image version
|
|
|
|
### Example Output
|
|
|
|
```
|
|
╔════════════════════════════════════════════════════════════════╗
|
|
║ Towerops - CI/CD Monitor ║
|
|
╚════════════════════════════════════════════════════════════════╝
|
|
|
|
Branch: main
|
|
Pipeline: #12345
|
|
Status: → running
|
|
|
|
Jobs:
|
|
[test] ✓ test 45s
|
|
[build] → build:main -
|
|
[release] ⏳ release:multiarch -
|
|
|
|
Refreshing in 5s... (Ctrl+C to stop)
|
|
```
|
|
|
|
### Troubleshooting
|
|
|
|
**"GITLAB_ACCESS_TOKEN not set"**
|
|
- Export your GitLab personal access token
|
|
|
|
**"jq is required"**
|
|
- Install: `brew install jq`
|
|
|
|
**"No pipeline found"**
|
|
- Push to trigger a new pipeline
|
|
- Check that you're on the correct branch
|
|
|
|
**"Deployment not found"**
|
|
- Only applies to main app on main branch
|
|
- Verify kubectl context: `/opt/homebrew/bin/kubectl config current-context`
|
|
|
|
### Project Detection
|
|
|
|
The script auto-detects the project from git remote URL:
|
|
- URL contains "towerops-agent" → Agent
|
|
- URL contains "towerops" → Main app
|
|
|
|
Manual override: `./scripts/monitor-deploy.sh [app|agent]`
|
|
|
|
### GitLab Project IDs
|
|
|
|
- Main app: `graham/towerops`
|
|
- Agent: `towerops/towerops-agent`
|
|
|
|
Update these in the script if your project paths differ.
|