The previous version checking didn't work because:
- GitLab CI only creates semver tags when pushing Git tags
- Most builds use SHA hash tags, not version tags
- Comparing versions was unreliable
New approach:
- Simply pulls latest tag every hour
- Checks docker pull output to see if image changed
- Only restarts if a new image was actually pulled
- Logs the last_updated timestamp from Docker Hub for visibility
Added comprehensive timestamp formatting to both agent logs and UI:
Rust Agent Logger:
- Added chrono dependency to Cargo.toml
- Updated SimpleLogger to include timestamps in log output
- Format: [2026-01-15 19:45:23.456] [LEVEL] message
- Shows full date, time, and milliseconds for precise log tracking
Phoenix UI Enhancements:
- Added format_datetime/1 - Full date/time with timezone
- Added format_date/1 - Short date format
- Added format_last_seen_with_date/1 - Relative time with full date
- Updated agent show page to display full timestamps in 'Last Seen' card
- Added comprehensive Timestamps section showing:
- Created date (inserted_at)
- Last Updated date (updated_at)
- Last Seen date (last_seen_at with heartbeat context)
- Last IP Address
- Updated agent index page to show full datetime alongside relative time
All timestamps now include both human-readable relative times ('5m ago')
and precise absolute dates for accurate record keeping and debugging.
Implemented two critical optimizations for handling large equipment counts:
1. **Concurrent polling limiter**: Added semaphore to limit concurrent
SNMP polling tasks to 100 at a time, preventing system overload when
polling 10,000+ devices simultaneously.
2. **Batched metrics flushing**: Increased batch size from 100 to 500
metrics and added loop to process up to 10,000 metrics per flush cycle
(20 batches × 500). Prevents metric backlog with high-volume polling.
Performance characteristics:
- 10,000 equipment with 5 sensors each = 50,000 metrics per poll cycle
- Flush cycle handles 10,000 metrics every 30 seconds
- Concurrent polling processes 100 devices at a time instead of unlimited
System resource usage remains bounded regardless of equipment count.
Replaced all .unwrap() calls on Mutex::lock() with proper error handling
using map_err to prevent panics when the mutex is poisoned.
This prevents silent crashes in production when any thread panics while
holding the storage mutex lock.
- Modified scheduler to poll equipment items concurrently using tokio::spawn
- Each equipment item now polls in its own async task
- Added Clone derives to Executor and SnmpClient to support parallel execution
- Sensors and interfaces within each equipment still poll in parallel via tokio::join!
- All tasks are awaited to ensure completion before returning
Performance improvement: Multiple devices can now be polled simultaneously
instead of sequentially, significantly reducing total polling time for
agents monitoring many devices.
Added /health HTTP endpoint on port 8080 that returns:
- Agent status and version
- Uptime in seconds
- Pending metrics count
- Last error (if any)
Implementation:
- Uses lightweight tiny_http server in background thread
- Non-blocking health checks
- Returns JSON for easy integration with monitoring tools
- Ready for Kubernetes liveness/readiness probes
Example response:
{
"status": "healthy",
"version": "0.1.0",
"uptime_seconds": 3600,
"config_last_fetch": "2026-01-14T23:00:00Z",
"metrics_pending": 0,
"last_error": null
}
The agent now checks for updates every hour and automatically updates
itself when a new version is available on Docker Hub.
Features:
- Periodic update checks (hourly via scheduler)
- Automatic Docker image pull when update available
- Graceful exit and restart with new version
- Non-blocking, runs in background task
- Requires Docker socket mount for self-update
Changes:
- Add UpdateInfo struct and get_update_info() function
- Add perform_self_update() to pull new image and restart
- Add update check ticker to scheduler (hourly)
- Include docker-cli in Dockerfile runtime stage
- Update docker-compose.example.yml with socket mount
- Update README and CLAUDE.md with auto-update docs
The agent will log update status:
- "Already running latest version" - no action
- "Performing self-update: X -> Y" - pulling new image
- "Exiting to allow restart with new version" - restarting
Requires:
- Docker socket mounted: /var/run/docker.sock:/var/run/docker.sock
- restart: unless-stopped in docker-compose (to restart after exit)
The agent now checks Docker Hub on startup to see if a newer version
is available and logs a warning if an update is detected.
Changes:
- Add version.rs module to query Docker Hub API
- Check for newer versions on startup (non-blocking)
- Log warning with docker pull command if update available
- Gracefully handle Docker Hub API failures
- Update CLAUDE.md with version checking documentation
The agent was failing with 'no TLS backend is configured' because
ureq was configured with 'default-features = false' which disables
TLS entirely. Removing this restriction allows ureq to use its default
TLS implementation (rustls on supported platforms).
This fixes HTTPS connections to the Towerops API.
The multi-architecture builds (amd64 + arm64) were taking 15-20+ minutes
due to QEMU emulation for ARM64 cross-compilation. This is too slow for
regular development iteration.
Changes:
- Main branch builds: amd64 only (much faster, ~3-5 minutes)
- Tagged releases: Still build multi-arch (amd64 + arm64)
- Added Docker layer caching to speed up subsequent builds
- Cache stored in registry for persistence across CI runs
The 'latest' tag will be amd64-only from main branch. Users needing
ARM64 should use a specific version tag (e.g., v0.1.0).
Fixed docker buildx errors in CI/CD pipeline:
1. Set DOCKER_TLS_CERTDIR to empty string to disable TLS
(buildx with docker-in-docker doesn't work well with TLS certs)
2. Added fallback to reuse existing builder if creation fails
(docker buildx create ... || docker buildx use ...)
This fixes the error:
'could not create a builder instance with TLS data loaded from environment'
The Docker container now handles data directory permissions automatically
without requiring manual user setup.
Changes:
- Added entrypoint.sh script that runs as root, fixes /data permissions,
then drops to non-root user (towerops) using su-exec
- Updated Dockerfile to install su-exec and use the entrypoint script
- Container starts as root but immediately drops privileges after fixing
permissions
The agent will now start successfully with just 'docker-compose up -d'
without users needing to run chown commands manually.
- Document Protocol Buffers integration in its own section
- Update API client description to mention protobuf usage
- Update build status to show only 1 warning (down from 7)
- Add note about GitLab CI/CD Docker Hub deployment
- Update fetch_config to use protobuf instead of JSON
- Add conversion functions from protobuf types to internal config types
- Update heartbeat to use protobuf encoding
- All API endpoints now use Protocol Buffers (config, metrics, heartbeat)
- Reduces 7 unused struct warnings to 1 (HeartbeatResponse)
- Change registry from GitLab to Docker Hub (gmcintire/towerops-agent)
- Update authentication to use DOCKERHUB_USERNAME and DOCKERHUB_TOKEN
- Remove vntx runner tags to use GitLab shared runners
