Implemented two critical optimizations for handling large equipment counts:
1. **Concurrent polling limiter**: Added semaphore to limit concurrent
SNMP polling tasks to 100 at a time, preventing system overload when
polling 10,000+ devices simultaneously.
2. **Batched metrics flushing**: Increased batch size from 100 to 500
metrics and added loop to process up to 10,000 metrics per flush cycle
(20 batches × 500). Prevents metric backlog with high-volume polling.
Performance characteristics:
- 10,000 equipment with 5 sensors each = 50,000 metrics per poll cycle
- Flush cycle handles 10,000 metrics every 30 seconds
- Concurrent polling processes 100 devices at a time instead of unlimited
System resource usage remains bounded regardless of equipment count.
Replaced all .unwrap() calls on Mutex::lock() with proper error handling
using map_err to prevent panics when the mutex is poisoned.
This prevents silent crashes in production when any thread panics while
holding the storage mutex lock.
- Modified scheduler to poll equipment items concurrently using tokio::spawn
- Each equipment item now polls in its own async task
- Added Clone derives to Executor and SnmpClient to support parallel execution
- Sensors and interfaces within each equipment still poll in parallel via tokio::join!
- All tasks are awaited to ensure completion before returning
Performance improvement: Multiple devices can now be polled simultaneously
instead of sequentially, significantly reducing total polling time for
agents monitoring many devices.
Added /health HTTP endpoint on port 8080 that returns:
- Agent status and version
- Uptime in seconds
- Pending metrics count
- Last error (if any)
Implementation:
- Uses lightweight tiny_http server in background thread
- Non-blocking health checks
- Returns JSON for easy integration with monitoring tools
- Ready for Kubernetes liveness/readiness probes
Example response:
{
"status": "healthy",
"version": "0.1.0",
"uptime_seconds": 3600,
"config_last_fetch": "2026-01-14T23:00:00Z",
"metrics_pending": 0,
"last_error": null
}
The agent now checks for updates every hour and automatically updates
itself when a new version is available on Docker Hub.
Features:
- Periodic update checks (hourly via scheduler)
- Automatic Docker image pull when update available
- Graceful exit and restart with new version
- Non-blocking, runs in background task
- Requires Docker socket mount for self-update
Changes:
- Add UpdateInfo struct and get_update_info() function
- Add perform_self_update() to pull new image and restart
- Add update check ticker to scheduler (hourly)
- Include docker-cli in Dockerfile runtime stage
- Update docker-compose.example.yml with socket mount
- Update README and CLAUDE.md with auto-update docs
The agent will log update status:
- "Already running latest version" - no action
- "Performing self-update: X -> Y" - pulling new image
- "Exiting to allow restart with new version" - restarting
Requires:
- Docker socket mounted: /var/run/docker.sock:/var/run/docker.sock
- restart: unless-stopped in docker-compose (to restart after exit)
The agent now checks Docker Hub on startup to see if a newer version
is available and logs a warning if an update is detected.
Changes:
- Add version.rs module to query Docker Hub API
- Check for newer versions on startup (non-blocking)
- Log warning with docker pull command if update available
- Gracefully handle Docker Hub API failures
- Update CLAUDE.md with version checking documentation
- Update fetch_config to use protobuf instead of JSON
- Add conversion functions from protobuf types to internal config types
- Update heartbeat to use protobuf encoding
- All API endpoints now use Protocol Buffers (config, metrics, heartbeat)
- Reduces 7 unused struct warnings to 1 (HeartbeatResponse)
