towerops/AGENT_NEXT_STEPS.md

Agent System - Next Steps

This document outlines the remaining work to complete the remote SNMP polling agent system.

Current Status

✅ Complete:

• Backend API with token authentication
• Agent management UI
• Hierarchical agent assignment (Equipment > Site > Organization)
• Organization/site/equipment-level configuration
• Rust agent architecture (compiles successfully)
• SNMP client fully implemented (GET and WALK operations)
• SQLite buffering
• Docker deployment files
• Protocol Buffers integration for all API endpoints
• 775 tests passing

Critical Path to Production

1. Integration Testing (Priority: HIGH) ⬅️ START HERE

Prerequisites: ✅ All backend code complete

Test Environment Setup:

1. Deploy test SNMP device (or use simulator)
2. Deploy Phoenix backend
3. Deploy Rust agent via Docker
4. Create agent token via UI
5. Assign test equipment to agent

Test Cases:

• Agent connects and authenticates
• Agent fetches configuration
• Agent polls sensors successfully
• Agent polls interfaces successfully
• Metrics appear in Phoenix database
• Threshold violations trigger events
• Agent survives API outage (test 24h buffering)
• Agent reconnects after network issues
• Heartbeat updates agent status

Duration: 1-2 days

2. Load & Performance Testing (Priority: MEDIUM)

Test Scenarios:

Scenario 1: Single Agent, Many Devices

• 100 devices
• 500 sensors
• 200 interfaces
• 60-second poll interval
• Monitor: Memory, CPU, database growth

Scenario 2: Many Agents

• 10 agents
• 10 devices each
• Concurrent polling
• Monitor: API performance, database connections

Scenario 3: Long-Running Stability

• 1 agent
• 50 devices
• Run for 7 days
• Monitor: Memory leaks, database size, errors

Duration: 3-5 days

3. Production Readiness (Priority: MEDIUM)

Docker Image:

• Build optimized image (11.8 MB)
• Publish to container registry
• Tag releases (v0.1.0, latest)
• Multi-architecture support (amd64, arm64)

Documentation:

• User guide with deployment methods (USER_GUIDE.md)
• Troubleshooting guide
• Network/firewall requirements
• Upgrade procedure
• Disaster recovery

Monitoring:

• Grafana dashboard for agent health
• Alerts for offline agents
• Metrics on agent performance
• Database queries for agent statistics (lib/towerops/agents/stats.ex)

Duration: 2-3 days

Optional Enhancements

4. Advanced Features (Priority: LOW)

SNMPv3 Support

• User authentication
• Privacy encryption
• Configuration UI changes

Agent-Side Filtering

• Threshold checks in agent
• Reduce bandwidth for high-sensor-count equipment
• Configurable sampling rates

Equipment Polling Modes

• Add polling_mode enum: :server, :agent, :both
• Allows gradual migration
• Enables A/B testing

Agent Health Metrics

• CPU/memory usage
• Metrics queue depth
• Polling success rate
• Average latency per device

Quick Start Guide

For Development

1. Test Locally

# Terminal 1: Start Phoenix
mix phx.server

# Terminal 2: Start agent
cd towerops-agent
cargo run -- \
  --api-url http://localhost:4000 \
  --token <your-test-token>

2. Create Test Agent

• Visit http://localhost:4000/orgs/:slug/agents
• Create agent, copy token
• Use token in step 1

For Production Deployment

1. Build Docker Image

cd towerops-agent
docker build -t towerops/agent:0.1.0 .
docker tag towerops/agent:0.1.0 towerops/agent:latest

2. Push to Registry

docker push towerops/agent:0.1.0
docker push towerops/agent:latest

3. Deploy to Customer

# Provide customer with docker-compose.yml
# They run:
docker-compose up -d

Estimated Timeline

Phase	Effort	Duration
✅ SNMP Integration	4-8 hours	1 day
✅ Hierarchical Agent Assignment	12 hours	1.5 days
Integration Testing	16 hours	2 days
Load Testing	24 hours	3 days
Production Prep	16 hours	2 days
Remaining	56 hours	7 days

Success Criteria

The agent system is ready for production when:

• Agent authenticates with token successfully
• Agent fetches configuration from API
• Agent polls SNMP devices (sensors + interfaces)
• Metrics appear in database within 60 seconds
• Threshold violations trigger events
• Agent survives 24h API outage without data loss
• UI shows agent status (online/offline)
• Token revocation works immediately
• Agent uses <256 MB memory with 50 devices
• Docker image is <50 MB
• Load test: 100 devices, 500 sensors, 200 interfaces
• Stability test: 7 days continuous operation

Questions & Decisions Needed

1. SNMP Library Choice: Stick with snmp v0.2 or switch?
2. Release Strategy: Beta testing with select customers first?
3. Support Plan: How will customers get help with agent deployment?
4. Monitoring: What metrics should we track about agent usage?
5. Pricing: Does remote agent feature affect pricing tiers?

Resources

• Implementation Doc: /Users/graham/dev/towerops/AGENT_IMPLEMENTATION.md
• Agent README: /Users/graham/dev/towerops/towerops-agent/README.md
• SNMP Crate Docs: https://docs.rs/snmp/0.2.2/snmp/
• Original Plan: /Users/graham/.claude/plans/melodic-coalescing-truffle.md

Contact

For questions about this implementation:

• Review AGENT_IMPLEMENTATION.md for architecture details
• Check git history for context on specific changes
• All code follows project conventions in CLAUDE.md