towerops-agent/CLAUDE.md

Towerops Agent - Development Notes

This file provides context for Claude Code when working on the Rust agent.

Project Overview

Lightweight Rust agent for remote SNMP polling. Deployed on customer networks to poll local SNMP devices and report metrics to Towerops API via HTTPS.

What's Complete ✅

Architecture & Design

• Complete module structure (12 source files)
• Configuration types matching Phoenix API responses
• Metric types (SensorReading, InterfaceStat) with proper serialization
• Event loop with 5 concurrent tasks (tokio::select!)
• SQLite buffering with 24-hour retention
• Error types and result handling throughout

Core Functionality

• API Client (api_client.rs)

  • fetch_config() - GET /api/v1/agent/config (Protocol Buffers)
  • submit_metrics() - POST /api/v1/agent/metrics (Protocol Buffers)
  • heartbeat() - POST /api/v1/agent/heartbeat (Protocol Buffers)
  • Uses ureq with rustls-tls (30s timeout)
  • Full Protocol Buffers integration for all endpoints

• Storage (buffer/storage.rs)

  • store_metric() - Save metrics to SQLite
  • get_pending_metrics() - Retrieve unsent metrics
  • mark_metrics_sent() - Track submission
  • cleanup_old_metrics() - Remove old data
  • Last poll time tracking per equipment

• Scheduler (poller/scheduler.rs)

  • Config refresh every 5 minutes
  • Metrics flush every 30 seconds
  • Heartbeat every 60 seconds
  • Cleanup every hour
  • Poll check every 5 seconds

• Executor (poller/executor.rs)

  • poll_sensors() - Poll configured sensors
  • poll_interfaces() - Poll interface statistics
  • Parallel polling with tokio::join!
  • Applies sensor divisors

• Main (main.rs)

  • CLI with clap (--api-url, --token, etc.)
  • Environment variable support
  • Logging with tracing
  • Graceful startup/shutdown

Protocol Buffers Integration

• Protobuf Definitions (proto/agent.proto)
  • AgentConfig, Equipment, SnmpConfig, Sensor, Interface
  • MetricBatch, Metric, SensorReading, InterfaceStat
  • HeartbeatMetadata, HeartbeatResponse
• Code Generation (build.rs)
  • Uses prost-build to compile protobuf definitions
  • Generates Rust types at build time
• API Communication
  • Config endpoint: Accepts application/x-protobuf, decodes response
  • Metrics endpoint: Encodes batch to protobuf, sends with proper content-type
  • Heartbeat endpoint: Encodes metadata to protobuf
  • Conversion functions between protobuf and internal types

Build & Deployment

• Cargo.toml with optimized release profile
  • opt-level = "z"
  • lto = true
  • codegen-units = 1
  • strip = true
• Multi-stage Dockerfile (Alpine, ~10-20 MB)
• docker-compose.example.yml
• README with user documentation
• .gitignore and .dockerignore
• GitLab CI/CD configured for Docker Hub

Build Status

✅ cargo build --release - SUCCESS
✅ cargo clippy - 0 warnings, 0 errors
📦 Target size optimized for minimal footprint
🚀 Protobuf integration complete

What's Incomplete 🔄

CRITICAL: SNMP Library Integration

Location: src/snmp/client.rs

Problem: The snmp crate v0.2 API is different than initially designed. Current implementation:

• Validates IP addresses
• Returns error for actual SNMP operations
• Compiles successfully but doesn't poll

Current Code:

pub async fn get(...) -> SnmpResult<SnmpValue> {
    // Validates IP
    // Returns: "SNMP implementation incomplete"
}

pub async fn walk(...) -> SnmpResult<Vec<(String, SnmpValue)>> {
    // Validates IP
    // Returns: empty vec
}

What's Needed:

1. Research the correct snmp crate 0.2 API:

cargo doc --open --package snmp

2. Implement actual SNMP operations:

   • Create session with proper parameters
   • Perform GET request for single OID
   • Perform GETNEXT/WALK for OID subtrees
   • Handle response parsing
   • Map errors (timeout, auth failure, etc.)

3. Key questions to answer:

   • How to create SyncSession? (signature changed)
   • How to send GET request? (method name/signature)
   • How to send GETNEXT? (for walk implementation)
   • How to parse responses? (varbinds format)
   • How to handle errors? (error enum changed)

4. Reference implementation:

   • Check snmp crate examples directory
   • Look at crate tests for usage patterns
   • May need to read source code directly

Alternatives if snmp v0.2 doesn't work:

• snmp-mp crate (more actively maintained)
• snmp-parser crate (lower level, more control)
• rasn-snmp crate (ASN.1 based)

Estimated Effort: 4-8 hours

Testing Gaps

• Unit tests for SNMP client (depends on SNMP integration)
• Unit tests for storage (SQLite operations)
• Unit tests for API client (mock server)
• Integration test with real SNMP device

Development Workflow

Quick Start

1. Build the agent:

cargo build --release

2. Run locally (needs Phoenix backend running):

cargo run -- \
  --api-url http://localhost:4000 \
  --token <get-from-ui> \
  --database-path ./test.db

3. Watch logs:

RUST_LOG=debug cargo run -- ...

Testing Changes

1. Check compilation:

cargo check

2. Run tests:

cargo test

3. Format code:

cargo fmt

4. Check for issues:

cargo clippy

Docker Testing

1. Build image:

docker build -t towerops-agent:test .

2. Run container:

docker run --rm \
  -e TOWEROPS_API_URL=http://host.docker.internal:4000 \
  -e TOWEROPS_AGENT_TOKEN=<token> \
  -e RUST_LOG=info \
  -v $(pwd)/data:/data \
  towerops-agent:test

CI/CD Pipeline

Automated builds via GitLab CI (.gitlab-ci.yml):

• Push to branch → test + build with branch tag
• Push to main → test + build + tag as latest
• Create tag (e.g., v0.1.0) → test + build + release

Registry: registry.gitlab.com/towerops/towerops-agent

See: DEPLOYMENT.md for complete CI/CD documentation

Integration with Phoenix Backend

API Endpoints (from agent perspective)

GET /api/v1/agent/config

• Headers: Authorization: Bearer <token>
• Response: Equipment list with sensors and interfaces
• Called every 5 minutes

POST /api/v1/agent/metrics

• Headers: Authorization: Bearer <token>
• Body: {"metrics": [...]}
• Response: {"status": "accepted", "received": N}
• Called every 30 seconds with pending metrics

POST /api/v1/agent/heartbeat

• Headers: Authorization: Bearer <token>
• Body: {"version": "0.1.0", "hostname": "...", "uptime_seconds": 3600}
• Response: {"status": "ok"}
• Called every 60 seconds

Getting a Test Token

1. Start Phoenix: mix phx.server
2. Navigate to: http://localhost:4000/orgs/:slug/agents
3. Click "Create New Agent"
4. Copy the token (shown only once)
5. Use in agent: --token <copied-token>

Architecture Decisions

Why Tokio?

• Async event loop for efficient I/O
• Multiple concurrent timers (config, metrics, heartbeat)
• Non-blocking SNMP operations via spawn_blocking

Why SQLite?

• Embedded, no external dependencies
• Persist metrics during API outages
• Small footprint (~100 MB for 24h of metrics)
• No configuration needed

Why Rust?

• Small binary size (~10-20 MB with Alpine)
• Low memory usage (<256 MB typical)
• Cross-compile to multiple architectures
• Strong type safety for reliability

Why Simplified SNMP?

• The snmp crate v0.2 API required research
• Chose to ship architecture first, SNMP second
• Allows testing of everything except SNMP polling
• Can be completed independently

Common Issues

"SNMP implementation incomplete" Error

Expected: This is the TODO for SNMP integration. The agent runs but can't poll devices yet.

"Failed to fetch config" Error

Check:

1. Is Phoenix backend running?
2. Is the token valid (not revoked)?
3. Is the API URL correct?
4. Is there network connectivity?

High Memory Usage

Check:

1. Database size: ls -lh /data/towerops-agent.db
2. Are metrics being submitted? (check logs)
3. Is cleanup running? (should see log every hour)

Agent Not Showing as Online

Check:

1. Is heartbeat working? (check Phoenix logs)
2. Check last_seen_at in database: SELECT last_seen_at FROM agent_tokens WHERE token_hash = ...
3. Time sync between agent and server

File Organization

towerops-agent/
├── src/
│   ├── main.rs              # Entry point, CLI, initialization
│   ├── config.rs            # Types matching API responses
│   ├── api_client.rs        # HTTP client for Towerops API
│   ├── metrics/
│   │   └── mod.rs          # Metric types (SensorReading, InterfaceStat)
│   ├── snmp/
│   │   ├── mod.rs          # Module exports
│   │   ├── client.rs       # ⚠️ INCOMPLETE - needs SNMP integration
│   │   └── types.rs        # SNMP types and errors
│   ├── buffer/
│   │   ├── mod.rs          # Module exports
│   │   └── storage.rs      # SQLite buffering
│   └── poller/
│       ├── mod.rs          # Module exports
│       ├── executor.rs     # Poll execution logic
│       └── scheduler.rs    # Main event loop
├── Cargo.toml              # Dependencies and build config
├── Dockerfile              # Multi-stage build
├── README.md               # User documentation
└── CLAUDE.md              # This file

Dependencies

Key Crates:

• tokio - Async runtime with full features
• reqwest - HTTP client (rustls-tls, no default features)
• rusqlite - SQLite (bundled)
• serde + serde_json - Serialization
• snmp - SNMP operations (v0.2) ⚠️ needs integration
• tracing + tracing-subscriber - Logging
• clap - CLI argument parsing
• chrono - Timestamps
• anyhow + thiserror - Error handling
• hostname - Get system hostname

Next Actions

Immediate (to make agent functional):

1. Research snmp crate v0.2 API documentation
2. Update src/snmp/client.rs with working implementation
3. Test with a real SNMP device or simulator
4. Add unit tests for SNMP operations

Short-term (for production readiness):

1. Add more comprehensive unit tests
2. Integration test with mock SNMP device
3. Load test with 100 devices
4. Stability test (7+ days continuous)

Long-term (nice to have):

1. SNMPv3 support
2. Agent-side threshold filtering
3. Configurable sampling rates
4. Agent health metrics endpoint

Resources

• Main Implementation Doc: /Users/graham/dev/towerops/AGENT_IMPLEMENTATION.md
• Next Steps Guide: /Users/graham/dev/towerops/AGENT_NEXT_STEPS.md
• SNMP Crate Docs: https://docs.rs/snmp/0.2.2/snmp/
• SNMP Crate Source: https://github.com/hroi/snmp-rs

Success Criteria

Agent is production-ready when:

• Compiles successfully
• Docker image builds
• API client works (config, metrics, heartbeat)
• SQLite buffering works
• Event loop runs without panics
• SNMP polling works ← ONLY REMAINING ITEM
• Metrics appear in Phoenix database
• Survives 24h API outage
• Uses <256 MB memory with 50 devices
• Runs for 7+ days without issues

Notes for Future Development

Adding New Metric Types

1. Add variant to Metric enum in src/metrics/mod.rs
2. Update metric_type() and timestamp() methods
3. Update Phoenix API to accept new type
4. Add serialization test

Adding New Configuration Fields

1. Update structs in src/config.rs
2. Update Phoenix API build_equipment_config/1
3. Consider backwards compatibility

Debugging SNMP Issues

• Set RUST_LOG=debug to see all SNMP operations
• Check IP reachability: ping <device-ip>
• Test SNMP manually: snmpget -v2c -c public <device-ip> <oid>
• Verify community string is correct
• Check firewall rules (UDP port 161)

---

Last Updated: January 9, 2026 Status: Architecture complete, SNMP integration pending Version: 0.1.0 (pre-release)