towerops/PROFILES.md

# Device Profile Management

This document describes how to manage SNMP device profiles in Towerops. Device profiles define how to discover and monitor different types of network equipment via SNMP.

## Overview

Towerops uses a database-driven profile system that supports 671+ device types dynamically without code changes. Profiles are imported from external YAML definitions and stored in the database.

## Architecture

**Profile Discovery Flow:**
1. Device performs SNMP discovery
2. System reads sysDescr and sysObjectID from device
3. Matches against detection rules in database
4. Uses matched profile to discover sensors, CPU, memory, etc.
5. Saves discovered data for monitoring and graphing

**Profile Components:**
- **Detection Rules**: sysObjectID prefix matching, sysDescr regex patterns
- **Sensor Definitions**: Temperature, voltage, current, power, fan speed, state sensors
- **Processor Definitions**: CPU usage monitoring (converted to percent sensors)
- **Memory Pool Definitions**: Memory usage monitoring
- **OS Definitions**: Firmware version, serial number, location data

## Exporting Profiles Locally

Use the `mix export_profiles` task to export profiles from an external source to JSON format:

```bash
# Export all profiles
mix export_profiles --librenms-path ~/dev/librenms --output profiles.json

# Export specific profiles only
mix export_profiles --librenms-path ~/dev/librenms --profiles epmp,unifi,airos-af --output profiles.json
```

**Options:**
- `--librenms-path PATH` - Path to external source installation (required)
- `--output PATH` - Output JSON file path (default: profiles.json)
- `--profiles LIST` - Comma-separated list of specific profiles to export (optional)

**Output:**
The task generates a JSON file containing all profile data (detection rules, sensor definitions, etc.) suitable for uploading to production.

## Importing Profiles to Production

**Requirements:**
- Superuser account (user with `is_superuser = true`)
- API token created by superuser account

### Step 1: Create Superuser API Token

1. Log in to Towerops as a superuser
2. Navigate to your organization settings
3. Create a new API token
4. Copy the token (shown only once) - it will start with `towerops_`

**Note**: Only API tokens created by superuser accounts can import device profiles.

### Step 2: Upload Profiles

```bash
curl -X POST https://towerops.net/api/v1/profiles/import \
  -H "Authorization: Bearer YOUR_SUPERUSER_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d @profiles.json
```

**Response:**
```json
{
  "status": "queued",
  "job_id": "uuid",
  "profile_count": 15,
  "message": "Import job queued successfully. 15 profiles will be imported in the background."
}
```

### Step 3: Monitor Import Progress

The import runs as a background job in the `maintenance` queue. Monitor progress via:

1. **Server logs:**
   ```bash
   kubectl logs -n towerops deployment/towerops | grep "profile import"
   ```

2. **Exq dashboard:**
   Navigate to `/dashboard` and check the Maintenance queue

**Log messages:**
- `Starting profile import job <job_id> with N profiles` - Job started
- `[<job_id>] Successfully imported profile: <os>` - Profile imported
- `[<job_id>] Failed to import profile <os>: <reason>` - Import error
- `[<job_id>] Import complete: N succeeded, M failed` - Job finished

## Profile Structure

### Detection YAML (os_detection/*)
```yaml
os: epmp
text: Cambium epmp
type: wireless
group: cambium
icon: cambium
discovery:
  - sysObjectID:
      - ".1.3.6.1.4.1.17713.21"
```

### Discovery YAML (os_discovery/*)
```yaml
mib: CAMBIUM-PMP80211-MIB
modules:
  os:
    version: CAMBIUM-PMP80211-MIB::cambiumCurrentuImageVersion.0
    serial: CAMBIUM-PMP80211-MIB::cambiumEPMPMSN.0

  processors:
    data:
      - oid: sysCPUUsage
        num_oid: ".1.3.6.1.4.1.17713.21.2.1.64.{{ $index }}"
        type: cambium-cpu
        precision: 10

  sensors:
    temperature:
      data:
        - oid: boardTemp
          num_oid: ".1.3.6.1.4.1.17713.21.1.2.1.0"
          descr: Board Temperature
          precision: 10
```

## Database Schema

**device_profiles** - Main profile table
- `os` - Operating system identifier (e.g., "epmp", "unifi")
- `text` - Display name (e.g., "Cambium epmp")
- `type` - Device type (wireless, switch, router, etc.)
- `group` - Manufacturer group (cambium, ubiquiti, cisco, etc.)
- `priority` - Matching priority (lower = higher priority)
- `enabled` - Whether profile is active

**profile_detection_rules** - Device detection patterns
- `rule_type` - "sysObjectID", "sysDescr", "sysDescr_regex"
- `pattern` - OID prefix or regex pattern
- `value` - Exact match value

**profile_sensor_definitions** - Sensor discovery definitions
- `sensor_class` - temperature, voltage, current, power, fanspeed, humidity, dbm, snr, percent, count, frequency, state
- `oid` - Symbolic OID name
- `num_oid` - Numeric OID (e.g., ".1.3.6.1.4.1.17713.21.2.1.36.{{ $index }}")
- `descr` - Sensor description
- `divisor` - Value divisor for unit conversion
- `precision` - Number of decimal places (stored in divisor field)

**profile_processor_definitions** - CPU discovery definitions
- `oid` - Symbolic OID name
- `num_oid` - Numeric OID
- `precision` - Divisor for CPU percentage (e.g., 10 means divide by 10)
- `type` - Processor type identifier

## Force Replacement

All profile imports use force replacement - existing profiles with the same `os` name are deleted before importing. This ensures clean updates without conflicts.

## Troubleshooting

**Import fails immediately:**
- Check you're logged in as a superuser (`is_superuser = true`)
- Verify session cookie is correct and not expired
- Check JSON format is valid: `jq . profiles.json`

**Profiles import but discovery doesn't work:**
- Check profile detection rules match device sysObjectID/sysDescr
- Verify sensor OIDs are numeric (not symbolic MIB names)
- Check logs for SNMP walk errors
- Test SNMP manually: `snmpwalk -v2c -c public <ip> <oid>`

**Sensor indices conflict:**
- Sensor indices must be unique per device
- Scalar sensors (index .0) use descriptive names (e.g., "dfs_detected_count")
- Table sensors use class prefix (e.g., "temperature_1", "voltage_2")

**CPU not showing:**
- Processor definitions are converted to "percent" type sensors
- Check divisor/precision is correct (usually 10 or 100)
- Verify num_oid is accessible via SNMP

## Examples

### Export ePMP and UniFi profiles
```bash
mix export_profiles \
  --librenms-path ~/dev/librenms \
  --profiles epmp,unifi \
  --output production_profiles.json
```

### Import to production
```bash
# Use your superuser API token
SUPERUSER_TOKEN="towerops_abc123..."

curl -X POST https://towerops.net/api/v1/profiles/import \
  -H "Authorization: Bearer ${SUPERUSER_TOKEN}" \
  -H "Content-Type: application/json" \
  -d @production_profiles.json
```

### Check import status
```bash
kubectl logs -n towerops deployment/towerops --tail=100 | grep "Import complete"
```

## Related Files

- `lib/mix/tasks/export_profiles.ex` - Export task
- `lib/mix/tasks/import_profiles.ex` - Local import task (for development)
- `lib/towerops/device_profiles/importer.ex` - Profile import logic
- `lib/towerops/snmp/profiles/dynamic.ex` - Dynamic profile discovery engine
- `lib/towerops/workers/profile_import_worker.ex` - Background import worker
- `lib/towerops_web/controllers/api/v1/profiles_controller.ex` - API endpoint
- `lib/towerops_web/plugs/require_superuser.ex` - Superuser authorization

## Security

- Profile import requires superuser API token
- Only tokens created by superusers can import profiles
- User who created the token must still be a superuser
- Runs in background to prevent timeouts
- Validates profile data before importing
- Logs all import operations with user email

## Supported Profile Count

As of January 2026, the system supports **671+ device profiles** including:
- Cambium ePMP, PMP, cnPilot
- Ubiquiti AirFiber, airMAX, UniFi
- Cisco IOS, IOS-XE, NX-OS
- MikroTik RouterOS
- Juniper JunOS
- And many more...

New profiles can be added without code changes by exporting and importing.