docs: add comprehensive profile management documentation
Document the complete workflow for exporting and importing device profiles: - Local export process with mix task - Production import via API endpoint - Profile structure and database schema - Troubleshooting guide - Security and authentication details File location: PROFILES.md (intentionally not in API docs)
This commit is contained in:
parent
e0bcd4feda
commit
2f7f6370e3
1 changed files with 244 additions and 0 deletions
244
PROFILES.md
Normal file
244
PROFILES.md
Normal file
|
|
@ -0,0 +1,244 @@
|
|||
# Device Profile Management
|
||||
|
||||
This document describes how to manage SNMP device profiles in Towerops. Device profiles define how to discover and monitor different types of network equipment via SNMP.
|
||||
|
||||
## Overview
|
||||
|
||||
Towerops uses a database-driven profile system that supports 671+ device types dynamically without code changes. Profiles are imported from external YAML definitions and stored in the database.
|
||||
|
||||
## Architecture
|
||||
|
||||
**Profile Discovery Flow:**
|
||||
1. Device performs SNMP discovery
|
||||
2. System reads sysDescr and sysObjectID from device
|
||||
3. Matches against detection rules in database
|
||||
4. Uses matched profile to discover sensors, CPU, memory, etc.
|
||||
5. Saves discovered data for monitoring and graphing
|
||||
|
||||
**Profile Components:**
|
||||
- **Detection Rules**: sysObjectID prefix matching, sysDescr regex patterns
|
||||
- **Sensor Definitions**: Temperature, voltage, current, power, fan speed, state sensors
|
||||
- **Processor Definitions**: CPU usage monitoring (converted to percent sensors)
|
||||
- **Memory Pool Definitions**: Memory usage monitoring
|
||||
- **OS Definitions**: Firmware version, serial number, location data
|
||||
|
||||
## Exporting Profiles Locally
|
||||
|
||||
Use the `mix export_profiles` task to export profiles from an external source to JSON format:
|
||||
|
||||
```bash
|
||||
# Export all profiles
|
||||
mix export_profiles --librenms-path ~/dev/librenms --output profiles.json
|
||||
|
||||
# Export specific profiles only
|
||||
mix export_profiles --librenms-path ~/dev/librenms --profiles epmp,unifi,airos-af --output profiles.json
|
||||
```
|
||||
|
||||
**Options:**
|
||||
- `--librenms-path PATH` - Path to external source installation (required)
|
||||
- `--output PATH` - Output JSON file path (default: profiles.json)
|
||||
- `--profiles LIST` - Comma-separated list of specific profiles to export (optional)
|
||||
|
||||
**Output:**
|
||||
The task generates a JSON file containing all profile data (detection rules, sensor definitions, etc.) suitable for uploading to production.
|
||||
|
||||
## Importing Profiles to Production
|
||||
|
||||
**Requirements:**
|
||||
- Superuser account (user with `is_superuser = true`)
|
||||
- Active browser session
|
||||
- Session cookie from browser
|
||||
|
||||
### Step 1: Get Session Cookie
|
||||
|
||||
1. Log in to Towerops as a superuser
|
||||
2. Open browser Developer Tools (F12)
|
||||
3. Navigate to: Application > Cookies > https://towerops.net
|
||||
4. Copy the value of `_towerops_key` cookie
|
||||
|
||||
### Step 2: Upload Profiles
|
||||
|
||||
```bash
|
||||
curl -X POST https://towerops.net/api/v1/admin/profiles/import \
|
||||
-H "Cookie: _towerops_key=YOUR_SESSION_COOKIE" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d @profiles.json
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"status": "queued",
|
||||
"job_id": "uuid",
|
||||
"profile_count": 15,
|
||||
"message": "Import job queued successfully. 15 profiles will be imported in the background."
|
||||
}
|
||||
```
|
||||
|
||||
### Step 3: Monitor Import Progress
|
||||
|
||||
The import runs as a background job in the `maintenance` queue. Monitor progress via:
|
||||
|
||||
1. **Server logs:**
|
||||
```bash
|
||||
kubectl logs -n towerops deployment/towerops | grep "profile import"
|
||||
```
|
||||
|
||||
2. **Exq dashboard:**
|
||||
Navigate to `/dashboard` and check the Maintenance queue
|
||||
|
||||
**Log messages:**
|
||||
- `Starting profile import job <job_id> with N profiles` - Job started
|
||||
- `[<job_id>] Successfully imported profile: <os>` - Profile imported
|
||||
- `[<job_id>] Failed to import profile <os>: <reason>` - Import error
|
||||
- `[<job_id>] Import complete: N succeeded, M failed` - Job finished
|
||||
|
||||
## Profile Structure
|
||||
|
||||
### Detection YAML (os_detection/*)
|
||||
```yaml
|
||||
os: epmp
|
||||
text: Cambium epmp
|
||||
type: wireless
|
||||
group: cambium
|
||||
icon: cambium
|
||||
discovery:
|
||||
- sysObjectID:
|
||||
- ".1.3.6.1.4.1.17713.21"
|
||||
```
|
||||
|
||||
### Discovery YAML (os_discovery/*)
|
||||
```yaml
|
||||
mib: CAMBIUM-PMP80211-MIB
|
||||
modules:
|
||||
os:
|
||||
version: CAMBIUM-PMP80211-MIB::cambiumCurrentuImageVersion.0
|
||||
serial: CAMBIUM-PMP80211-MIB::cambiumEPMPMSN.0
|
||||
|
||||
processors:
|
||||
data:
|
||||
- oid: sysCPUUsage
|
||||
num_oid: ".1.3.6.1.4.1.17713.21.2.1.64.{{ $index }}"
|
||||
type: cambium-cpu
|
||||
precision: 10
|
||||
|
||||
sensors:
|
||||
temperature:
|
||||
data:
|
||||
- oid: boardTemp
|
||||
num_oid: ".1.3.6.1.4.1.17713.21.1.2.1.0"
|
||||
descr: Board Temperature
|
||||
precision: 10
|
||||
```
|
||||
|
||||
## Database Schema
|
||||
|
||||
**device_profiles** - Main profile table
|
||||
- `os` - Operating system identifier (e.g., "epmp", "unifi")
|
||||
- `text` - Display name (e.g., "Cambium epmp")
|
||||
- `type` - Device type (wireless, switch, router, etc.)
|
||||
- `group` - Manufacturer group (cambium, ubiquiti, cisco, etc.)
|
||||
- `priority` - Matching priority (lower = higher priority)
|
||||
- `enabled` - Whether profile is active
|
||||
|
||||
**profile_detection_rules** - Device detection patterns
|
||||
- `rule_type` - "sysObjectID", "sysDescr", "sysDescr_regex"
|
||||
- `pattern` - OID prefix or regex pattern
|
||||
- `value` - Exact match value
|
||||
|
||||
**profile_sensor_definitions** - Sensor discovery definitions
|
||||
- `sensor_class` - temperature, voltage, current, power, fanspeed, humidity, dbm, snr, percent, count, frequency, state
|
||||
- `oid` - Symbolic OID name
|
||||
- `num_oid` - Numeric OID (e.g., ".1.3.6.1.4.1.17713.21.2.1.36.{{ $index }}")
|
||||
- `descr` - Sensor description
|
||||
- `divisor` - Value divisor for unit conversion
|
||||
- `precision` - Number of decimal places (stored in divisor field)
|
||||
|
||||
**profile_processor_definitions** - CPU discovery definitions
|
||||
- `oid` - Symbolic OID name
|
||||
- `num_oid` - Numeric OID
|
||||
- `precision` - Divisor for CPU percentage (e.g., 10 means divide by 10)
|
||||
- `type` - Processor type identifier
|
||||
|
||||
## Force Replacement
|
||||
|
||||
All profile imports use force replacement - existing profiles with the same `os` name are deleted before importing. This ensures clean updates without conflicts.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**Import fails immediately:**
|
||||
- Check you're logged in as a superuser (`is_superuser = true`)
|
||||
- Verify session cookie is correct and not expired
|
||||
- Check JSON format is valid: `jq . profiles.json`
|
||||
|
||||
**Profiles import but discovery doesn't work:**
|
||||
- Check profile detection rules match device sysObjectID/sysDescr
|
||||
- Verify sensor OIDs are numeric (not symbolic MIB names)
|
||||
- Check logs for SNMP walk errors
|
||||
- Test SNMP manually: `snmpwalk -v2c -c public <ip> <oid>`
|
||||
|
||||
**Sensor indices conflict:**
|
||||
- Sensor indices must be unique per device
|
||||
- Scalar sensors (index .0) use descriptive names (e.g., "dfs_detected_count")
|
||||
- Table sensors use class prefix (e.g., "temperature_1", "voltage_2")
|
||||
|
||||
**CPU not showing:**
|
||||
- Processor definitions are converted to "percent" type sensors
|
||||
- Check divisor/precision is correct (usually 10 or 100)
|
||||
- Verify num_oid is accessible via SNMP
|
||||
|
||||
## Examples
|
||||
|
||||
### Export ePMP and UniFi profiles
|
||||
```bash
|
||||
mix export_profiles \
|
||||
--librenms-path ~/dev/librenms \
|
||||
--profiles epmp,unifi \
|
||||
--output production_profiles.json
|
||||
```
|
||||
|
||||
### Import to production
|
||||
```bash
|
||||
# Get session cookie from browser first
|
||||
SESSION_COOKIE="SFMyNTY.g3QAAAACbQ..."
|
||||
|
||||
curl -X POST https://towerops.net/api/v1/admin/profiles/import \
|
||||
-H "Cookie: _towerops_key=${SESSION_COOKIE}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d @production_profiles.json
|
||||
```
|
||||
|
||||
### Check import status
|
||||
```bash
|
||||
kubectl logs -n towerops deployment/towerops --tail=100 | grep "Import complete"
|
||||
```
|
||||
|
||||
## Related Files
|
||||
|
||||
- `lib/mix/tasks/export_profiles.ex` - Export task
|
||||
- `lib/mix/tasks/import_profiles.ex` - Local import task (for development)
|
||||
- `lib/towerops/device_profiles/importer.ex` - Profile import logic
|
||||
- `lib/towerops/snmp/profiles/dynamic.ex` - Dynamic profile discovery engine
|
||||
- `lib/towerops/workers/profile_import_worker.ex` - Background import worker
|
||||
- `lib/towerops_web/controllers/api/v1/profiles_controller.ex` - API endpoint
|
||||
- `lib/towerops_web/plugs/require_superuser.ex` - Superuser authorization
|
||||
|
||||
## Security
|
||||
|
||||
- Profile import requires superuser authentication
|
||||
- Uses browser session cookies (not API tokens)
|
||||
- Runs in background to prevent timeouts
|
||||
- Validates profile data before importing
|
||||
- Logs all import operations
|
||||
|
||||
## Supported Profile Count
|
||||
|
||||
As of January 2026, the system supports **671+ device profiles** including:
|
||||
- Cambium ePMP, PMP, cnPilot
|
||||
- Ubiquiti AirFiber, airMAX, UniFi
|
||||
- Cisco IOS, IOS-XE, NX-OS
|
||||
- MikroTik RouterOS
|
||||
- Juniper JunOS
|
||||
- And many more...
|
||||
|
||||
New profiles can be added without code changes by exporting and importing.
|
||||
Loading…
Add table
Reference in a new issue