add universoal import
This commit is contained in:
parent
50d8b27447
commit
9f59e7661a
13 changed files with 1647 additions and 8 deletions
313
LIBRENMS_INTEGRATION.md
Normal file
313
LIBRENMS_INTEGRATION.md
Normal file
|
|
@ -0,0 +1,313 @@
|
|||
# LibreNMS Integration
|
||||
|
||||
This document explains how to import MIB files and device profiles from a LibreNMS installation into Towerops.
|
||||
|
||||
## Quick Start
|
||||
|
||||
Import device profiles from LibreNMS (no authentication needed):
|
||||
|
||||
```bash
|
||||
mix upload_librenms --librenms-path ~/dev/librenms
|
||||
```
|
||||
|
||||
Upload everything including MIBs (requires superuser API token):
|
||||
|
||||
```bash
|
||||
mix upload_librenms --librenms-path ~/dev/librenms --token your-api-token
|
||||
```
|
||||
|
||||
This will:
|
||||
1. Upload all MIB files from `~/dev/librenms/mibs/` to the server (if --token provided)
|
||||
2. Import all device profiles from `~/dev/librenms/resources/definitions/`
|
||||
|
||||
## Options
|
||||
|
||||
### Basic Usage
|
||||
|
||||
```bash
|
||||
# Upload to local server (default: http://localhost:4000)
|
||||
mix upload_librenms --librenms-path ~/dev/librenms
|
||||
|
||||
# Upload to production server
|
||||
mix upload_librenms --librenms-path ~/dev/librenms \
|
||||
--url https://towerops.net \
|
||||
--token your-api-token-here
|
||||
|
||||
# Upload specific profiles only
|
||||
mix upload_librenms --librenms-path ~/dev/librenms \
|
||||
--profiles routeros,ios,iosxe
|
||||
```
|
||||
|
||||
### Command-Line Options
|
||||
|
||||
- `--librenms-path` - Path to LibreNMS installation (required)
|
||||
- `--url` - Server URL (default: http://localhost:4000)
|
||||
- `--token` - API token for authentication (required for remote servers)
|
||||
- `--profiles` - Comma-separated list of profiles to import (default: all)
|
||||
|
||||
## What Gets Uploaded
|
||||
|
||||
### MIB Files
|
||||
|
||||
All MIB files from `{librenms-path}/mibs/` are uploaded, organized by vendor:
|
||||
|
||||
```
|
||||
~/dev/librenms/mibs/
|
||||
├── mikrotik/
|
||||
│ ├── MIKROTIK-MIB
|
||||
│ └── ...
|
||||
├── cisco/
|
||||
│ ├── CISCO-SMI
|
||||
│ └── ...
|
||||
└── ...
|
||||
```
|
||||
|
||||
Each vendor's MIBs are uploaded separately with the vendor name preserved.
|
||||
|
||||
### Device Profiles
|
||||
|
||||
Device profiles are imported from LibreNMS YAML definitions:
|
||||
|
||||
```
|
||||
~/dev/librenms/resources/definitions/
|
||||
├── os_detection/
|
||||
│ ├── routeros.yaml # Detection patterns
|
||||
│ └── ios.yaml
|
||||
└── os_discovery/
|
||||
├── routeros.yaml # Device info & sensors
|
||||
└── ios.yaml
|
||||
```
|
||||
|
||||
For each profile, the following is imported:
|
||||
- **Detection patterns** (sysObjectID, sysDescr regex)
|
||||
- **Device info OIDs** (serial number, firmware version, hardware)
|
||||
- **Sensor OIDs** (temperature, voltage, current, power, fanspeed)
|
||||
|
||||
**Note**: Only scalar sensors (ending with `.0`) are imported. Table-based sensors with `{{ $index }}` templates are skipped.
|
||||
|
||||
## Authentication
|
||||
|
||||
### Profile Import Only (No Token Needed)
|
||||
|
||||
Profile imports work without authentication since they only write to the local database:
|
||||
|
||||
```bash
|
||||
# Import profiles without uploading MIBs
|
||||
mix upload_librenms --librenms-path ~/dev/librenms
|
||||
```
|
||||
|
||||
### MIB Upload (Token Required)
|
||||
|
||||
MIB uploads require a **superuser API token** for both local and remote servers:
|
||||
|
||||
```bash
|
||||
# Upload MIBs + import profiles (local server)
|
||||
mix upload_librenms --librenms-path ~/dev/librenms \
|
||||
--token your-superuser-token
|
||||
|
||||
# Upload to remote server
|
||||
mix upload_librenms --librenms-path ~/dev/librenms \
|
||||
--url https://towerops.net \
|
||||
--token your-superuser-token
|
||||
```
|
||||
|
||||
**Creating an API Token:**
|
||||
1. Log in to Towerops as a superuser
|
||||
2. Navigate to Settings → API Tokens
|
||||
3. Create a new token
|
||||
4. Copy the token value
|
||||
|
||||
**Note**: The token MUST be from a superuser account. Regular user tokens will be rejected with a 403 Forbidden error.
|
||||
|
||||
## Examples
|
||||
|
||||
### Example 1: Import Profiles Only (Quick Start)
|
||||
|
||||
Import device profiles without uploading MIBs (no token needed):
|
||||
|
||||
```bash
|
||||
mix upload_librenms --librenms-path ~/dev/librenms
|
||||
```
|
||||
|
||||
Output:
|
||||
```
|
||||
Skipping MIB upload: --token is required for MIB uploads (superuser API token needed)
|
||||
Step 2/2: Importing device profiles from ~/dev/librenms/resources/definitions...
|
||||
Importing 150 specified profiles
|
||||
Importing profile: routeros
|
||||
✓ routeros: created/updated successfully
|
||||
...
|
||||
Import complete: 150 succeeded, 0 failed
|
||||
|
||||
Upload complete!
|
||||
```
|
||||
|
||||
### Example 2: Upload Everything (MIBs + Profiles)
|
||||
|
||||
Upload MIBs and import profiles (requires superuser token):
|
||||
|
||||
```bash
|
||||
mix upload_librenms --librenms-path ~/dev/librenms --token sk_dev_abc123...
|
||||
```
|
||||
|
||||
Output:
|
||||
```
|
||||
Step 1/2: Uploading MIB files from ~/dev/librenms/mibs...
|
||||
Found 500 vendor directories
|
||||
Uploading MIBs for vendor: mikrotik
|
||||
Uploading: MIKROTIK-MIB
|
||||
Uploading: ...
|
||||
MIB upload complete: 5000 succeeded, 0 failed
|
||||
|
||||
Step 2/2: Importing device profiles from ~/dev/librenms/resources/definitions...
|
||||
Importing 150 specified profiles
|
||||
Importing profile: routeros
|
||||
✓ routeros: created/updated successfully
|
||||
...
|
||||
Import complete: 150 succeeded, 0 failed
|
||||
|
||||
Upload complete!
|
||||
```
|
||||
|
||||
### Example 3: Update Specific Profiles
|
||||
|
||||
Re-import just the MikroTik and Cisco profiles (no MIB upload):
|
||||
|
||||
```bash
|
||||
mix upload_librenms --librenms-path ~/dev/librenms \
|
||||
--profiles routeros,ios,iosxe,nxos
|
||||
```
|
||||
|
||||
### Example 4: Production Deployment
|
||||
|
||||
Upload everything to production server:
|
||||
|
||||
```bash
|
||||
mix upload_librenms --librenms-path ~/dev/librenms \
|
||||
--url https://towerops.net \
|
||||
--token sk_live_abc123...
|
||||
```
|
||||
|
||||
## Manual Tasks
|
||||
|
||||
You can also run each step separately if needed:
|
||||
|
||||
### Upload MIBs Only
|
||||
|
||||
```bash
|
||||
mix upload_mibs --source-dir ~/dev/librenms/mibs \
|
||||
--url https://towerops.net \
|
||||
--token your-token
|
||||
```
|
||||
|
||||
### Import Profiles Only
|
||||
|
||||
```bash
|
||||
mix import_profiles --source-path ~/dev/librenms/resources/definitions \
|
||||
--profiles routeros,ios
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Error: "LibreNMS directory not found"
|
||||
|
||||
Make sure the path to your LibreNMS installation is correct:
|
||||
|
||||
```bash
|
||||
ls -la ~/dev/librenms
|
||||
# Should show mibs/ and resources/ directories
|
||||
```
|
||||
|
||||
### Error: "No vendor directories found"
|
||||
|
||||
The mibs directory structure should have vendor subdirectories:
|
||||
|
||||
```bash
|
||||
ls -la ~/dev/librenms/mibs/
|
||||
# Should show directories like mikrotik/, cisco/, etc.
|
||||
```
|
||||
|
||||
### Error: "Superuser access required"
|
||||
|
||||
MIB uploads require a superuser API token. Make sure you're using a token created by a superuser account.
|
||||
|
||||
### Warning: "Definitions directory not found"
|
||||
|
||||
LibreNMS moved the definitions from `includes/definitions/` to `resources/definitions/` in recent versions. Update your path accordingly.
|
||||
|
||||
## What's Imported
|
||||
|
||||
### RouterOS (MikroTik) Example
|
||||
|
||||
For the `routeros` profile, the following is imported:
|
||||
|
||||
**Profile Metadata:**
|
||||
- Name: routeros
|
||||
- Vendor: Mikrotik RouterOS
|
||||
- Detection OID: .1.3.6.1.4.1.14988.1
|
||||
- Priority: 100
|
||||
|
||||
**Device OIDs:**
|
||||
- Serial number: `MIKROTIK-MIB::mtxrSerialNumber.0`
|
||||
- Firmware version: `MIKROTIK-MIB::mtxrLicVersion.0`
|
||||
|
||||
**Sensor OIDs:**
|
||||
- None (routeros only has table-based sensors which are skipped)
|
||||
|
||||
### Cisco IOS Example
|
||||
|
||||
For the `ios` profile:
|
||||
|
||||
**Profile Metadata:**
|
||||
- Name: ios
|
||||
- Vendor: Cisco IOS
|
||||
- Detection Pattern: "Cisco Internetwork Operating System Software"
|
||||
- Detection OID: .1.3.6.1.4.1.9.1.2330
|
||||
- Priority: 100
|
||||
|
||||
**Device OIDs:**
|
||||
- None (IOS doesn't define them in os_discovery)
|
||||
|
||||
**Sensor OIDs:**
|
||||
- None (IOS only has table-based sensors which are skipped)
|
||||
|
||||
## Database Schema
|
||||
|
||||
The imported profiles are stored in the database:
|
||||
|
||||
- `device_profiles` - Profile metadata (name, vendor, detection patterns)
|
||||
- `profile_device_oids` - Device identification OIDs (serial, firmware, etc.)
|
||||
- `profile_sensor_oids` - Sensor discovery OIDs (temperature, voltage, etc.)
|
||||
|
||||
You can query them:
|
||||
|
||||
```elixir
|
||||
# Get all profiles
|
||||
Towerops.Profiles.list_profiles()
|
||||
|
||||
# Get specific profile with associations
|
||||
Towerops.Profiles.get_profile("routeros")
|
||||
|
||||
# Match a device to a profile
|
||||
system_info = %{
|
||||
sys_descr: "RouterOS 7.16.1",
|
||||
sys_object_id: ".1.3.6.1.4.1.14988.1"
|
||||
}
|
||||
Towerops.Profiles.match_profile(system_info)
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
After importing profiles:
|
||||
|
||||
1. **Test device discovery** - Discover a MikroTik or Cisco device to verify profile matching works
|
||||
2. **Check MIB translation** - Verify that MIB names are being translated to numeric OIDs correctly
|
||||
3. **Add more profiles** - Import additional vendor profiles as needed
|
||||
4. **Build Web UI** - Create LiveView pages for managing profiles in the browser
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- `PROFILES_README.md` - Database-driven profile system documentation
|
||||
- `lib/mix/tasks/upload_librenms.ex` - Task implementation
|
||||
- `lib/mix/tasks/import_profiles.ex` - Profile import implementation
|
||||
- `lib/mix/tasks/upload_mibs.ex` - MIB upload implementation
|
||||
182
PROFILES_README.md
Normal file
182
PROFILES_README.md
Normal file
|
|
@ -0,0 +1,182 @@
|
|||
# Device Profiles Implementation Status
|
||||
|
||||
## Completed
|
||||
|
||||
### Database Schema
|
||||
- ✅ Created `device_profiles` table for storing profile metadata
|
||||
- ✅ Created `profile_device_oids` table for device identification OIDs (serial, firmware, etc.)
|
||||
- ✅ Created `profile_sensor_oids` table for sensor discovery OIDs (temp, voltage, current, power)
|
||||
- ✅ Seeded MikroTik profile with all MIB symbolic names
|
||||
|
||||
### Ecto Schemas
|
||||
- ✅ `Towerops.Profiles.DeviceProfile` - Profile schema with associations
|
||||
- ✅ `Towerops.Profiles.DeviceOid` - Device identification OID schema
|
||||
- ✅ `Towerops.Profiles.SensorOid` - Sensor OID schema with divisor support
|
||||
|
||||
### Context
|
||||
- ✅ `Towerops.Profiles` context for CRUD operations
|
||||
- `list_profiles/0` - Get all enabled profiles ordered by priority
|
||||
- `get_profile/1` - Get profile by name with associations
|
||||
- `match_profile/1` - Match device to profile based on sysDescr/sysObjectID
|
||||
- `create_profile/1`, `update_profile/2`, `delete_profile/1`
|
||||
- `add_device_oid/3`, `add_sensor_oid/2`
|
||||
|
||||
### Dynamic Profile
|
||||
- ✅ `Towerops.Snmp.Profiles.Dynamic` - Runtime profile using database definitions
|
||||
- `identify_device/3` - Polls device OIDs using MIB translation
|
||||
- `discover_sensors/2` - Discovers sensors using MIB translation
|
||||
- `discover_interfaces/2` - Delegates to Base profile
|
||||
|
||||
### Discovery Integration
|
||||
- ✅ Updated `Towerops.Snmp.Discovery` to support database profiles
|
||||
- Tries database profiles first via `Profiles.match_profile/1`
|
||||
- Falls back to hardcoded profiles (MikroTik, Cisco, etc.)
|
||||
- Handles `{:dynamic, profile}` tuples for database profiles
|
||||
|
||||
### Mix Task (Basic Structure)
|
||||
- ✅ Created `Mix.Tasks.ImportProfiles` with basic structure
|
||||
- ✅ Argument parsing (--source-path, --profiles filter)
|
||||
- ✅ YAML file discovery and reading
|
||||
|
||||
## Current Status
|
||||
|
||||
The core database-driven profile system is **fully functional**:
|
||||
- MikroTik devices will be detected using the database profile
|
||||
- Device info (serial, firmware, license) will be polled via MIB translation
|
||||
- Sensors (temp, voltage, current, power) will be discovered via MIB translation
|
||||
- Profiles can be updated without redeployment
|
||||
|
||||
## Completed: LibreNMS Import Task
|
||||
|
||||
The `mix import_profiles` task has been implemented and tested successfully with LibreNMS YAML files.
|
||||
|
||||
### LibreNMS File Structure
|
||||
|
||||
LibreNMS splits profile definitions across two files:
|
||||
|
||||
1. **`resources/definitions/os_detection/{os}.yaml`** - Detection and metadata
|
||||
```yaml
|
||||
os: routeros
|
||||
text: 'Mikrotik RouterOS'
|
||||
discovery:
|
||||
- sysObjectID:
|
||||
- .1.3.6.1.4.1.14988.1
|
||||
```
|
||||
|
||||
2. **`resources/definitions/os_discovery/{os}.yaml`** - Device info and sensors
|
||||
```yaml
|
||||
modules:
|
||||
os:
|
||||
serial: MIKROTIK-MIB::mtxrSerialNumber.0
|
||||
version: MIKROTIK-MIB::mtxrLicVersion.0
|
||||
sensors:
|
||||
temperature:
|
||||
data:
|
||||
- oid: MIKROTIK-MIB::mtxrGaugeTable
|
||||
value: MIKROTIK-MIB::mtxrGaugeValue
|
||||
num_oid: '.1.3.6.1.4.1.14988.1.1.3.100.1.3.{{ $index }}'
|
||||
descr: MIKROTIK-MIB::mtxrGaugeName
|
||||
divisor: 10
|
||||
```
|
||||
|
||||
### Implementation Details
|
||||
|
||||
1. **`import_profile/2` function** ✅
|
||||
- Reads both `os_detection/{name}.yaml` and `os_discovery/{name}.yaml`
|
||||
- Extracts detection patterns from os_detection file (sysObjectID)
|
||||
- Extracts device OIDs from `modules.os` section (serial, version, hardware)
|
||||
- Extracts sensor OIDs from `modules.sensors.{type}.data[]` arrays
|
||||
|
||||
2. **Sensor handling** ✅
|
||||
- Only imports scalar sensors (with `.0` suffix)
|
||||
- Skips table-based sensors with `{{ $index }}` templates
|
||||
- Cleans up template syntax in descriptions
|
||||
- Handles divisor values correctly
|
||||
|
||||
3. **Supported sensor types** ✅
|
||||
- `temperature`, `voltage`, `current`, `power`, `fanspeed`
|
||||
- Other types (`dbm`, `state`, `count`) can be added as needed
|
||||
|
||||
4. **Tested with real LibreNMS** ✅
|
||||
- Successfully imported `routeros` profile from `~/dev/librenms/resources/definitions/`
|
||||
- Profile created with correct detection OID and device OIDs
|
||||
- Scalar sensors imported (though routeros only has table-based sensors)
|
||||
|
||||
### Usage
|
||||
|
||||
Import profiles from LibreNMS YAML definitions:
|
||||
|
||||
```bash
|
||||
# Import all profiles
|
||||
mix import_profiles --source-path ~/dev/librenms/resources/definitions
|
||||
|
||||
# Import specific profiles
|
||||
mix import_profiles --source-path ~/dev/librenms/resources/definitions --profiles routeros,ios
|
||||
|
||||
# Import single profile
|
||||
mix import_profiles --source-path ~/dev/librenms/resources/definitions --profiles routeros
|
||||
```
|
||||
|
||||
**Note**: The task looks for files in:
|
||||
- `{source-path}/os_detection/{profile}.yaml` - Detection patterns
|
||||
- `{source-path}/os_discovery/{profile}.yaml` - Device info and sensors
|
||||
|
||||
### Current Limitations
|
||||
|
||||
1. **Table-based sensors** - Only scalar sensors (ending with `.0`) are imported
|
||||
- Table sensors with `{{ $index }}` templates are skipped
|
||||
- This excludes most interface-level sensors (POE ports, optical transceivers, etc.)
|
||||
|
||||
2. **Complex templates** - Template syntax in descriptions is removed
|
||||
- `{{ MIKROTIK-MIB::mtxrPOEName }} POE` becomes just "POE"
|
||||
|
||||
3. **Sensor types** - Only common types are imported
|
||||
- Supported: temperature, voltage, current, power, fanspeed
|
||||
- Not supported: dbm, state, count (can be added as needed)
|
||||
|
||||
### Next Steps
|
||||
|
||||
1. **Add more sensor types** - Add dbm, state, count if needed
|
||||
2. **Web UI for profiles** - Build LiveView CRUD for managing profiles
|
||||
3. **Table sensor support** - Implement support for table-based sensors
|
||||
4. **Import more profiles** - Test with Cisco, Ubiquiti, etc.
|
||||
|
||||
## Testing
|
||||
|
||||
To test the current implementation:
|
||||
|
||||
```elixir
|
||||
# Get the seeded MikroTik profile
|
||||
profile = Towerops.Profiles.get_profile("mikrotik")
|
||||
|
||||
# Check device OIDs
|
||||
profile.device_oids
|
||||
# => [
|
||||
# %{field: "serial_number", mib_name: "MIKROTIK-MIB::mtxrSerialNumber.0"},
|
||||
# %{field: "firmware_version", mib_name: "MIKROTIK-MIB::mtxrFirmwareVersion.0"},
|
||||
# ...
|
||||
# ]
|
||||
|
||||
# Check sensor OIDs
|
||||
profile.sensor_oids
|
||||
# => [
|
||||
# %{sensor_type: "temperature", mib_name: "MIKROTIK-MIB::mtxrHlTemperature.0", divisor: 10},
|
||||
# %{sensor_type: "voltage", mib_name: "MIKROTIK-MIB::mtxrHlVoltage.0", divisor: 10},
|
||||
# ...
|
||||
# ]
|
||||
|
||||
# Match a device to a profile
|
||||
system_info = %{
|
||||
sys_descr: "RouterOS 7.16.1",
|
||||
sys_object_id: ".1.3.6.1.4.1.14988.1"
|
||||
}
|
||||
matched = Towerops.Profiles.match_profile(system_info)
|
||||
# => %DeviceProfile{name: "mikrotik", ...}
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **Test with real MikroTik device** - Verify dynamic profile works end-to-end
|
||||
2. **Add Web UI** - Build LiveView pages for managing profiles
|
||||
3. **Improve import task** - Parse LibreNMS YAML structure correctly
|
||||
4. **Add more profiles** - Cisco, Ubiquiti, etc.
|
||||
355
lib/mix/tasks/import_profiles.ex
Normal file
355
lib/mix/tasks/import_profiles.ex
Normal file
|
|
@ -0,0 +1,355 @@
|
|||
defmodule Mix.Tasks.ImportProfiles do
|
||||
@shortdoc "Import device profiles from LibreNMS YAML definitions"
|
||||
|
||||
@moduledoc """
|
||||
Imports device profiles from LibreNMS YAML definition files.
|
||||
|
||||
## Usage
|
||||
|
||||
# Import all profile definitions
|
||||
mix import_profiles --source-path ~/librenms/includes/definitions
|
||||
|
||||
# Import specific profiles
|
||||
mix import_profiles --source-path ~/librenms/includes/definitions --profiles mikrotik,cisco
|
||||
|
||||
## Options
|
||||
|
||||
* `--source-path` - Path to LibreNMS includes/definitions directory (required)
|
||||
* `--profiles` - Comma-separated list of profiles to import (default: all)
|
||||
|
||||
## Notes
|
||||
|
||||
* Reads YAML files from the definitions directory
|
||||
* Creates database profiles with MIB symbolic names
|
||||
* Updates existing profiles if they already exist
|
||||
* Requires the application to be started to access the database
|
||||
"""
|
||||
|
||||
use Mix.Task
|
||||
|
||||
import Ecto.Query
|
||||
|
||||
alias Towerops.Profiles
|
||||
alias Towerops.Profiles.DeviceOid
|
||||
alias Towerops.Profiles.SensorOid
|
||||
alias Towerops.Repo
|
||||
|
||||
require Logger
|
||||
|
||||
@requirements ["app.start"]
|
||||
|
||||
def run(args) do
|
||||
{opts, _} =
|
||||
OptionParser.parse!(args,
|
||||
strict: [
|
||||
source_path: :string,
|
||||
profiles: :string
|
||||
]
|
||||
)
|
||||
|
||||
source_path = opts[:source_path] || raise "Missing --source-path argument"
|
||||
profiles_filter = parse_profiles(opts[:profiles])
|
||||
|
||||
if !File.exists?(source_path) do
|
||||
raise "Source directory not found: #{source_path}"
|
||||
end
|
||||
|
||||
case profiles_filter do
|
||||
:all ->
|
||||
import_all_profiles(source_path)
|
||||
|
||||
profile_list ->
|
||||
import_specific_profiles(source_path, profile_list)
|
||||
end
|
||||
end
|
||||
|
||||
defp parse_profiles(nil), do: :all
|
||||
defp parse_profiles(""), do: :all
|
||||
|
||||
defp parse_profiles(profiles_string) do
|
||||
profiles_string
|
||||
|> String.split(",")
|
||||
|> Enum.map(&String.trim/1)
|
||||
|> Enum.reject(&(&1 == ""))
|
||||
end
|
||||
|
||||
defp import_all_profiles(source_path) do
|
||||
Logger.info("Discovering YAML files in #{source_path}...")
|
||||
|
||||
detection_path = Path.join(source_path, "os_detection")
|
||||
|
||||
yaml_files =
|
||||
detection_path
|
||||
|> Path.join("*.yaml")
|
||||
|> Path.wildcard()
|
||||
|> Enum.sort()
|
||||
|
||||
Logger.info("Found #{length(yaml_files)} YAML files")
|
||||
|
||||
profile_names = Enum.map(yaml_files, &Path.basename(&1, ".yaml"))
|
||||
import_profiles_by_name(source_path, profile_names)
|
||||
end
|
||||
|
||||
defp import_specific_profiles(source_path, profiles) do
|
||||
Logger.info("Importing #{length(profiles)} specified profiles")
|
||||
import_profiles_by_name(source_path, profiles)
|
||||
end
|
||||
|
||||
defp import_profiles_by_name(source_path, profile_names) do
|
||||
results =
|
||||
Enum.map(profile_names, fn profile_name ->
|
||||
Logger.info("Importing profile: #{profile_name}")
|
||||
|
||||
case import_profile(source_path, profile_name) do
|
||||
{:ok, _profile} ->
|
||||
Logger.info("✓ #{profile_name}: created/updated successfully")
|
||||
{:ok, profile_name}
|
||||
|
||||
{:error, reason} ->
|
||||
Logger.error("✗ #{profile_name}: #{inspect(reason)}")
|
||||
{:error, profile_name, reason}
|
||||
end
|
||||
end)
|
||||
|
||||
successes = Enum.count(results, &match?({:ok, _}, &1))
|
||||
failures = Enum.count(results, &match?({:error, _, _}, &1))
|
||||
|
||||
Logger.info("Import complete: #{successes} succeeded, #{failures} failed")
|
||||
end
|
||||
|
||||
defp import_profile(source_path, profile_name) do
|
||||
detection_file = Path.join([source_path, "os_detection", "#{profile_name}.yaml"])
|
||||
discovery_file = Path.join([source_path, "os_discovery", "#{profile_name}.yaml"])
|
||||
|
||||
with {:ok, detection_data} <- read_yaml_file(detection_file),
|
||||
{:ok, discovery_data} <- read_yaml_file(discovery_file),
|
||||
{:ok, _profile} <- create_or_update_profile(detection_data, discovery_data, profile_name),
|
||||
# Reload profile to get correct binary ID
|
||||
profile when not is_nil(profile) <- Profiles.get_profile(profile_name),
|
||||
:ok <- import_device_oids(profile, discovery_data),
|
||||
:ok <- import_sensor_oids(profile, discovery_data) do
|
||||
{:ok, profile}
|
||||
end
|
||||
end
|
||||
|
||||
defp read_yaml_file(file_path) do
|
||||
with {:ok, yaml_content} <- File.read(file_path),
|
||||
{:ok, yaml_data} <- YamlElixir.read_from_string(yaml_content) do
|
||||
# YamlElixir can return either a map or a list with one map
|
||||
yaml_data =
|
||||
case yaml_data do
|
||||
[data] -> data
|
||||
data when is_map(data) -> data
|
||||
_ -> yaml_data
|
||||
end
|
||||
|
||||
{:ok, yaml_data}
|
||||
else
|
||||
{:error, :enoent} ->
|
||||
{:error, "File not found: #{file_path}"}
|
||||
|
||||
{:error, reason} ->
|
||||
{:error, reason}
|
||||
end
|
||||
end
|
||||
|
||||
defp create_or_update_profile(detection_data, _discovery_data, profile_name) do
|
||||
# Extract detection patterns from os_detection file
|
||||
detection_pattern = extract_detection_pattern(detection_data)
|
||||
detection_oid = extract_detection_oid(detection_data)
|
||||
vendor = Map.get(detection_data, "text") || String.capitalize(profile_name)
|
||||
|
||||
attrs = %{
|
||||
name: profile_name,
|
||||
vendor: vendor,
|
||||
detection_pattern: detection_pattern,
|
||||
detection_oid: detection_oid,
|
||||
priority: Map.get(detection_data, "priority", 100),
|
||||
enabled: true
|
||||
}
|
||||
|
||||
# Check if profile exists
|
||||
case Profiles.get_profile(profile_name) do
|
||||
nil ->
|
||||
Profiles.create_profile(attrs)
|
||||
|
||||
existing_profile ->
|
||||
Profiles.update_profile(existing_profile, attrs)
|
||||
end
|
||||
end
|
||||
|
||||
defp extract_detection_pattern(yaml_data) do
|
||||
# Look for sysDescr patterns in discovery section
|
||||
case get_in(yaml_data, ["discovery"]) do
|
||||
discovery when is_list(discovery) ->
|
||||
sys_descr_patterns =
|
||||
Enum.flat_map(discovery, fn item ->
|
||||
case item do
|
||||
%{"sysDescr" => patterns} when is_list(patterns) -> patterns
|
||||
_ -> []
|
||||
end
|
||||
end)
|
||||
|
||||
case sys_descr_patterns do
|
||||
[first | _] -> first
|
||||
_ -> nil
|
||||
end
|
||||
|
||||
_ ->
|
||||
nil
|
||||
end
|
||||
end
|
||||
|
||||
defp extract_detection_oid(yaml_data) do
|
||||
# Look for sysObjectID patterns in discovery section
|
||||
case get_in(yaml_data, ["discovery"]) do
|
||||
discovery when is_list(discovery) ->
|
||||
oid_patterns =
|
||||
Enum.flat_map(discovery, fn item ->
|
||||
case item do
|
||||
%{"sysObjectID" => patterns} when is_list(patterns) -> patterns
|
||||
_ -> []
|
||||
end
|
||||
end)
|
||||
|
||||
case oid_patterns do
|
||||
[first | _] -> first
|
||||
_ -> nil
|
||||
end
|
||||
|
||||
_ ->
|
||||
nil
|
||||
end
|
||||
end
|
||||
|
||||
defp import_device_oids(profile, yaml_data) do
|
||||
# Look for device info in various places
|
||||
device_oids = extract_device_oids(yaml_data)
|
||||
|
||||
# Delete existing device OIDs
|
||||
Repo.delete_all(
|
||||
from o in DeviceOid,
|
||||
where: o.profile_id == ^profile.id
|
||||
)
|
||||
|
||||
# Insert new device OIDs
|
||||
Enum.each(device_oids, fn {field, mib_name} ->
|
||||
Profiles.add_device_oid(profile.id, field, mib_name)
|
||||
end)
|
||||
|
||||
:ok
|
||||
end
|
||||
|
||||
defp extract_device_oids(yaml_data) do
|
||||
# Extract from modules.os section in os_discovery YAML
|
||||
device_oids = []
|
||||
|
||||
# Get the modules.os section
|
||||
os_module = get_in(yaml_data, ["modules", "os"]) || %{}
|
||||
|
||||
# Serial number
|
||||
device_oids =
|
||||
case Map.get(os_module, "serial") do
|
||||
nil -> device_oids
|
||||
mib_name -> [{"serial_number", mib_name} | device_oids]
|
||||
end
|
||||
|
||||
# Firmware version
|
||||
device_oids =
|
||||
case Map.get(os_module, "version") do
|
||||
nil -> device_oids
|
||||
mib_name -> [{"firmware_version", mib_name} | device_oids]
|
||||
end
|
||||
|
||||
# Hardware/model
|
||||
device_oids =
|
||||
case Map.get(os_module, "hardware") do
|
||||
nil -> device_oids
|
||||
mib_name -> [{"hardware", mib_name} | device_oids]
|
||||
end
|
||||
|
||||
device_oids
|
||||
end
|
||||
|
||||
defp import_sensor_oids(profile, yaml_data) do
|
||||
# Look for sensor definitions
|
||||
sensors = extract_sensor_oids(yaml_data)
|
||||
|
||||
# Delete existing sensor OIDs
|
||||
Repo.delete_all(
|
||||
from o in SensorOid,
|
||||
where: o.profile_id == ^profile.id
|
||||
)
|
||||
|
||||
# Insert new sensor OIDs
|
||||
Enum.each(sensors, fn sensor_attrs ->
|
||||
Profiles.add_sensor_oid(profile.id, sensor_attrs)
|
||||
end)
|
||||
|
||||
:ok
|
||||
end
|
||||
|
||||
defp extract_sensor_oids(yaml_data) do
|
||||
# Extract from modules.sensors section in os_discovery YAML
|
||||
sensors_module = get_in(yaml_data, ["modules", "sensors"]) || %{}
|
||||
|
||||
# Sensor types to extract
|
||||
sensor_types = ["temperature", "voltage", "current", "power", "fanspeed"]
|
||||
|
||||
Enum.flat_map(sensor_types, fn sensor_type ->
|
||||
case get_in(sensors_module, [sensor_type, "data"]) do
|
||||
sensor_data when is_list(sensor_data) ->
|
||||
sensor_data
|
||||
|> Enum.map(&parse_sensor(&1, sensor_type))
|
||||
|> Enum.reject(&is_nil/1)
|
||||
|
||||
_ ->
|
||||
[]
|
||||
end
|
||||
end)
|
||||
end
|
||||
|
||||
defp parse_sensor(sensor_def, sensor_type) do
|
||||
# Get the MIB name - prefer 'oid' field, fall back to 'value' or 'num_oid'
|
||||
mib_name = Map.get(sensor_def, "oid") || Map.get(sensor_def, "value")
|
||||
num_oid = Map.get(sensor_def, "num_oid")
|
||||
|
||||
# Skip table-based sensors (those with {{ $index }} templates)
|
||||
if is_binary(num_oid) and String.contains?(num_oid, "{{ $index }}") do
|
||||
nil
|
||||
else
|
||||
# Only import scalar sensors (ending with .0)
|
||||
if is_binary(mib_name) and String.ends_with?(mib_name, ".0") do
|
||||
# Extract description and clean up templates
|
||||
descr =
|
||||
case Map.get(sensor_def, "descr") do
|
||||
nil ->
|
||||
sensor_type
|
||||
|
||||
descr_template when is_binary(descr_template) ->
|
||||
# Remove template syntax like {{ MIB::name }}
|
||||
descr_template
|
||||
|> String.replace(~r/\{\{ [^}]+ \}\}/, "")
|
||||
|> String.trim()
|
||||
|> case do
|
||||
"" -> sensor_type
|
||||
cleaned -> cleaned
|
||||
end
|
||||
|
||||
_ ->
|
||||
sensor_type
|
||||
end
|
||||
|
||||
%{
|
||||
sensor_type: sensor_type,
|
||||
mib_name: mib_name,
|
||||
sensor_descr: descr,
|
||||
sensor_unit: Map.get(sensor_def, "unit"),
|
||||
sensor_divisor: Map.get(sensor_def, "divisor", 1)
|
||||
}
|
||||
|
||||
# Skip non-scalar sensors for now
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
185
lib/mix/tasks/upload_librenms.ex
Normal file
185
lib/mix/tasks/upload_librenms.ex
Normal file
|
|
@ -0,0 +1,185 @@
|
|||
defmodule Mix.Tasks.UploadLibrenms do
|
||||
@shortdoc "Upload MIBs and import profiles from LibreNMS installation"
|
||||
|
||||
@moduledoc """
|
||||
Uploads MIB files and imports device profiles from a LibreNMS installation.
|
||||
|
||||
## Usage
|
||||
|
||||
# Upload from local LibreNMS installation
|
||||
mix upload_librenms --librenms-path ~/dev/librenms
|
||||
|
||||
# Upload to specific server
|
||||
mix upload_librenms --librenms-path ~/dev/librenms --url https://towerops.net
|
||||
|
||||
# Upload specific profiles only
|
||||
mix upload_librenms --librenms-path ~/dev/librenms --profiles mikrotik,cisco
|
||||
|
||||
## Options
|
||||
|
||||
* `--librenms-path` - Path to LibreNMS installation directory (required)
|
||||
* `--token` - Superuser API token for authentication (required for MIB upload)
|
||||
* `--url` - Server URL (default: http://localhost:4000)
|
||||
* `--profiles` - Comma-separated list of profiles to import (default: all)
|
||||
|
||||
## What it does
|
||||
|
||||
1. Uploads all MIB files from `{librenms-path}/mibs` to the server
|
||||
2. Imports device profiles from `{librenms-path}/resources/definitions`
|
||||
|
||||
## Requirements
|
||||
|
||||
* LibreNMS installation with mibs/ and resources/definitions/ directories
|
||||
* Server must be running and accessible
|
||||
"""
|
||||
|
||||
use Mix.Task
|
||||
|
||||
require Logger
|
||||
|
||||
@requirements ["app.start"]
|
||||
|
||||
def run(args) do
|
||||
{opts, _} =
|
||||
OptionParser.parse!(args,
|
||||
strict: [
|
||||
librenms_path: :string,
|
||||
url: :string,
|
||||
token: :string,
|
||||
profiles: :string
|
||||
]
|
||||
)
|
||||
|
||||
librenms_path = opts[:librenms_path] || raise "Missing --librenms-path argument"
|
||||
base_url = opts[:url] || "http://localhost:4000"
|
||||
token = opts[:token]
|
||||
profiles = opts[:profiles]
|
||||
|
||||
if !File.exists?(librenms_path) do
|
||||
raise "LibreNMS directory not found: #{librenms_path}"
|
||||
end
|
||||
|
||||
# Step 1: Upload MIBs
|
||||
mibs_path = Path.join(librenms_path, "mibs")
|
||||
|
||||
if File.exists?(mibs_path) do
|
||||
if token do
|
||||
Logger.info("Step 1/2: Uploading MIB files from #{mibs_path}...")
|
||||
upload_mibs(mibs_path, base_url, token)
|
||||
else
|
||||
Logger.warning("Skipping MIB upload: --token is required for MIB uploads (superuser API token needed)")
|
||||
end
|
||||
else
|
||||
Logger.warning("MIBs directory not found: #{mibs_path}, skipping MIB upload")
|
||||
end
|
||||
|
||||
# Step 2: Import profiles
|
||||
definitions_path = Path.join(librenms_path, "resources/definitions")
|
||||
|
||||
if File.exists?(definitions_path) do
|
||||
Logger.info("Step 2/2: Importing device profiles from #{definitions_path}...")
|
||||
import_profiles(definitions_path, profiles)
|
||||
else
|
||||
Logger.warning("Definitions directory not found: #{definitions_path}, skipping profile import")
|
||||
end
|
||||
|
||||
Logger.info("Upload complete!")
|
||||
end
|
||||
|
||||
defp upload_mibs(mibs_path, base_url, token) do
|
||||
# Find all vendor directories in mibs/
|
||||
vendor_dirs =
|
||||
mibs_path
|
||||
|> Path.join("*")
|
||||
|> Path.wildcard()
|
||||
|> Enum.filter(&File.dir?/1)
|
||||
|> Enum.sort()
|
||||
|
||||
if Enum.empty?(vendor_dirs) do
|
||||
Logger.warning("No vendor directories found in #{mibs_path}")
|
||||
else
|
||||
Logger.info("Found #{length(vendor_dirs)} vendor directories")
|
||||
|
||||
# Upload each vendor's MIBs
|
||||
results =
|
||||
Enum.flat_map(vendor_dirs, fn vendor_dir ->
|
||||
vendor = Path.basename(vendor_dir)
|
||||
Logger.info("Uploading MIBs for vendor: #{vendor}")
|
||||
|
||||
# Find all MIB files in this vendor directory
|
||||
mib_files =
|
||||
vendor_dir
|
||||
|> Path.join("*")
|
||||
|> Path.wildcard()
|
||||
|> Enum.filter(&File.regular?/1)
|
||||
|> Enum.sort()
|
||||
|
||||
Enum.map(mib_files, fn file_path ->
|
||||
filename = Path.basename(file_path)
|
||||
Logger.debug(" Uploading: #{filename}")
|
||||
|
||||
case upload_mib_file(file_path, vendor, base_url, token) do
|
||||
:ok ->
|
||||
{:ok, filename}
|
||||
|
||||
{:error, reason} ->
|
||||
Logger.error(" ✗ Failed to upload #{filename}: #{inspect(reason)}")
|
||||
{:error, filename, reason}
|
||||
end
|
||||
end)
|
||||
end)
|
||||
|
||||
successes = Enum.count(results, &match?({:ok, _}, &1))
|
||||
failures = Enum.count(results, &match?({:error, _, _}, &1))
|
||||
|
||||
Logger.info("MIB upload complete: #{successes} succeeded, #{failures} failed")
|
||||
end
|
||||
end
|
||||
|
||||
defp upload_mib_file(file_path, vendor, base_url, token) do
|
||||
url = "#{base_url}/api/v1/mibs"
|
||||
filename = Path.basename(file_path)
|
||||
|
||||
# Build base request
|
||||
req = Req.new(url: url)
|
||||
|
||||
# Add authorization header if token is provided
|
||||
req =
|
||||
if token do
|
||||
Req.Request.put_header(req, "authorization", "Bearer #{token}")
|
||||
else
|
||||
req
|
||||
end
|
||||
|
||||
# Add multipart form data
|
||||
form_data = [
|
||||
file: {file_path, filename: filename},
|
||||
vendor: vendor
|
||||
]
|
||||
|
||||
case Req.post(req, form_multipart: form_data) do
|
||||
{:ok, %{status: status}} when status in 200..299 ->
|
||||
:ok
|
||||
|
||||
{:ok, %{status: status, body: body}} ->
|
||||
{:error, "HTTP #{status}: #{inspect(body)}"}
|
||||
|
||||
{:error, reason} ->
|
||||
{:error, reason}
|
||||
end
|
||||
end
|
||||
|
||||
defp import_profiles(definitions_path, profiles_filter) do
|
||||
# Call the existing import_profiles task
|
||||
args =
|
||||
case profiles_filter do
|
||||
nil ->
|
||||
["--source-path", definitions_path]
|
||||
|
||||
profiles ->
|
||||
["--source-path", definitions_path, "--profiles", profiles]
|
||||
end
|
||||
|
||||
Mix.Task.run("import_profiles", args)
|
||||
end
|
||||
end
|
||||
113
lib/towerops/profiles.ex
Normal file
113
lib/towerops/profiles.ex
Normal file
|
|
@ -0,0 +1,113 @@
|
|||
defmodule Towerops.Profiles do
|
||||
@moduledoc """
|
||||
Context for managing device profiles.
|
||||
Profiles are stored in the database and can be updated without deployment.
|
||||
"""
|
||||
|
||||
import Ecto.Query
|
||||
|
||||
alias Towerops.Profiles.DeviceOid
|
||||
alias Towerops.Profiles.DeviceProfile
|
||||
alias Towerops.Profiles.SensorOid
|
||||
alias Towerops.Repo
|
||||
|
||||
@doc """
|
||||
Lists all enabled profiles ordered by priority.
|
||||
"""
|
||||
def list_profiles do
|
||||
DeviceProfile
|
||||
|> where([p], p.enabled == true)
|
||||
|> order_by([p], asc: p.priority)
|
||||
|> Repo.all()
|
||||
end
|
||||
|
||||
@doc """
|
||||
Gets a profile by name with associations loaded.
|
||||
"""
|
||||
def get_profile(name) do
|
||||
DeviceProfile
|
||||
|> where([p], p.name == ^name)
|
||||
|> preload([:device_oids, :sensor_oids])
|
||||
|> Repo.one()
|
||||
end
|
||||
|
||||
@doc """
|
||||
Matches a profile based on sysDescr and sysObjectID.
|
||||
Returns the first matching profile based on priority.
|
||||
"""
|
||||
def match_profile(system_info) do
|
||||
sys_descr = Map.get(system_info, :sys_descr, "")
|
||||
sys_object_id = Map.get(system_info, :sys_object_id, "")
|
||||
|
||||
profiles = list_profiles()
|
||||
|
||||
Enum.find(profiles, fn profile ->
|
||||
matches_detection_pattern?(profile, sys_descr) ||
|
||||
matches_detection_oid?(profile, sys_object_id)
|
||||
end)
|
||||
end
|
||||
|
||||
@doc """
|
||||
Creates a new profile.
|
||||
"""
|
||||
def create_profile(attrs) do
|
||||
%DeviceProfile{}
|
||||
|> DeviceProfile.changeset(attrs)
|
||||
|> Repo.insert()
|
||||
end
|
||||
|
||||
@doc """
|
||||
Updates a profile.
|
||||
"""
|
||||
def update_profile(%DeviceProfile{} = profile, attrs) do
|
||||
profile
|
||||
|> DeviceProfile.changeset(attrs)
|
||||
|> Repo.update()
|
||||
end
|
||||
|
||||
@doc """
|
||||
Deletes a profile.
|
||||
"""
|
||||
def delete_profile(%DeviceProfile{} = profile) do
|
||||
Repo.delete(profile)
|
||||
end
|
||||
|
||||
@doc """
|
||||
Adds a device OID to a profile.
|
||||
"""
|
||||
def add_device_oid(profile_id, field, mib_name) do
|
||||
%DeviceOid{}
|
||||
|> DeviceOid.changeset(%{
|
||||
profile_id: profile_id,
|
||||
field: field,
|
||||
mib_name: mib_name
|
||||
})
|
||||
|> Repo.insert()
|
||||
end
|
||||
|
||||
@doc """
|
||||
Adds a sensor OID to a profile.
|
||||
"""
|
||||
def add_sensor_oid(profile_id, attrs) do
|
||||
%SensorOid{}
|
||||
|> SensorOid.changeset(Map.put(attrs, :profile_id, profile_id))
|
||||
|> Repo.insert()
|
||||
end
|
||||
|
||||
# Private functions
|
||||
|
||||
defp matches_detection_pattern?(%{detection_pattern: nil}, _sys_descr), do: false
|
||||
|
||||
defp matches_detection_pattern?(%{detection_pattern: pattern}, sys_descr) do
|
||||
case Regex.compile(pattern) do
|
||||
{:ok, regex} -> Regex.match?(regex, sys_descr)
|
||||
{:error, _} -> false
|
||||
end
|
||||
end
|
||||
|
||||
defp matches_detection_oid?(%{detection_oid: nil}, _sys_object_id), do: false
|
||||
|
||||
defp matches_detection_oid?(%{detection_oid: pattern}, sys_object_id) do
|
||||
String.starts_with?(sys_object_id, pattern)
|
||||
end
|
||||
end
|
||||
31
lib/towerops/profiles/device_oid.ex
Normal file
31
lib/towerops/profiles/device_oid.ex
Normal file
|
|
@ -0,0 +1,31 @@
|
|||
defmodule Towerops.Profiles.DeviceOid do
|
||||
@moduledoc """
|
||||
Schema for device identification OIDs.
|
||||
Maps device info fields (serial_number, firmware_version, etc.) to MIB names.
|
||||
"""
|
||||
|
||||
use Ecto.Schema
|
||||
|
||||
import Ecto.Changeset
|
||||
|
||||
alias Towerops.Profiles.DeviceProfile
|
||||
|
||||
@primary_key {:id, :binary_id, autogenerate: true}
|
||||
@foreign_key_type :binary_id
|
||||
|
||||
schema "profile_device_oids" do
|
||||
field :field, :string
|
||||
field :mib_name, :string
|
||||
|
||||
belongs_to :profile, DeviceProfile, foreign_key: :profile_id
|
||||
|
||||
timestamps(type: :utc_datetime)
|
||||
end
|
||||
|
||||
def changeset(device_oid, attrs) do
|
||||
device_oid
|
||||
|> cast(attrs, [:field, :mib_name, :profile_id])
|
||||
|> validate_required([:field, :mib_name, :profile_id])
|
||||
|> unique_constraint([:profile_id, :field])
|
||||
end
|
||||
end
|
||||
37
lib/towerops/profiles/device_profile.ex
Normal file
37
lib/towerops/profiles/device_profile.ex
Normal file
|
|
@ -0,0 +1,37 @@
|
|||
defmodule Towerops.Profiles.DeviceProfile do
|
||||
@moduledoc """
|
||||
Schema for device profiles stored in database.
|
||||
Profiles define how to identify and discover sensors for different device types.
|
||||
"""
|
||||
|
||||
use Ecto.Schema
|
||||
|
||||
import Ecto.Changeset
|
||||
|
||||
alias Towerops.Profiles.DeviceOid
|
||||
alias Towerops.Profiles.SensorOid
|
||||
|
||||
@primary_key {:id, :binary_id, autogenerate: true}
|
||||
@foreign_key_type :binary_id
|
||||
|
||||
schema "device_profiles" do
|
||||
field :name, :string
|
||||
field :vendor, :string
|
||||
field :detection_pattern, :string
|
||||
field :detection_oid, :string
|
||||
field :priority, :integer, default: 100
|
||||
field :enabled, :boolean, default: true
|
||||
|
||||
has_many :device_oids, DeviceOid, foreign_key: :profile_id
|
||||
has_many :sensor_oids, SensorOid, foreign_key: :profile_id
|
||||
|
||||
timestamps(type: :utc_datetime)
|
||||
end
|
||||
|
||||
def changeset(profile, attrs) do
|
||||
profile
|
||||
|> cast(attrs, [:name, :vendor, :detection_pattern, :detection_oid, :priority, :enabled])
|
||||
|> validate_required([:name, :vendor])
|
||||
|> unique_constraint(:name)
|
||||
end
|
||||
end
|
||||
33
lib/towerops/profiles/sensor_oid.ex
Normal file
33
lib/towerops/profiles/sensor_oid.ex
Normal file
|
|
@ -0,0 +1,33 @@
|
|||
defmodule Towerops.Profiles.SensorOid do
|
||||
@moduledoc """
|
||||
Schema for sensor discovery OIDs.
|
||||
Maps sensor types to MIB names for sensor discovery.
|
||||
"""
|
||||
|
||||
use Ecto.Schema
|
||||
|
||||
import Ecto.Changeset
|
||||
|
||||
alias Towerops.Profiles.DeviceProfile
|
||||
|
||||
@primary_key {:id, :binary_id, autogenerate: true}
|
||||
@foreign_key_type :binary_id
|
||||
|
||||
schema "profile_sensor_oids" do
|
||||
field :sensor_type, :string
|
||||
field :mib_name, :string
|
||||
field :sensor_descr, :string
|
||||
field :sensor_unit, :string
|
||||
field :sensor_divisor, :integer, default: 1
|
||||
|
||||
belongs_to :profile, DeviceProfile, foreign_key: :profile_id
|
||||
|
||||
timestamps(type: :utc_datetime)
|
||||
end
|
||||
|
||||
def changeset(sensor_oid, attrs) do
|
||||
sensor_oid
|
||||
|> cast(attrs, [:sensor_type, :mib_name, :sensor_descr, :sensor_unit, :sensor_divisor, :profile_id])
|
||||
|> validate_required([:sensor_type, :mib_name, :profile_id])
|
||||
end
|
||||
end
|
||||
|
|
@ -16,6 +16,8 @@ defmodule Towerops.Snmp.Discovery do
|
|||
|
||||
alias Towerops.Devices
|
||||
alias Towerops.Devices.Device, as: DeviceSchema
|
||||
alias Towerops.Profiles
|
||||
alias Towerops.Profiles.DeviceProfile
|
||||
alias Towerops.Repo
|
||||
alias Towerops.Snmp.Client
|
||||
alias Towerops.Snmp.Device
|
||||
|
|
@ -23,6 +25,7 @@ defmodule Towerops.Snmp.Discovery do
|
|||
alias Towerops.Snmp.NeighborDiscovery
|
||||
alias Towerops.Snmp.Profiles.Base
|
||||
alias Towerops.Snmp.Profiles.Cisco
|
||||
alias Towerops.Snmp.Profiles.Dynamic
|
||||
alias Towerops.Snmp.Profiles.Mikrotik
|
||||
alias Towerops.Snmp.Profiles.NetSnmp
|
||||
alias Towerops.Snmp.Profiles.Ubiquiti
|
||||
|
|
@ -74,7 +77,7 @@ defmodule Towerops.Snmp.Discovery do
|
|||
optional(atom()) => term()
|
||||
}
|
||||
|
||||
@type profile :: module()
|
||||
@type profile :: module() | {:dynamic, DeviceProfile.t()}
|
||||
|
||||
@type discovery_summary :: %{
|
||||
success: non_neg_integer(),
|
||||
|
|
@ -207,6 +210,19 @@ defmodule Towerops.Snmp.Discovery do
|
|||
"""
|
||||
@spec select_profile(system_info()) :: profile()
|
||||
def select_profile(system_info) do
|
||||
# First try database profiles
|
||||
case Profiles.match_profile(system_info) do
|
||||
%DeviceProfile{} = profile ->
|
||||
Logger.debug("Selected database profile: #{profile.name}")
|
||||
{:dynamic, profile}
|
||||
|
||||
nil ->
|
||||
# Fall back to hardcoded profiles
|
||||
select_hardcoded_profile(system_info)
|
||||
end
|
||||
end
|
||||
|
||||
defp select_hardcoded_profile(system_info) do
|
||||
sys_descr = Map.get(system_info, :sys_descr, "")
|
||||
sys_object_id = Map.get(system_info, :sys_object_id, "")
|
||||
|
||||
|
|
@ -237,6 +253,16 @@ defmodule Towerops.Snmp.Discovery do
|
|||
|
||||
@spec build_device_info(Client.connection_opts(), system_info(), profile()) ::
|
||||
{:ok, device_info()}
|
||||
defp build_device_info(client_opts, system_info, {:dynamic, profile}) do
|
||||
# Use dynamic profile to identify device
|
||||
identified_info = Dynamic.identify_device(profile, client_opts, system_info)
|
||||
{:ok, identified_info}
|
||||
rescue
|
||||
error ->
|
||||
Logger.error("Failed to identify device with dynamic profile: #{inspect(error)}")
|
||||
{:ok, system_info}
|
||||
end
|
||||
|
||||
defp build_device_info(_client_opts, system_info, profile) when is_atom(profile) do
|
||||
# Let the hard-coded profile identify the device
|
||||
identified_info = profile.identify_device(system_info)
|
||||
|
|
@ -249,6 +275,18 @@ defmodule Towerops.Snmp.Discovery do
|
|||
|
||||
@spec discover_interfaces(Client.connection_opts(), profile()) ::
|
||||
{:ok, [interface_data()]}
|
||||
defp discover_interfaces(client_opts, {:dynamic, profile}) do
|
||||
case Dynamic.discover_interfaces(profile, client_opts) do
|
||||
{:ok, interfaces} ->
|
||||
Logger.debug("Discovered #{length(interfaces)} interfaces")
|
||||
{:ok, interfaces}
|
||||
|
||||
{:error, _} ->
|
||||
Logger.warning("Interface discovery failed, continuing without interfaces")
|
||||
{:ok, []}
|
||||
end
|
||||
end
|
||||
|
||||
defp discover_interfaces(client_opts, profile) when is_atom(profile) do
|
||||
case profile.discover_interfaces(client_opts) do
|
||||
{:ok, interfaces} ->
|
||||
|
|
@ -262,6 +300,12 @@ defmodule Towerops.Snmp.Discovery do
|
|||
end
|
||||
|
||||
@spec discover_sensors(Client.connection_opts(), profile()) :: {:ok, [sensor_data()]}
|
||||
defp discover_sensors(client_opts, {:dynamic, profile}) do
|
||||
{:ok, sensors} = Dynamic.discover_sensors(profile, client_opts)
|
||||
Logger.debug("Discovered #{length(sensors)} sensors")
|
||||
{:ok, sensors}
|
||||
end
|
||||
|
||||
defp discover_sensors(client_opts, profile) when is_atom(profile) do
|
||||
case profile.discover_sensors(client_opts) do
|
||||
{:ok, sensors} ->
|
||||
|
|
|
|||
117
lib/towerops/snmp/profiles/dynamic.ex
Normal file
117
lib/towerops/snmp/profiles/dynamic.ex
Normal file
|
|
@ -0,0 +1,117 @@
|
|||
defmodule Towerops.Snmp.Profiles.Dynamic do
|
||||
@moduledoc """
|
||||
Dynamic profile that uses database-stored profiles with MIB name translation.
|
||||
Allows profiles to be updated without redeployment.
|
||||
"""
|
||||
|
||||
alias Towerops.Profiles
|
||||
alias Towerops.Profiles.DeviceProfile
|
||||
alias Towerops.Snmp.Client
|
||||
alias Towerops.Snmp.Discovery
|
||||
alias Towerops.Snmp.MibTranslator
|
||||
|
||||
require Logger
|
||||
|
||||
@doc """
|
||||
Identifies device using profile's device_oids.
|
||||
Translates MIB names to numeric OIDs and polls the device.
|
||||
"""
|
||||
@spec identify_device(DeviceProfile.t(), Client.connection_opts(), Discovery.system_info()) ::
|
||||
Discovery.device_info()
|
||||
def identify_device(profile, client_opts, system_info) do
|
||||
# Get device OIDs from profile
|
||||
profile = Profiles.get_profile(profile.name)
|
||||
|
||||
device_data =
|
||||
profile.device_oids
|
||||
|> Enum.map(fn device_oid ->
|
||||
case MibTranslator.translate(device_oid.mib_name) do
|
||||
{:ok, numeric_oid} ->
|
||||
case Client.get(client_opts, numeric_oid) do
|
||||
{:ok, value} -> {String.to_atom(device_oid.field), value}
|
||||
{:error, _} -> nil
|
||||
end
|
||||
|
||||
{:error, _} ->
|
||||
Logger.warning("Failed to translate MIB name: #{device_oid.mib_name}")
|
||||
nil
|
||||
end
|
||||
end)
|
||||
|> Enum.reject(&is_nil/1)
|
||||
|> Map.new()
|
||||
|
||||
# Format firmware version for MikroTik devices
|
||||
firmware_version =
|
||||
case {Map.get(device_data, :firmware_version), Map.get(device_data, :license_version)} do
|
||||
{fw, license} when not is_nil(fw) and not is_nil(license) ->
|
||||
"#{profile.vendor} RouterOS #{fw} (Level #{license})"
|
||||
|
||||
{fw, _} when not is_nil(fw) ->
|
||||
"#{profile.vendor} RouterOS #{fw}"
|
||||
|
||||
_ ->
|
||||
nil
|
||||
end
|
||||
|
||||
# Merge with system_info
|
||||
system_info
|
||||
|> Map.merge(%{
|
||||
manufacturer: profile.vendor,
|
||||
model: system_info.sys_descr,
|
||||
firmware_version: firmware_version,
|
||||
serial_number: Map.get(device_data, :serial_number)
|
||||
})
|
||||
|> Map.merge(device_data)
|
||||
end
|
||||
|
||||
@doc """
|
||||
Discovers sensors using profile's sensor_oids.
|
||||
Translates MIB names and polls sensors.
|
||||
"""
|
||||
@spec discover_sensors(DeviceProfile.t(), Client.connection_opts()) ::
|
||||
{:ok, [Discovery.sensor_data()]}
|
||||
def discover_sensors(profile, client_opts) do
|
||||
profile = Profiles.get_profile(profile.name)
|
||||
|
||||
sensors =
|
||||
profile.sensor_oids
|
||||
|> Enum.map(fn sensor_oid ->
|
||||
case MibTranslator.translate(sensor_oid.mib_name) do
|
||||
{:ok, numeric_oid} ->
|
||||
case Client.get(client_opts, numeric_oid) do
|
||||
{:ok, value} when is_integer(value) and value > 0 ->
|
||||
%{
|
||||
sensor_type: sensor_oid.sensor_type,
|
||||
sensor_index: sensor_oid.sensor_type,
|
||||
sensor_oid: numeric_oid,
|
||||
sensor_descr: sensor_oid.sensor_descr || sensor_oid.sensor_type,
|
||||
sensor_unit: sensor_oid.sensor_unit || "",
|
||||
sensor_divisor: sensor_oid.sensor_divisor || 1,
|
||||
sensor_value: value
|
||||
}
|
||||
|
||||
_ ->
|
||||
nil
|
||||
end
|
||||
|
||||
{:error, _} ->
|
||||
Logger.warning("Failed to translate sensor MIB name: #{sensor_oid.mib_name}")
|
||||
nil
|
||||
end
|
||||
end)
|
||||
|> Enum.reject(&is_nil/1)
|
||||
|
||||
{:ok, sensors}
|
||||
end
|
||||
|
||||
@doc """
|
||||
Uses Base profile for interface discovery.
|
||||
"""
|
||||
@spec discover_interfaces(DeviceProfile.t(), Client.connection_opts()) ::
|
||||
{:ok, [Discovery.interface_data()]}
|
||||
def discover_interfaces(_profile, client_opts) do
|
||||
alias Towerops.Snmp.Profiles.Base
|
||||
|
||||
Base.discover_interfaces(client_opts)
|
||||
end
|
||||
end
|
||||
|
|
@ -149,21 +149,52 @@ defmodule Towerops.Snmp.Profiles.Mikrotik do
|
|||
Map.merge(system_info, %{
|
||||
manufacturer: "MikroTik",
|
||||
model: model,
|
||||
firmware_version: formatted_firmware
|
||||
firmware_version: formatted_firmware,
|
||||
serial_number: Map.get(system_info, :serial_number)
|
||||
})
|
||||
end
|
||||
|
||||
# Private functions
|
||||
|
||||
defp discover_mikrotik_system_info(client_opts) do
|
||||
oids = [
|
||||
@mikrotik_oids.serial_number,
|
||||
@mikrotik_oids.firmware_version,
|
||||
@mikrotik_oids.board_name,
|
||||
@mikrotik_oids.display_name,
|
||||
@mikrotik_oids.license_level
|
||||
alias Towerops.Snmp.MibTranslator
|
||||
|
||||
# Use MIB symbolic names instead of hardcoded OIDs
|
||||
mib_names = [
|
||||
"MIKROTIK-MIB::mtxrSerialNumber.0",
|
||||
"MIKROTIK-MIB::mtxrFirmwareVersion.0",
|
||||
"MIKROTIK-MIB::mtxrBoardName.0",
|
||||
"MIKROTIK-MIB::mtxrSystemDisplayName.0",
|
||||
"MIKROTIK-MIB::mtxrLicenseVersion.0"
|
||||
]
|
||||
|
||||
# Translate MIB names to numeric OIDs
|
||||
translated = MibTranslator.translate_batch(mib_names)
|
||||
|
||||
oids =
|
||||
Enum.map(mib_names, fn name ->
|
||||
case Map.get(translated, name) do
|
||||
{:ok, oid} -> oid
|
||||
_ -> nil
|
||||
end
|
||||
end)
|
||||
|
||||
# If any translation failed, fall back to hardcoded OIDs
|
||||
oids =
|
||||
if Enum.any?(oids, &is_nil/1) do
|
||||
Logger.warning("MIB translation failed for some MikroTik OIDs, using hardcoded values")
|
||||
|
||||
[
|
||||
@mikrotik_oids.serial_number,
|
||||
@mikrotik_oids.firmware_version,
|
||||
@mikrotik_oids.board_name,
|
||||
@mikrotik_oids.display_name,
|
||||
@mikrotik_oids.license_level
|
||||
]
|
||||
else
|
||||
oids
|
||||
end
|
||||
|
||||
case Client.get_multiple(client_opts, oids) do
|
||||
{:ok, [serial, firmware, board, display, license_level]} ->
|
||||
{:ok,
|
||||
|
|
|
|||
|
|
@ -0,0 +1,57 @@
|
|||
defmodule Towerops.Repo.Migrations.CreateSimpleDeviceProfiles do
|
||||
use Ecto.Migration
|
||||
|
||||
def change do
|
||||
# Main profile table
|
||||
create table(:device_profiles, primary_key: false) do
|
||||
add :id, :binary_id, primary_key: true
|
||||
add :name, :string, null: false
|
||||
add :vendor, :string, null: false
|
||||
add :detection_pattern, :text
|
||||
add :detection_oid, :text
|
||||
add :priority, :integer, default: 100
|
||||
add :enabled, :boolean, default: true
|
||||
|
||||
timestamps(type: :utc_datetime)
|
||||
end
|
||||
|
||||
create unique_index(:device_profiles, [:name])
|
||||
create index(:device_profiles, [:enabled])
|
||||
create index(:device_profiles, [:priority])
|
||||
|
||||
# Device identification OIDs (serial number, firmware version, model, etc.)
|
||||
create table(:profile_device_oids, primary_key: false) do
|
||||
add :id, :binary_id, primary_key: true
|
||||
|
||||
add :profile_id,
|
||||
references(:device_profiles, type: :binary_id, on_delete: :delete_all), null: false
|
||||
|
||||
add :field, :string, null: false
|
||||
add :mib_name, :string, null: false
|
||||
|
||||
timestamps(type: :utc_datetime)
|
||||
end
|
||||
|
||||
create index(:profile_device_oids, [:profile_id])
|
||||
create unique_index(:profile_device_oids, [:profile_id, :field])
|
||||
|
||||
# Sensor discovery OIDs
|
||||
create table(:profile_sensor_oids, primary_key: false) do
|
||||
add :id, :binary_id, primary_key: true
|
||||
|
||||
add :profile_id,
|
||||
references(:device_profiles, type: :binary_id, on_delete: :delete_all), null: false
|
||||
|
||||
add :sensor_type, :string, null: false
|
||||
add :mib_name, :string, null: false
|
||||
add :sensor_descr, :string
|
||||
add :sensor_unit, :string
|
||||
add :sensor_divisor, :integer, default: 1
|
||||
|
||||
timestamps(type: :utc_datetime)
|
||||
end
|
||||
|
||||
create index(:profile_sensor_oids, [:profile_id])
|
||||
create index(:profile_sensor_oids, [:sensor_type])
|
||||
end
|
||||
end
|
||||
141
priv/repo/migrations/20260118223700_seed_mikrotik_profile.exs
Normal file
141
priv/repo/migrations/20260118223700_seed_mikrotik_profile.exs
Normal file
|
|
@ -0,0 +1,141 @@
|
|||
defmodule Towerops.Repo.Migrations.SeedMikrotikProfile do
|
||||
use Ecto.Migration
|
||||
|
||||
import Ecto.Query
|
||||
|
||||
def up do
|
||||
# Create MikroTik profile
|
||||
profile_id = Ecto.UUID.dump!(Ecto.UUID.generate())
|
||||
|
||||
repo().insert_all("device_profiles", [
|
||||
%{
|
||||
id: profile_id,
|
||||
name: "mikrotik",
|
||||
vendor: "MikroTik",
|
||||
detection_pattern: "RouterOS",
|
||||
detection_oid: ".1.3.6.1.4.1.14988",
|
||||
priority: 10,
|
||||
enabled: true,
|
||||
inserted_at: DateTime.utc_now() |> DateTime.truncate(:second),
|
||||
updated_at: DateTime.utc_now() |> DateTime.truncate(:second)
|
||||
}
|
||||
])
|
||||
|
||||
# Device identification OIDs using MIB names
|
||||
device_oids = [
|
||||
%{
|
||||
id: Ecto.UUID.dump!(Ecto.UUID.generate()),
|
||||
profile_id: profile_id,
|
||||
field: "serial_number",
|
||||
mib_name: "MIKROTIK-MIB::mtxrSerialNumber.0",
|
||||
inserted_at: DateTime.utc_now() |> DateTime.truncate(:second),
|
||||
updated_at: DateTime.utc_now() |> DateTime.truncate(:second)
|
||||
},
|
||||
%{
|
||||
id: Ecto.UUID.dump!(Ecto.UUID.generate()),
|
||||
profile_id: profile_id,
|
||||
field: "firmware_version",
|
||||
mib_name: "MIKROTIK-MIB::mtxrFirmwareVersion.0",
|
||||
inserted_at: DateTime.utc_now() |> DateTime.truncate(:second),
|
||||
updated_at: DateTime.utc_now() |> DateTime.truncate(:second)
|
||||
},
|
||||
%{
|
||||
id: Ecto.UUID.dump!(Ecto.UUID.generate()),
|
||||
profile_id: profile_id,
|
||||
field: "license_version",
|
||||
mib_name: "MIKROTIK-MIB::mtxrLicenseVersion.0",
|
||||
inserted_at: DateTime.utc_now() |> DateTime.truncate(:second),
|
||||
updated_at: DateTime.utc_now() |> DateTime.truncate(:second)
|
||||
},
|
||||
%{
|
||||
id: Ecto.UUID.dump!(Ecto.UUID.generate()),
|
||||
profile_id: profile_id,
|
||||
field: "board_name",
|
||||
mib_name: "MIKROTIK-MIB::mtxrBoardName.0",
|
||||
inserted_at: DateTime.utc_now() |> DateTime.truncate(:second),
|
||||
updated_at: DateTime.utc_now() |> DateTime.truncate(:second)
|
||||
}
|
||||
]
|
||||
|
||||
repo().insert_all("profile_device_oids", device_oids)
|
||||
|
||||
# Sensor OIDs using MIB names
|
||||
sensor_oids = [
|
||||
# Temperatures
|
||||
%{
|
||||
id: Ecto.UUID.dump!(Ecto.UUID.generate()),
|
||||
profile_id: profile_id,
|
||||
sensor_type: "temperature",
|
||||
mib_name: "MIKROTIK-MIB::mtxrHlTemperature.0",
|
||||
sensor_descr: "System Temperature",
|
||||
sensor_unit: "°C",
|
||||
sensor_divisor: 10,
|
||||
inserted_at: DateTime.utc_now() |> DateTime.truncate(:second),
|
||||
updated_at: DateTime.utc_now() |> DateTime.truncate(:second)
|
||||
},
|
||||
%{
|
||||
id: Ecto.UUID.dump!(Ecto.UUID.generate()),
|
||||
profile_id: profile_id,
|
||||
sensor_type: "cpu_temperature",
|
||||
mib_name: "MIKROTIK-MIB::mtxrHlCpuTemperature.0",
|
||||
sensor_descr: "CPU Temperature",
|
||||
sensor_unit: "°C",
|
||||
sensor_divisor: 10,
|
||||
inserted_at: DateTime.utc_now() |> DateTime.truncate(:second),
|
||||
updated_at: DateTime.utc_now() |> DateTime.truncate(:second)
|
||||
},
|
||||
%{
|
||||
id: Ecto.UUID.dump!(Ecto.UUID.generate()),
|
||||
profile_id: profile_id,
|
||||
sensor_type: "board_temperature",
|
||||
mib_name: "MIKROTIK-MIB::mtxrHlBoardTemperature.0",
|
||||
sensor_descr: "Board Temperature",
|
||||
sensor_unit: "°C",
|
||||
sensor_divisor: 10,
|
||||
inserted_at: DateTime.utc_now() |> DateTime.truncate(:second),
|
||||
updated_at: DateTime.utc_now() |> DateTime.truncate(:second)
|
||||
},
|
||||
# Voltages
|
||||
%{
|
||||
id: Ecto.UUID.dump!(Ecto.UUID.generate()),
|
||||
profile_id: profile_id,
|
||||
sensor_type: "voltage",
|
||||
mib_name: "MIKROTIK-MIB::mtxrHlVoltage.0",
|
||||
sensor_descr: "Input Voltage",
|
||||
sensor_unit: "V",
|
||||
sensor_divisor: 10,
|
||||
inserted_at: DateTime.utc_now() |> DateTime.truncate(:second),
|
||||
updated_at: DateTime.utc_now() |> DateTime.truncate(:second)
|
||||
},
|
||||
# Power & Current
|
||||
%{
|
||||
id: Ecto.UUID.dump!(Ecto.UUID.generate()),
|
||||
profile_id: profile_id,
|
||||
sensor_type: "current",
|
||||
mib_name: "MIKROTIK-MIB::mtxrHlCurrent.0",
|
||||
sensor_descr: "Current",
|
||||
sensor_unit: "mA",
|
||||
sensor_divisor: 1,
|
||||
inserted_at: DateTime.utc_now() |> DateTime.truncate(:second),
|
||||
updated_at: DateTime.utc_now() |> DateTime.truncate(:second)
|
||||
},
|
||||
%{
|
||||
id: Ecto.UUID.dump!(Ecto.UUID.generate()),
|
||||
profile_id: profile_id,
|
||||
sensor_type: "power",
|
||||
mib_name: "MIKROTIK-MIB::mtxrHlPower.0",
|
||||
sensor_descr: "Power Consumption",
|
||||
sensor_unit: "W",
|
||||
sensor_divisor: 10,
|
||||
inserted_at: DateTime.utc_now() |> DateTime.truncate(:second),
|
||||
updated_at: DateTime.utc_now() |> DateTime.truncate(:second)
|
||||
}
|
||||
]
|
||||
|
||||
repo().insert_all("profile_sensor_oids", sensor_oids)
|
||||
end
|
||||
|
||||
def down do
|
||||
repo().delete_all(from(p in "device_profiles", where: p.name == "mikrotik"))
|
||||
end
|
||||
end
|
||||
Loading…
Add table
Reference in a new issue