# Preseem API Integration Design

## Overview

Integrate Preseem's QoE (Quality of Experience) data with Towerops' SNMP-based device monitoring to provide network engineers with combined infrastructure health + traffic quality intelligence.

**Primary persona (Phase 1):** Network engineer — planning capacity, upgrading towers, optimizing RF.

**Future personas:** NOC technician (alert correlation), ISP owner/manager (investment prioritization). See `TODO.md` at project root.

## What Preseem Provides

Preseem sits inline on WISP networks and measures per-subscriber, per-AP, per-tower:
- **QoE metrics**: latency, loss, jitter, throughput (per subscriber and aggregate)
- **AP capacity scores**: airtime utilization, busy hours, subscriber density
- **RF metrics**: RSSI, modulation, frequency, channel width
- **Subscriber data**: plan rates, actual throughput, data usage

Preseem has a Model API (beta, in production use) and REST APIs.

## What Towerops Provides

- SNMP-based device inventory (hardware, firmware, serial numbers)
- 570+ vendor device profiles
- Network topology (LLDP/CDP neighbors, ARP discovery)
- Sensor readings (temperature, voltage, power, signal strength)
- Interface statistics (traffic counters, errors, discards)
- MikroTik configuration tracking

## The Value of Combining Both

Towerops knows *about devices* (hardware, config, topology, sensor health).
Preseem knows *about traffic* (QoE, capacity, subscriber experience).

Together: "This AF5XHD at Tower 3 is running hot (Towerops sensor), serving 45 subscribers (Preseem), with QoE dropping during busy hours (Preseem), and its firmware is 2 versions behind the best-performing instances in your fleet (both)."

---

## Data Model

### Integration Settings

`integrations` — Per-organization, generic for future integrations.

| Column | Type | Notes |
|--------|------|-------|
| id | binary_id | PK |
| organization_id | binary_id | FK to organizations |
| provider | string | "preseem" (future: "splynx", "sonar", etc.) |
| enabled | boolean | default false |
| credentials | encrypted JSONB | API key, base URL (Cloak AES-256-GCM) |
| sync_interval_minutes | integer | default 10 |
| last_synced_at | utc_datetime | nullable |
| last_sync_status | string | "success", "partial", "failed", "never" |
| timestamps | utc_datetime | |

Unique constraint on `{organization_id, provider}`.

### Preseem Access Points

`preseem_access_points` — Central entity synced from Preseem API.

| Column | Type | Notes |
|--------|------|-------|
| id | binary_id | PK |
| organization_id | binary_id | FK, tenant isolation |
| preseem_id | string | Preseem's identifier, unique per org |
| name | string | AP name from Preseem |
| mac_address | string | for device matching |
| ip_address | string | for device matching |
| model | string | for fleet analysis |
| firmware | string | for fleet analysis |
| capacity_score | float | Preseem computed score |
| qoe_score | float | Preseem computed score |
| rf_score | float | Preseem computed score |
| busy_hours | integer | hours per day at/near capacity |
| airtime_utilization | float | percentage |
| subscriber_count | integer | connected subscribers |
| device_id | binary_id | nullable FK to devices (matched) |
| match_confidence | string | "auto_mac", "auto_ip", "auto_hostname", "manual", "ambiguous", "unmatched" |
| raw_data | map (JSONB) | full Preseem API response |
| timestamps | utc_datetime | |

Unique constraint on `{organization_id, preseem_id}`.

### Preseem Subscriber Metrics (Time-Series)

`preseem_subscriber_metrics` — Aggregate QoE per AP, stored as time-series.

| Column | Type | Notes |
|--------|------|-------|
| id | binary_id | PK |
| preseem_access_point_id | binary_id | FK |
| avg_latency | float | ms |
| avg_jitter | float | ms |
| avg_loss | float | percentage |
| avg_throughput | float | Mbps |
| p95_latency | float | ms, key QoE indicator |
| subscriber_count | integer | at time of measurement |
| recorded_at | utc_datetime | hypertable partition key |

### Preseem Sync Logs

`preseem_sync_logs` — Audit trail per sync operation.

| Column | Type | Notes |
|--------|------|-------|
| id | binary_id | PK |
| organization_id | binary_id | FK |
| integration_id | binary_id | FK |
| status | string | "success", "partial", "failed" |
| records_synced | integer | |
| errors | map (JSONB) | error details |
| duration_ms | integer | |
| inserted_at | utc_datetime | |

---

## Device Matching Engine

When sync pulls Preseem APs, a matching pipeline runs against Towerops devices (same org).

### Match Strategy (ordered by confidence)

1. **MAC address** — Normalize (lowercase, strip delimiters), match against `devices.mac_address` and `interfaces.mac_address`. Unique match → `:auto_mac`.
2. **IP address** — Match against `devices.ip_address`. Unique match → `:auto_ip`.
3. **Hostname** — Normalize (lowercase, strip domain suffix), match against `devices.name`. Unique match → `:auto_hostname`.
4. **No match** → `:unmatched`.

### Conflict Handling

If multiple Towerops devices match a single Preseem AP → `:ambiguous`, surfaced for manual review.

### Manual Override

- Link/unlink from integrations page (unmatched/ambiguous list)
- Link/unlink from device detail page
- Manual links stored as `:manual`, never overwritten by auto-matching

---

## Analysis Engine

### Individual Device Baselines

For each matched device, compute a rolling 14-day baseline (nightly Oban worker):

- **QoE baseline**: typical qoe_score, p95_latency, subscriber_count (busy vs off-peak)
- **Capacity baseline**: normal airtime_utilization, busy_hours, throughput ranges
- **RF baseline**: typical rf_score, Towerops SNMP signal sensors

Stored in `preseem_device_baselines`:
- Per-metric: mean, stddev, p5, p95
- Separate busy-hour vs off-peak profiles

When fresh sync data arrives, compare against baseline → flag deviations.

### Fleet Intelligence

Aggregate across all matched devices per org, grouped by device model (manufacturer + model from SNMP profile):

Stored in `preseem_fleet_profiles` keyed by `{organization_id, manufacturer, model}`:

- **Model performance profiles**: avg subscribers, avg QoE per model
- **Capacity ceilings**: subscriber count where QoE degrades per model
- **Firmware correlation**: QoE/capacity grouped by firmware version
- **Configuration patterns**: correlate config differences with performance (using MikroTik backup tracking)

Computed nightly.

---

## Insight Generation & Delivery

### Insight Types

| Insight | Source | Delivery |
|---------|--------|----------|
| QoE degradation on specific AP | Baseline deviation | Proactive alert |
| AP approaching capacity saturation | Baseline trend + fleet ceiling | Proactive alert |
| Firmware upgrade opportunity | Fleet firmware correlation | Contextual guidance |
| Frequency/interference conflict | Preseem RF + Towerops topology | Contextual guidance |
| Device model underperforming fleet avg | Fleet profile comparison | Passive feed |
| Subscriber growth toward capacity ceiling | Baseline trend + fleet intel | Passive feed |
| Config drift from best-performing peers | Config tracking + fleet correlation | Passive feed |

### Delivery Channels

1. **Proactive alerts** — Generated as Towerops monitoring checks. Uses existing alerting infrastructure (soft/hard state transitions, flapping detection, notifications).

2. **Contextual guidance** — On device detail page, new "Insights" section when device is matched to a Preseem AP. Dismissable recommendation cards.

3. **Passive insights feed** — New LiveView page under network section. Filterable, sortable list of all insights across the org.

---

## UI Components

### Org Settings > Integrations

- List of available integration providers with enable/disable toggle
- Per-integration config panel:
  - Preseem: API key field, optional base URL override
  - "Test Connection" button (validates API key)
  - Sync status, last sync time, record counts
  - Sync history log

### Device Detail > Preseem Tab (when matched)

- Current QoE score, capacity score, RF score
- Subscriber count and busy hours
- Trend charts (from preseem_subscriber_metrics time-series)
- Contextual insights/recommendations
- Link to Preseem dashboard for this AP

### Unmatched Devices Page (under Integrations)

- List of Preseem APs not yet matched to Towerops devices
- Ambiguous matches needing resolution
- Search/select to manually link

### Network Insights Page

- All generated insights across the org
- Filter by: type, urgency, device, site
- Sortable by urgency, date, device
- Bulk dismiss

---

## Sync Architecture

### PreseemSyncWorker (Oban Cron)

Runs every 10 minutes (configurable per org). For each org with enabled Preseem integration:

1. Fetch API key from encrypted credentials
2. Pull AP list from Preseem Model API
3. Upsert `preseem_access_points` (by preseem_id)
4. Pull QoE metrics per AP
5. Insert `preseem_subscriber_metrics` time-series records
6. Run device matching pipeline for unmatched/new APs
7. Log sync result to `preseem_sync_logs`

### PreseemBaselineWorker (Oban Cron, nightly)

1. Compute individual device baselines from 14-day window
2. Compute fleet profiles grouped by model
3. Compare current data against baselines
4. Generate insights for deviations

### Preseem API Client

`Towerops.Preseem.Client` — Uses `:req` (only approved HTTP client).

- `list_access_points/1` — Pull all APs
- `get_access_point_metrics/2` — QoE metrics for specific AP
- `test_connection/1` — Validate API key

---

## Context Structure

```
Towerops.Integrations          — Generic integration CRUD (settings, credentials)
Towerops.Preseem               — Preseem-specific context
Towerops.Preseem.Client        — API client (Req-based)
Towerops.Preseem.Sync          — Sync logic (upsert, metrics insert)
Towerops.Preseem.DeviceMatcher — Matching pipeline
Towerops.Preseem.Baseline      — Individual device baselining
Towerops.Preseem.FleetIntelligence — Fleet-wide model analysis
Towerops.Preseem.Insights      — Insight generation and storage
```

---

## Implementation Stages

### Stage 1: Foundation
- `integrations` schema + migration
- Org Settings > Integrations UI (generic)
- Preseem API client with test connection
- Encrypted credential storage

### Stage 2: Data Sync
- `preseem_access_points` + `preseem_subscriber_metrics` schemas + migrations
- `preseem_sync_logs` schema + migration
- PreseemSyncWorker
- Device matching engine

### Stage 3: UI — Device Integration
- Preseem tab on device detail page
- Unmatched devices management page
- Manual link/unlink UI

### Stage 4: Analysis Engine
- `preseem_device_baselines` + `preseem_fleet_profiles` schemas
- PreseemBaselineWorker (nightly)
- Fleet intelligence computation

### Stage 5: Insights & Guidance
- Insight generation from baselines + fleet data
- Proactive alerts (monitoring check integration)
- Contextual guidance on device pages
- Network insights feed page
