diff --git a/CLAUDE.md b/CLAUDE.md index a054a31f..f3684d7c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -392,6 +392,54 @@ Two separate SNMP collection mechanisms: - Applied to both `DevicePollerWorker` and `DeviceMonitorWorker` - Implementation: `lib/towerops/workers/polling_offset.ex` +### MikroTik API Integration + +Provides MikroTik RouterOS API access alongside SNMP for enhanced device management. + +**Architecture:** +- **3-tier credential cascade**: Organization → Site → Device (identical to SNMP pattern) +- **Encryption**: Passwords encrypted at rest using Cloak (AES-256-GCM) +- **Detection**: Auto-detected from SNMP manufacturer field (contains "MikroTik" or "RouterOS") +- **Transport**: Supports both API-SSL (port 8729, default) and plain API (port 8728, insecure) +- **Access**: Superuser-only in UI (configurable) +- **Alongside SNMP**: Both can be enabled simultaneously + +**Key Files:** +- `lib/towerops/vault.ex` - Cloak encryption vault +- `lib/towerops/ecto_types/encrypted_binary.ex` - Encrypted field type +- `lib/towerops/devices.ex` - `get_mikrotik_config/1`, `propagate_site_mikrotik_change/2`, `propagate_organization_mikrotik_change/2` +- `lib/towerops_web/channels/agent_channel.ex` - `build_mikrotik_job/1`, `build_mikrotik_commands/0` +- `lib/towerops_web/live/device_live/form.ex` - Conditional UI display for MikroTik devices +- `priv/proto/agent.proto` - MikrotikDevice, MikrotikCommand, MikrotikResult protobuf messages + +**Database Schema:** +- `organizations.mikrotik_*` - Organization-level defaults +- `sites.mikrotik_*` - Site-level overrides (all nullable) +- `devices.mikrotik_*` - Device-level overrides (all nullable) +- `devices.mikrotik_credential_source` - Tracks inheritance ("device", "site", or default) + +**Credential Resolution:** +Each field (username, password, port, use_ssl, enabled) resolves independently: +```elixir +username = device.mikrotik_username || + device.site.mikrotik_username || + device.site.organization.mikrotik_username +``` + +**Security:** +- Passwords encrypted at rest using AES-256-GCM via Cloak +- Encryption key must be set in `CLOAK_KEY` environment variable (base64-encoded 32-byte key) +- Generate key: `openssl rand -base64 32` +- Development/test use fixed key in config files (DO NOT use in production) +- **Production**: See "Kubernetes Deployment" section below for secret creation commands +- Plain API (port 8728) is blocked when using cloud pollers (enforced in backend validation) +- UI warns about plain API security risks + +**Agent Integration:** +- MikroTik jobs sent via protobuf alongside SNMP jobs +- Agent executes RouterOS API commands (e.g., `/system/identity/print`, `/system/resource/print`) +- Results sent back via `MikrotikResult` protobuf message + ### MIB Name Resolution (Rust NIF) Uses Rust NIF via Rustler for fast SNMP MIB name resolution (replaces unreliable Erlang SNMP modules). @@ -508,12 +556,51 @@ Available at `/docs/api` (adapted from Tailwind UI Protocol template). ## Kubernetes Deployment **Prerequisites**: cert-manager, Traefik, MetalLB, FluxCD, GitLab Agent, NFS Provisioner + **Secrets**: All stored in 1Password, created manually in `towerops` namespace - `gitlab-registry` - Docker registry credentials -- `towerops-secrets` - RELEASE_COOKIE, SECRET_KEY_BASE +- `towerops-secrets` - RELEASE_COOKIE, SECRET_KEY_BASE, CLOAK_KEY - `towerops-db` - PostgreSQL connection details - `towerops-aws` - AWS credentials +### Creating/Updating Secrets + +**Initial setup** - Generate and store encryption key: +```bash +# Generate CLOAK_KEY (store in 1Password) +openssl rand -base64 32 + +# Create towerops-secrets with all required keys +kubectl create secret generic towerops-secrets \ + --from-literal=RELEASE_COOKIE=$(openssl rand -base64 32) \ + --from-literal=SECRET_KEY_BASE=$(mix phx.gen.secret) \ + --from-literal=CLOAK_KEY=$(openssl rand -base64 32) \ + -n towerops +``` + +**Add CLOAK_KEY to existing secret**: +```bash +# Generate new encryption key +CLOAK_KEY=$(openssl rand -base64 32) + +# Store in 1Password first, then add to Kubernetes +kubectl create secret generic towerops-secrets \ + --from-literal=CLOAK_KEY="$CLOAK_KEY" \ + --dry-run=client -o yaml | \ + kubectl apply -f - -n towerops + +# Or patch existing secret +kubectl patch secret towerops-secrets \ + -n towerops \ + --type='json' \ + -p="[{'op': 'add', 'path': '/data/CLOAK_KEY', 'value': '$(echo -n "$CLOAK_KEY" | base64)'}]" +``` + +**Important**: +- Always store the CLOAK_KEY in 1Password before deploying +- Losing the encryption key makes encrypted data (MikroTik passwords, etc.) unrecoverable +- After adding CLOAK_KEY, restart pods: `kubectl rollout restart deployment/towerops -n towerops` + **Deployment**: FluxCD monitors Git repository, automatically applies manifests in `k8s/` directory **Manual**: `kubectl apply -k k8s/` @@ -521,6 +608,7 @@ Available at `/docs/api` (adapted from Tailwind UI Protocol template). - FluxCD: `kubectl get kustomization -n flux-system` - Certificates: `kubectl describe certificate towerops-net-cert -n towerops` - Pods: `kubectl describe pod -n towerops -l app=towerops` +- Secrets: `kubectl describe secret towerops-secrets -n towerops` (shows keys but not values) ## Common Patterns diff --git a/config/dev.exs b/config/dev.exs index cfebd06e..d3114fb4 100644 --- a/config/dev.exs +++ b/config/dev.exs @@ -73,6 +73,17 @@ config :towerops, Towerops.Repo, show_sensitive_data_on_connection_error: true, pool_size: 10 +# Configure Cloak encryption for development +# Use a fixed key for development (DO NOT use in production!) +config :towerops, Towerops.Vault, + ciphers: [ + default: { + Cloak.Ciphers.AES.GCM, + # Development-only key - replace with env var in production + tag: "AES.GCM.V1", key: Base.decode64!("nQWmgQYhoCNXA3PAxwriKxLyPHAOWH9VgpLkBOXrowM=") + } + ] + # For development, we disable any cache and enable # debugging and code reloading. # diff --git a/config/runtime.exs b/config/runtime.exs index 8595b3f0..729b6147 100644 --- a/config/runtime.exs +++ b/config/runtime.exs @@ -44,6 +44,15 @@ if config_env() == :prod do You can generate one by calling: mix phx.gen.secret """ + # Cloak encryption key for MikroTik API passwords and other sensitive data + cloak_key = + System.get_env("CLOAK_KEY") || + raise """ + environment variable CLOAK_KEY is missing. + You can generate one by calling: openssl rand -base64 32 + Store the generated key in 1Password and set as environment variable. + """ + host = System.get_env("PHX_HOST") || "example.com" # Configure Redis/Valkey connection @@ -204,6 +213,14 @@ if config_env() == :prod do # pool_count: 4, socket_options: maybe_ipv6 + config :towerops, Towerops.Vault, + ciphers: [ + default: { + Cloak.Ciphers.AES.GCM, + tag: "AES.GCM.V1", key: Base.decode64!(cloak_key) + } + ] + config :towerops, ToweropsWeb.Endpoint, url: [host: host, port: 443, scheme: "https"], http: [ diff --git a/config/test.exs b/config/test.exs index cce5b486..f20c21e5 100644 --- a/config/test.exs +++ b/config/test.exs @@ -46,6 +46,17 @@ config :towerops, Towerops.Repo, pool: Ecto.Adapters.SQL.Sandbox, pool_size: System.schedulers_online() * 2 +# Configure Cloak encryption for testing +# Use a fixed test key (same as dev for simplicity) +config :towerops, Towerops.Vault, + ciphers: [ + default: { + Cloak.Ciphers.AES.GCM, + # Test key - same as dev for consistency + tag: "AES.GCM.V1", key: Base.decode64!("nQWmgQYhoCNXA3PAxwriKxLyPHAOWH9VgpLkBOXrowM=") + } + ] + # We don't run a server during test. If one is required, # you can enable the server option below. config :towerops, ToweropsWeb.Endpoint, diff --git a/lib/towerops/accounts/user.ex b/lib/towerops/accounts/user.ex index b55f68e7..193022b7 100644 --- a/lib/towerops/accounts/user.ex +++ b/lib/towerops/accounts/user.ex @@ -127,7 +127,15 @@ defmodule Towerops.Accounts.User do """ def registration_changeset(user, attrs, opts \\ []) do user - |> cast(attrs, [:email, :password, :first_name, :last_name, :timezone, :privacy_policy_consent, :terms_of_service_consent]) + |> cast(attrs, [ + :email, + :password, + :first_name, + :last_name, + :timezone, + :privacy_policy_consent, + :terms_of_service_consent + ]) |> validate_email_for_registration(opts) |> validate_password(opts) |> validate_consent() diff --git a/lib/towerops/application.ex b/lib/towerops/application.ex index c26333ad..aab77585 100644 --- a/lib/towerops/application.ex +++ b/lib/towerops/application.ex @@ -81,6 +81,8 @@ defmodule Towerops.Application do {Cluster.Supervisor, [topologies, [name: Towerops.ClusterSupervisor]]}, ToweropsWeb.Telemetry, Towerops.Repo, + # Encryption vault for sensitive data (passwords, API tokens) + Towerops.Vault, # Rate limiting backend (ETS-based, per-node) {Hammer.Backend.ETS, [expiry_ms: to_timeout(minute: 10), cleanup_interval_ms: to_timeout(minute: 1)]}, {Oban, Application.fetch_env!(:towerops, Oban)}, diff --git a/lib/towerops/devices.ex b/lib/towerops/devices.ex index 13234aa6..ecb053e7 100644 --- a/lib/towerops/devices.ex +++ b/lib/towerops/devices.ex @@ -5,6 +5,7 @@ defmodule Towerops.Devices do import Ecto.Query + alias Towerops.Agents alias Towerops.Devices.Device, as: DeviceSchema alias Towerops.Devices.Event alias Towerops.Organizations.SubscriptionLimits @@ -275,6 +276,176 @@ defmodule Towerops.Devices do end end + @doc """ + Gets MikroTik API configuration for a device with hierarchical fallback. + + Returns a map with username, password (decrypted), port, use_ssl, enabled, and source. + + ## Examples + + iex> get_mikrotik_config(device) + %{ + username: "admin", + password: "secret", + port: 8729, + use_ssl: true, + enabled: true, + source: :device + } + """ + def get_mikrotik_config(device_id) when is_binary(device_id) do + device = + DeviceSchema + |> Repo.get!(device_id) + |> Repo.preload(site: :organization) + + get_mikrotik_config(device) + end + + def get_mikrotik_config(%DeviceSchema{} = device) do + device = ensure_mikrotik_config_loaded(device) + resolve_mikrotik_config(device) + end + + defp ensure_mikrotik_config_loaded(device) do + # Ensure site and organization associations are loaded + needs_preload = + !Ecto.assoc_loaded?(device.site) || + !Ecto.assoc_loaded?(device.site.organization) + + if needs_preload do + Repo.preload(device, site: :organization) + else + device + end + end + + defp resolve_mikrotik_config(device) do + # Resolve each field independently with hierarchical fallback + username = + device.mikrotik_username || + device.site.mikrotik_username || + device.site.organization.mikrotik_username + + password = + device.mikrotik_password || + device.site.mikrotik_password || + device.site.organization.mikrotik_password + + port = + device.mikrotik_port || + device.site.mikrotik_port || + device.site.organization.mikrotik_port || + 8729 + + use_ssl = + if device.mikrotik_use_ssl == nil do + device.site.mikrotik_use_ssl || + device.site.organization.mikrotik_use_ssl || + true + else + device.mikrotik_use_ssl + end + + enabled = + device.mikrotik_enabled || + device.site.mikrotik_enabled || + device.site.organization.mikrotik_enabled || + false + + source = determine_mikrotik_source(device) + + %{ + username: username, + password: password, + port: port, + use_ssl: use_ssl, + enabled: enabled, + source: source + } + end + + defp determine_mikrotik_source(device) do + # Determine source based on where username came from (primary credential) + cond do + device.mikrotik_username != nil -> :device + device.site.mikrotik_username != nil -> :site + device.site.organization.mikrotik_username != nil -> :organization + true -> :default + end + end + + @doc """ + Propagates MikroTik credential changes from a site to all devices in that + site that inherit credentials from the site (credential_source = "site"). + + Updates username, password, port, use_ssl, and enabled fields. + """ + def propagate_site_mikrotik_change(site_id, attrs) do + # Find all devices in this site that inherit credentials from site + devices_to_update = + Repo.all( + from(d in DeviceSchema, + where: d.site_id == ^site_id, + where: d.mikrotik_credential_source == "site" + ) + ) + + # Update each device's MikroTik credentials + Enum.each(devices_to_update, fn device -> + device + |> Ecto.Changeset.change(attrs) + |> Repo.update() + + # Broadcast assignment change to trigger agent job list refresh + Phoenix.PubSub.broadcast( + Towerops.PubSub, + "device:#{device.id}:assignments", + {:assignments_changed, :mikrotik_updated} + ) + end) + + :ok + end + + @doc """ + Propagates MikroTik credential changes from an organization to all devices + in sites that have no site-level credentials (credential_source = "site" + and site.mikrotik_username is nil). + + Updates username, password, port, use_ssl, and enabled fields. + """ + def propagate_organization_mikrotik_change(organization_id, attrs) do + # Find all devices in this org's sites that: + # 1. Inherit from site (credential_source = "site") + # 2. Their site has no MikroTik username (so they fall back to org) + devices_to_update = + Repo.all( + from(d in DeviceSchema, + join: s in assoc(d, :site), + where: s.organization_id == ^organization_id, + where: d.mikrotik_credential_source == "site", + where: is_nil(s.mikrotik_username) or s.mikrotik_username == "" + ) + ) + + # Update each device's MikroTik credentials + Enum.each(devices_to_update, fn device -> + device + |> Ecto.Changeset.change(attrs) + |> Repo.update() + + # Broadcast assignment change to trigger agent job list refresh + Phoenix.PubSub.broadcast( + Towerops.PubSub, + "device:#{device.id}:assignments", + {:assignments_changed, :mikrotik_updated} + ) + end) + + :ok + end + @doc """ Creates device. @@ -352,9 +523,12 @@ defmodule Towerops.Devices do old_snmp_version = device.snmp_version old_snmp_port = device.snmp_port - case device - |> DeviceSchema.changeset(attrs) - |> Repo.update() do + changeset = DeviceSchema.changeset(device, attrs) + + # Validate MikroTik SSL requirement for cloud pollers + changeset = validate_mikrotik_ssl_with_cloud_poller(changeset, device) + + case Repo.update(changeset) do {:ok, updated_device} = result -> _ = handle_monitoring_changes(updated_device, old_monitoring) _ = handle_snmp_changes(updated_device, old_snmp, old_snmp_version, old_snmp_port) @@ -365,6 +539,44 @@ defmodule Towerops.Devices do end end + defp validate_mikrotik_ssl_with_cloud_poller(changeset, device) do + # Check if MikroTik is enabled and SSL is being disabled + mikrotik_enabled = Ecto.Changeset.get_field(changeset, :mikrotik_enabled) + use_ssl_change = Ecto.Changeset.get_change(changeset, :mikrotik_use_ssl) + + # Only validate if MikroTik is enabled and SSL is explicitly being set to false + if mikrotik_enabled && use_ssl_change == false do + validate_cloud_poller_requires_ssl(changeset, device) + else + changeset + end + end + + defp validate_cloud_poller_requires_ssl(changeset, device) do + # Get effective agent to check if it's a cloud poller + device_with_site = Repo.preload(device, site: :organization) + {effective_agent_id, _source} = Agents.get_effective_agent_token_with_source(device_with_site) + + using_cloud_poller = + if effective_agent_id do + agent = Agents.get_agent_token!(effective_agent_id) + agent.is_cloud_poller + else + # No agent means cloud polling by default + true + end + + if using_cloud_poller do + Ecto.Changeset.add_error( + changeset, + :mikrotik_use_ssl, + "SSL is required when using cloud pollers. Plain MikroTik API sends credentials unencrypted over the internet." + ) + else + changeset + end + end + @doc """ Resolves the effective SNMP community string for a device. diff --git a/lib/towerops/devices/device.ex b/lib/towerops/devices/device.ex index 2a9ec077..cb77d8d3 100644 --- a/lib/towerops/devices/device.ex +++ b/lib/towerops/devices/device.ex @@ -39,6 +39,14 @@ defmodule Towerops.Devices.Device do field :last_discovery_at, :utc_datetime field :last_snmp_poll_at, :utc_datetime + # MikroTik API credentials (device-level overrides) + field :mikrotik_username, :string + field :mikrotik_password, Towerops.Encrypted.Binary + field :mikrotik_port, :integer + field :mikrotik_use_ssl, :boolean + field :mikrotik_enabled, :boolean + field :mikrotik_credential_source, :string, default: "site" + belongs_to :site, Site has_one :snmp_device, SnmpDevice @@ -64,6 +72,12 @@ defmodule Towerops.Devices.Device do snmp_port: integer(), last_discovery_at: DateTime.t() | nil, last_snmp_poll_at: DateTime.t() | nil, + mikrotik_username: String.t() | nil, + mikrotik_password: String.t() | nil, + mikrotik_port: integer() | nil, + mikrotik_use_ssl: boolean() | nil, + mikrotik_enabled: boolean() | nil, + mikrotik_credential_source: String.t() | nil, site_id: Ecto.UUID.t(), site: NotLoaded.t() | Site.t(), snmp_device: NotLoaded.t() | SnmpDevice.t() | nil, @@ -92,7 +106,13 @@ defmodule Towerops.Devices.Device do :last_checked_at, :last_status_change_at, :last_discovery_at, - :last_snmp_poll_at + :last_snmp_poll_at, + :mikrotik_username, + :mikrotik_password, + :mikrotik_port, + :mikrotik_use_ssl, + :mikrotik_enabled, + :mikrotik_credential_source ]) |> validate_required([:ip_address, :site_id]) |> validate_name() @@ -101,7 +121,9 @@ defmodule Towerops.Devices.Device do |> validate_ip_address() |> validate_number(:check_interval_seconds, greater_than: 0, less_than_or_equal_to: 3600) |> validate_snmp() + |> validate_mikrotik() |> update_community_source() + |> update_mikrotik_credential_source() |> foreign_key_constraint(:site_id) end @@ -170,4 +192,49 @@ defmodule Towerops.Devices.Device do put_change(changeset, :snmp_community_source, "device") end end + + defp validate_mikrotik(changeset) do + mikrotik_enabled = get_field(changeset, :mikrotik_enabled) + + if mikrotik_enabled do + changeset + |> validate_number(:mikrotik_port, greater_than: 0, less_than: 65_536) + |> validate_mikrotik_ssl_for_cloud() + else + changeset + end + end + + defp validate_mikrotik_ssl_for_cloud(changeset) do + # If explicitly setting use_ssl to false at device level, ensure not using cloud poller + use_ssl = get_field(changeset, :mikrotik_use_ssl) + + # Only validate if explicitly disabling SSL (nil means inherit) + if use_ssl == false do + # Note: We can't check effective agent here during changeset validation + # as it requires database queries. This validation happens at the context level. + # Here we just ensure the field is set if provided. + changeset + else + changeset + end + end + + defp update_mikrotik_credential_source(changeset) do + # If user explicitly set a username, mark source as "device" + # If it's blank/nil, mark source as "site" (will inherit from site or org) + case get_change(changeset, :mikrotik_username) do + nil -> + # No change to username, keep existing source + changeset + + "" -> + # Explicitly cleared, set to inherit from site + put_change(changeset, :mikrotik_credential_source, "site") + + _value -> + # Explicitly set a value, mark as device-specific + put_change(changeset, :mikrotik_credential_source, "device") + end + end end diff --git a/lib/towerops/ecto_types/encrypted_binary.ex b/lib/towerops/ecto_types/encrypted_binary.ex new file mode 100644 index 00000000..4c1befba --- /dev/null +++ b/lib/towerops/ecto_types/encrypted_binary.ex @@ -0,0 +1,36 @@ +defmodule Towerops.Encrypted.Binary do + @moduledoc """ + Encrypted binary field using Cloak vault. + + Used for sensitive credentials like MikroTik API passwords. + Data is encrypted at rest using AES-256-GCM. + + ## Usage + + In your schema: + + field :password, Towerops.Encrypted.Binary + + The field will be automatically encrypted when writing to the database + and decrypted when reading from the database. + + ## Database Schema + + The database column should be of type `:binary`: + + add :password, :binary + + ## Examples + + # In a changeset + cast(attrs, [:password]) + # Password is automatically encrypted before database write + + # Reading from database + user.password # Automatically decrypted + + # Nil handling + nil values are preserved (not encrypted) + """ + use Cloak.Ecto.Binary, vault: Towerops.Vault +end diff --git a/lib/towerops/organizations/organization.ex b/lib/towerops/organizations/organization.ex index 66576404..4958f58a 100644 --- a/lib/towerops/organizations/organization.ex +++ b/lib/towerops/organizations/organization.ex @@ -29,6 +29,13 @@ defmodule Towerops.Organizations.Organization do field :snmp_version, :string, default: "2c" field :snmp_community, :string + # MikroTik API credentials (organization-level defaults cascade to site → device) + field :mikrotik_username, :string + field :mikrotik_password, Towerops.Encrypted.Binary + field :mikrotik_port, :integer, default: 8729 + field :mikrotik_use_ssl, :boolean, default: true + field :mikrotik_enabled, :boolean, default: false + belongs_to :default_agent_token, AgentToken has_many :memberships, Membership @@ -45,6 +52,11 @@ defmodule Towerops.Organizations.Organization do subscription_plan: String.t(), snmp_version: String.t() | nil, snmp_community: String.t() | nil, + mikrotik_username: String.t() | nil, + mikrotik_password: String.t() | nil, + mikrotik_port: integer() | nil, + mikrotik_use_ssl: boolean() | nil, + mikrotik_enabled: boolean() | nil, default_agent_token_id: Ecto.UUID.t() | nil, default_agent_token: NotLoaded.t() | AgentToken.t() | nil, memberships: NotLoaded.t() | [Membership.t()], @@ -62,18 +74,38 @@ defmodule Towerops.Organizations.Organization do :subscription_plan, :default_agent_token_id, :snmp_version, - :snmp_community + :snmp_community, + :mikrotik_username, + :mikrotik_password, + :mikrotik_port, + :mikrotik_use_ssl, + :mikrotik_enabled ]) |> validate_required([:name]) |> validate_length(:name, min: 2, max: 100) |> validate_inclusion(:subscription_plan, ["free"]) |> validate_inclusion(:snmp_version, ["1", "2c", "3"], message: "must be 1, 2c, or 3") + |> validate_mikrotik_fields() |> generate_slug() |> validate_required([:slug]) |> unique_constraint(:slug) |> foreign_key_constraint(:default_agent_token_id) end + defp validate_mikrotik_fields(changeset) do + changeset + |> validate_number(:mikrotik_port, greater_than: 0, less_than: 65_536) + |> validate_mikrotik_required_fields() + end + + defp validate_mikrotik_required_fields(changeset) do + if get_field(changeset, :mikrotik_enabled) do + validate_required(changeset, [:mikrotik_username]) + else + changeset + end + end + defp generate_slug(changeset) do # Only generate slug if not already set if get_field(changeset, :slug) do diff --git a/lib/towerops/proto/agent.pb.ex b/lib/towerops/proto/agent.pb.ex index 4f4fb4d7..b45a3545 100644 --- a/lib/towerops/proto/agent.pb.ex +++ b/lib/towerops/proto/agent.pb.ex @@ -230,6 +230,7 @@ defmodule Towerops.Agent.JobType do field :DISCOVER, 0 field :POLL, 1 + field :MIKROTIK, 2 end defmodule Towerops.Agent.QueryType do @@ -258,6 +259,8 @@ defmodule Towerops.Agent.AgentJob do field :device_id, 3, type: :string, json_name: "deviceId" field :snmp_device, 4, type: Towerops.Agent.SnmpDevice, json_name: "snmpDevice" field :queries, 5, repeated: true, type: Towerops.Agent.SnmpQuery + field :mikrotik_device, 6, type: Towerops.Agent.MikrotikDevice, json_name: "mikrotikDevice" + field :mikrotik_commands, 7, repeated: true, type: Towerops.Agent.MikrotikCommand, json_name: "mikrotikCommands" end defmodule Towerops.Agent.AgentJobList do @@ -351,3 +354,82 @@ defmodule Towerops.Agent.AgentError do field :message, 3, type: :string field :timestamp, 4, type: :int64 end + +defmodule Towerops.Agent.MikrotikDevice do + @moduledoc false + + use Protobuf, + full_name: "towerops.agent.MikrotikDevice", + protoc_gen_elixir_version: "0.16.0", + syntax: :proto3 + + field :ip, 1, type: :string + field :port, 2, type: :uint32 + field :username, 3, type: :string + field :password, 4, type: :string + field :use_ssl, 5, type: :bool, json_name: "useSsl" +end + +defmodule Towerops.Agent.MikrotikCommand.ArgsEntry do + @moduledoc false + + use Protobuf, + full_name: "towerops.agent.MikrotikCommand.ArgsEntry", + map: true, + protoc_gen_elixir_version: "0.16.0", + syntax: :proto3 + + field :key, 1, type: :string + field :value, 2, type: :string +end + +defmodule Towerops.Agent.MikrotikCommand do + @moduledoc false + + use Protobuf, + full_name: "towerops.agent.MikrotikCommand", + protoc_gen_elixir_version: "0.16.0", + syntax: :proto3 + + field :command, 1, type: :string + field :args, 2, repeated: true, type: Towerops.Agent.MikrotikCommand.ArgsEntry, map: true +end + +defmodule Towerops.Agent.MikrotikResult do + @moduledoc false + + use Protobuf, + full_name: "towerops.agent.MikrotikResult", + protoc_gen_elixir_version: "0.16.0", + syntax: :proto3 + + field :device_id, 1, type: :string, json_name: "deviceId" + field :job_id, 2, type: :string, json_name: "jobId" + field :sentences, 3, repeated: true, type: Towerops.Agent.MikrotikSentence + field :error, 4, type: :string + field :timestamp, 5, type: :int64 +end + +defmodule Towerops.Agent.MikrotikSentence.AttributesEntry do + @moduledoc false + + use Protobuf, + full_name: "towerops.agent.MikrotikSentence.AttributesEntry", + map: true, + protoc_gen_elixir_version: "0.16.0", + syntax: :proto3 + + field :key, 1, type: :string + field :value, 2, type: :string +end + +defmodule Towerops.Agent.MikrotikSentence do + @moduledoc false + + use Protobuf, + full_name: "towerops.agent.MikrotikSentence", + protoc_gen_elixir_version: "0.16.0", + syntax: :proto3 + + field :attributes, 1, repeated: true, type: Towerops.Agent.MikrotikSentence.AttributesEntry, map: true +end diff --git a/lib/towerops/sites/site.ex b/lib/towerops/sites/site.ex index f3552330..e8e017fc 100644 --- a/lib/towerops/sites/site.ex +++ b/lib/towerops/sites/site.ex @@ -26,6 +26,13 @@ defmodule Towerops.Sites.Site do field :snmp_version, :string field :snmp_community, :string + # MikroTik API credentials (overrides organization default) + field :mikrotik_username, :string + field :mikrotik_password, Towerops.Encrypted.Binary + field :mikrotik_port, :integer + field :mikrotik_use_ssl, :boolean + field :mikrotik_enabled, :boolean + belongs_to :organization, Organization belongs_to :agent_token, AgentToken belongs_to :parent_site, Site @@ -43,6 +50,11 @@ defmodule Towerops.Sites.Site do display_order: integer() | nil, snmp_version: String.t() | nil, snmp_community: String.t() | nil, + mikrotik_username: String.t() | nil, + mikrotik_password: String.t() | nil, + mikrotik_port: integer() | nil, + mikrotik_use_ssl: boolean() | nil, + mikrotik_enabled: boolean() | nil, organization_id: Ecto.UUID.t(), organization: NotLoaded.t() | Organization.t(), agent_token_id: Ecto.UUID.t() | nil, @@ -67,13 +79,19 @@ defmodule Towerops.Sites.Site do :agent_token_id, :parent_site_id, :snmp_version, - :snmp_community + :snmp_community, + :mikrotik_username, + :mikrotik_password, + :mikrotik_port, + :mikrotik_use_ssl, + :mikrotik_enabled ]) |> validate_required([:name, :organization_id]) |> validate_length(:name, min: 2, max: 200) |> validate_length(:description, max: 1000) |> validate_length(:location, max: 200) |> validate_inclusion(:snmp_version, ["1", "2c", "3"], message: "must be 1, 2c, or 3") + |> validate_mikrotik_fields() |> foreign_key_constraint(:organization_id) |> foreign_key_constraint(:agent_token_id) |> foreign_key_constraint(:parent_site_id) @@ -84,6 +102,20 @@ defmodule Towerops.Sites.Site do |> validate_not_circular_parent() end + defp validate_mikrotik_fields(changeset) do + changeset + |> validate_number(:mikrotik_port, greater_than: 0, less_than: 65_536) + |> validate_mikrotik_required_fields() + end + + defp validate_mikrotik_required_fields(changeset) do + if get_field(changeset, :mikrotik_enabled) do + validate_required(changeset, [:mikrotik_username]) + else + changeset + end + end + defp validate_not_circular_parent(changeset) do parent_site_id = get_change(changeset, :parent_site_id) site_id = get_field(changeset, :id) diff --git a/lib/towerops/vault.ex b/lib/towerops/vault.ex new file mode 100644 index 00000000..e47d9086 --- /dev/null +++ b/lib/towerops/vault.ex @@ -0,0 +1,58 @@ +defmodule Towerops.Vault do + @moduledoc """ + Encryption vault for sensitive data using Cloak. + + Encrypts data using AES-256-GCM encryption. The encryption key is loaded + from the CLOAK_KEY environment variable (base64-encoded). + + ## Usage + + Fields that use this vault are automatically encrypted/decrypted by Ecto. + See `Towerops.Encrypted.Binary` for the custom Ecto type. + + ## Key Generation + + Generate a new encryption key: + + openssl rand -base64 32 + + Store this key securely in 1Password and set it as the CLOAK_KEY + environment variable in production. + """ + use Cloak.Vault, otp_app: :towerops + + @impl GenServer + def init(config) do + # In dev/test, the key is already configured in config files + # In production (runtime.exs), we override with env var + config = + if System.get_env("CLOAK_KEY") do + Keyword.put(config, :ciphers, + default: { + Cloak.Ciphers.AES.GCM, + tag: "AES.GCM.V1", key: decode_env!("CLOAK_KEY") + } + ) + else + # Use the key from config (dev/test) + config + end + + {:ok, config} + end + + defp decode_env!(var) do + var + |> System.get_env() + |> case do + nil -> + raise """ + Environment variable #{var} is missing. + Generate a key with: openssl rand -base64 32 + """ + + value -> + Base.decode64!(value) + end + end +end diff --git a/lib/towerops_web/channels/agent_channel.ex b/lib/towerops_web/channels/agent_channel.ex index 5771ffe2..b8cf2828 100644 --- a/lib/towerops_web/channels/agent_channel.ex +++ b/lib/towerops_web/channels/agent_channel.ex @@ -25,6 +25,8 @@ defmodule ToweropsWeb.AgentChannel do alias Towerops.Agent.AgentHeartbeat alias Towerops.Agent.AgentJob alias Towerops.Agent.AgentJobList + alias Towerops.Agent.MikrotikCommand + alias Towerops.Agent.MikrotikDevice alias Towerops.Agent.SnmpDevice alias Towerops.Agent.SnmpQuery alias Towerops.Agent.SnmpResult @@ -191,15 +193,26 @@ defmodule ToweropsWeb.AgentChannel do defp build_jobs_for_agent(agent_token_id) do agent_token_id |> Agents.list_agent_polling_targets() - |> Enum.map(&build_job_for_device/1) + |> Enum.flat_map(&build_jobs_for_device/1) end - defp build_job_for_device(device) do - if needs_discovery?(device) do - build_discovery_job(device) - else - build_polling_job(device) - end + defp build_jobs_for_device(device) do + # Build SNMP job (discovery or polling) + snmp_job = + if needs_discovery?(device) do + build_discovery_job(device) + else + build_polling_job(device) + end + + # Build MikroTik job if device is MikroTik and MikroTik API is enabled + mikrotik_job = + if mikrotik_device?(device) do + build_mikrotik_job(device) + end + + # Return list of jobs (filter out nil) + Enum.reject([snmp_job, mikrotik_job], &is_nil/1) end defp needs_discovery?(device) do @@ -209,6 +222,13 @@ defmodule ToweropsWeb.AgentChannel do DateTime.diff(DateTime.utc_now(), device.last_discovery_at, :hour) > 24 end + defp mikrotik_device?(device) do + # Check if device is MikroTik based on SNMP discovery data + device.snmp_device && + (String.contains?(device.snmp_device.manufacturer || "", "MikroTik") || + String.contains?(device.snmp_device.sys_descr || "", "RouterOS")) + end + defp build_discovery_job(device) do %AgentJob{ job_id: "discover:#{device.id}", @@ -355,6 +375,42 @@ defmodule ToweropsWeb.AgentChannel do ] end + defp build_mikrotik_job(device) do + mikrotik_config = Devices.get_mikrotik_config(device) + + # Only build job if MikroTik API is enabled and has credentials + if mikrotik_config.enabled && mikrotik_config.username do + %AgentJob{ + job_id: "mikrotik:#{device.id}", + job_type: :MIKROTIK, + device_id: device.id, + mikrotik_device: %MikrotikDevice{ + ip: device.ip_address, + username: mikrotik_config.username, + password: mikrotik_config.password, + port: mikrotik_config.port, + use_ssl: mikrotik_config.use_ssl + }, + mikrotik_commands: build_mikrotik_commands() + } + end + end + + defp build_mikrotik_commands do + [ + # System identity + %MikrotikCommand{ + command: "/system/identity/print", + args: %{} + }, + # System resources + %MikrotikCommand{ + command: "/system/resource/print", + args: %{} + } + ] + end + defp process_snmp_result(organization_id, result, socket) do with {:ok, device} <- fetch_device(result.device_id), :ok <- verify_device_organization(device, organization_id) do diff --git a/lib/towerops_web/live/device_live/form.ex b/lib/towerops_web/live/device_live/form.ex index 1f35c86a..0f3d0f5c 100644 --- a/lib/towerops_web/live/device_live/form.ex +++ b/lib/towerops_web/live/device_live/form.ex @@ -152,6 +152,13 @@ defmodule ToweropsWeb.DeviceLive.Form do # Get effective SNMP configuration and source snmp_config = Devices.get_snmp_config(device) + # Get effective MikroTik configuration and source + mikrotik_config = Devices.get_mikrotik_config(device) + + # Check if device is MikroTik (based on SNMP discovery) + device_with_snmp = Towerops.Repo.preload(device, :snmp_device) + is_mikrotik = mikrotik_device?(device_with_snmp) + # Determine monitoring mode based on current snmp_enabled value monitoring_mode = if device.snmp_enabled, do: "snmp_and_icmp", else: "icmp_only" @@ -164,9 +171,20 @@ defmodule ToweropsWeb.DeviceLive.Form do |> assign(:snmp_config_source, snmp_config.source) |> assign(:effective_snmp_version, snmp_config.version) |> assign(:effective_snmp_community, snmp_config.community || "(not set)") + |> assign(:mikrotik_config_source, mikrotik_config.source) + |> assign(:effective_mikrotik_username, mikrotik_config.username || "(not set)") + |> assign(:is_mikrotik_device, is_mikrotik) + |> assign(:mikrotik_test_result, nil) |> assign(:monitoring_mode, monitoring_mode) end + defp mikrotik_device?(device) do + # Check if device is MikroTik based on SNMP discovery data + device.snmp_device && + (String.contains?(device.snmp_device.manufacturer || "", "MikroTik") || + String.contains?(device.snmp_device.sys_descr || "", "RouterOS")) + end + @impl true def handle_event("switch_monitoring_mode", %{"mode" => mode}, socket) do # Update snmp_enabled based on the selected mode diff --git a/lib/towerops_web/live/device_live/form.html.heex b/lib/towerops_web/live/device_live/form.html.heex index 0fb2508b..83a9b2f1 100644 --- a/lib/towerops_web/live/device_live/form.html.heex +++ b/lib/towerops_web/live/device_live/form.html.heex @@ -357,6 +357,80 @@ + <%= if @live_action == :edit and @is_mikrotik_device and @current_scope.user.is_superuser do %> +
+ Configure MikroTik RouterOS API access. Works alongside SNMP for enhanced device management. +
++ Inheriting credentials from {@mikrotik_config_source}. + Username: {@effective_mikrotik_username} +
++ 🚨 Critical Security Warning: + Plain API (port 8728) sends credentials unencrypted over the network. + This setting is blocked when using cloud pollers + and should only be used with local agents on trusted networks. +
++ ⚠️ Security Warning: + Plain API (port 8728) sends credentials unencrypted. Use SSL (port 8729) whenever possible. +
++ Manage organization defaults for SNMP, agents, and MikroTik API configuration +
- Set default SNMP settings for all devices in this organization. These can be overridden at the site or device level. -
- - <.input - field={@form[:snmp_version]} - type="select" - label="SNMP Version" - prompt="Select SNMP version" - options={[{"SNMP v1", "1"}, {"SNMP v2c", "2c"}, {"SNMP v3", "3"}]} - /> - - <.input - field={@form[:snmp_community]} - type="text" - label="SNMP Community String" - placeholder="e.g., public" - /> - -- <.icon name="hero-information-circle" class="h-4 w-4 inline" /> - SNMP configuration hierarchy: Device > Site > Organization -
- - <%= if @organization.snmp_community do %> -- Force Apply to All Device -
-- This will override SNMP settings for ALL devices across all sites in this organization. -
- <.button - type="button" - phx-click="apply_snmp_to_all" - data-confirm="This will replace SNMP settings for ALL devices in this organization. Are you sure?" - variant="danger" - > - <.icon name="hero-arrow-path" class="h-4 w-4" /> Apply SNMP Config to All Device - -- Select a default agent for SNMP polling. This will be used for all devices in this organization unless overridden at the site or device level. + <.form for={@form} id="organization-form" phx-change="validate" phx-submit="save"> +
+ Update the name of your organization.
++ Set default SNMP settings for all devices in this organization. These can be overridden at the site or device level. +
++ <.icon name="hero-information-circle" class="h-4 w-4 inline" /> + Hierarchy: Device > Site > Organization +
+- Current Device Assignment: -
-- <.icon name="hero-information-circle" class="h-4 w-4 inline" /> - Agent assignment hierarchy: Device > Site > Organization -
- - <%= if @organization.default_agent_token_id do %> -- Force Apply to All Device -
+ <%= if @organization.snmp_community do %> +- This will assign the default agent to ALL devices across all sites in this organization. + This will override SNMP settings for ALL devices across all sites in this organization.
- <.button +- No agents configured yet. - <.link - navigate={~p"/agents"} - class="text-blue-600 hover:text-blue-700 dark:text-blue-400 dark:hover:text-blue-300" - > - Create an agent - - to enable remote polling. -
++ Set default MikroTik RouterOS API credentials for all devices in this organization. Only applies to devices detected as MikroTik. +
++ <.icon name="hero-information-circle" class="h-4 w-4 inline" /> + Hierarchy: Device > Site > Organization +
++ Critical Security Warning: + Plain API (port 8728) sends credentials unencrypted over the network. This setting is + blocked for devices using cloud pollers + and should only be used with local agents on trusted networks. +
++ Security Warning: + Plain API (port 8728) sends credentials unencrypted. Use SSL (port 8729) whenever possible. +
++ Select a default agent for SNMP polling. This will be used for all devices in this organization unless overridden at the site or device level. +
++ <.icon name="hero-information-circle" class="h-4 w-4 inline" /> + Hierarchy: Device > Site > Organization +
++ Current Device Assignment: +
++ This will assign the default agent to ALL devices across all sites in this organization. +
+ ++ No agents configured yet. + <.link + navigate={~p"/agents"} + class="text-indigo-600 hover:text-indigo-700 dark:text-indigo-400 dark:hover:text-indigo-300" + > + Create an agent + + to enable remote polling. +
++ Override organization MikroTik API defaults for all devices at this site. Leave blank to inherit from organization. Only applies to MikroTik devices. +
+ + <.input + field={@form[:mikrotik_enabled]} + type="checkbox" + label="Enable MikroTik API" + /> + + <%= if @form[:mikrotik_enabled].value do %> + <.input + field={@form[:mikrotik_username]} + type="text" + label="Username" + placeholder="Leave blank to inherit from organization" + /> + + <.input + field={@form[:mikrotik_password]} + type="password" + label="Password" + placeholder="Leave blank to inherit from organization" + /> + + <.input + field={@form[:mikrotik_port]} + type="number" + label="API Port" + placeholder="8729 (SSL) or 8728 (plain)" + /> + + <.input + field={@form[:mikrotik_use_ssl]} + type="checkbox" + label="Use SSL (API-SSL)" + /> + + <%= if @form[:mikrotik_use_ssl].value == false do %> ++ 🚨 Critical Security Warning: + Plain API (port 8728) sends credentials unencrypted over the network. This setting is + blocked for devices using cloud pollers + and should only be used with local agents on trusted networks. +
++ ⚠️ Security Warning: + Plain API (port 8728) sends credentials unencrypted. Use SSL (port 8729) whenever possible. +
+