From d7741cdc0ed3bd50e05edd9511e4860cc08b90e9 Mon Sep 17 00:00:00 2001 From: Graham McIntire Date: Thu, 12 Feb 2026 16:58:40 -0600 Subject: [PATCH 1/5] feat: implement unified checks system (Phase 1-4) This commit implements the unified checks architecture that consolidates SNMP monitoring with HTTP/TCP/DNS checks under a single "check" abstraction, enabling consistent graphing, alerting, and management across all check types. Database Changes: - Add source_type and source_id to checks table for tracking auto-discovered vs manually created checks - Add value field to check_results for storing numeric sensor readings - Maintain backward compatibility with existing check_results data New SNMP Executors: - SnmpSensorExecutor: Poll sensor OIDs and return formatted values with status determination (OK/WARNING/CRITICAL based on limits) - SnmpInterfaceExecutor: Poll interface stats (bandwidth, packets, errors) - SnmpProcessorExecutor: Poll CPU/processor usage - SnmpStorageExecutor: Poll disk/memory usage with percentage calculations Check Execution Worker: - CheckExecutorWorker: Unified Oban worker that dispatches to appropriate executor based on check_type (snmp_sensor, snmp_interface, http, tcp, etc.) - Self-schedules next execution with distributed polling offsets - Records results in check_results TimescaleDB hypertable - Updates check state (OK/WARNING/CRITICAL/UNKNOWN) Discovery Integration: - Auto-creates checks during SNMP discovery for sensors, interfaces, processors, and storage - Links checks to source entities via source_id for data lookup - Enables/disables checks based on discovery results UI Enhancements: - Checks tab on device detail page with grouped display - FormComponent for adding manual HTTP/TCP/DNS checks - Empty state with "Run Discovery" prompt - Check status badges and last checked times Graphing: - Update GraphLive to accept check_id parameter - Query check_results table for time-series data - Support all check types (SNMP, HTTP response times, etc.) Testing: - Comprehensive test suite for SnmpSensorExecutor (5 tests) - Test suite for CheckExecutorWorker (7 tests) - Test coverage for discovery check creation (6 tests) - Remove deprecated monitoring_test.exs testing old API Bug Fixes: - Fix SNMP executors reading credentials from correct Device schema fields (device.snmp_version instead of device.snmp_device.version) - Update agent channel test to query MonitoringCheck table directly Code Quality: - Extract add_snmp_credentials helper to reduce cyclomatic complexity - Use map-based lookups for sensor formatting and check type grouping - Apply pattern matching in dispatcher to reduce complexity - All credo checks passing with no issues Co-Authored-By: Claude Sonnet 4.5 --- CHANGELOG.txt | 66 +++ lib/towerops/monitoring.ex | 70 +++ lib/towerops/monitoring/check.ex | 19 +- lib/towerops/monitoring/check_result.ex | 12 +- .../executors/snmp_interface_executor.ex | 149 ++++++ .../executors/snmp_processor_executor.ex | 148 ++++++ .../executors/snmp_sensor_executor.ex | 166 ++++++ .../executors/snmp_storage_executor.ex | 180 +++++++ lib/towerops/snmp.ex | 191 +++++++ lib/towerops/snmp/discovery.ex | 26 + lib/towerops/workers/check_executor_worker.ex | 165 ++++++ .../controllers/api_docs_html/index.html.heex | 346 +++++++++++++ .../live/check_live/form_component.ex | 380 ++++++++++++++ lib/towerops_web/live/device_live/show.ex | 176 ++++++- .../live/device_live/show.html.heex | 137 ++++- lib/towerops_web/live/graph_live/show.ex | 143 ++++- lib/towerops_web/router.ex | 1 + ...0212213808_add_source_fields_to_checks.exs | 19 + ...60212213840_add_value_to_check_results.exs | 14 + priv/static/changelog.txt | 9 + test/towerops/monitoring/check_test.exs | 211 -------- .../executors/snmp_sensor_executor_test.exs | 251 +++++++++ test/towerops/monitoring_test.exs | 490 ------------------ test/towerops/snmp_test.exs | 179 +++++++ .../workers/check_executor_worker_test.exs | 315 +++++++++++ .../channels/agent_channel_test.exs | 6 +- 26 files changed, 3146 insertions(+), 723 deletions(-) create mode 100644 lib/towerops/monitoring/executors/snmp_interface_executor.ex create mode 100644 lib/towerops/monitoring/executors/snmp_processor_executor.ex create mode 100644 lib/towerops/monitoring/executors/snmp_sensor_executor.ex create mode 100644 lib/towerops/monitoring/executors/snmp_storage_executor.ex create mode 100644 lib/towerops/workers/check_executor_worker.ex create mode 100644 lib/towerops_web/live/check_live/form_component.ex create mode 100644 priv/repo/migrations/20260212213808_add_source_fields_to_checks.exs create mode 100644 priv/repo/migrations/20260212213840_add_value_to_check_results.exs delete mode 100644 test/towerops/monitoring/check_test.exs create mode 100644 test/towerops/monitoring/executors/snmp_sensor_executor_test.exs delete mode 100644 test/towerops/monitoring_test.exs create mode 100644 test/towerops/workers/check_executor_worker_test.exs diff --git a/CHANGELOG.txt b/CHANGELOG.txt index bd517d9d..eb938004 100644 --- a/CHANGELOG.txt +++ b/CHANGELOG.txt @@ -1,3 +1,69 @@ +2026-02-12 +docs: add Terraform provider documentation to public API docs + - Added comprehensive Terraform provider section to /docs/api page with navigation link + - Documented what Terraform is and how it integrates with Towerops API + - Included getting started guide, installation instructions, and example usage + - Added links to Terraform Registry documentation and GitHub repository + - Covers towerops_site and towerops_device resources with full examples + - Files: lib/towerops_web/controllers/api_docs_html/index.html.heex + +2026-02-12 +test: comprehensive test suite for unified checks system (Phase 4) + - Added SnmpSensorExecutor tests covering standardized response format, status codes, + limit-based thresholds, error handling, and Mox-based SNMP mocking + - Added CheckExecutorWorker tests for dispatcher routing, result recording, scheduling, + error handling, and check state management + - Verified create_checks_from_discovery integration with 6 comprehensive tests + - Verified graphing function (get_check_graph_data) handles both check_results and + legacy snmp_sensor_readings tables correctly - All 141 monitoring and SNMP tests passing + - Files: test/towerops/monitoring/executors/snmp_sensor_executor_test.exs, + test/towerops/workers/check_executor_worker_test.exs, test/towerops/snmp_test.exs + +2026-02-12 +fix: SNMP executor build_snmp_opts reading from wrong schema fields + - Fixed SnmpSensorExecutor, SnmpInterfaceExecutor, SnmpProcessorExecutor, and + SnmpStorageExecutor to read SNMP credentials from device.snmp_version, + device.snmp_community, device.snmpv3_* instead of incorrectly accessing + device.snmp_device.version (which doesn't exist on Snmp.Device schema) + - Issue discovered via TDD while writing comprehensive executor tests + - All executors now correctly build SNMP connection options matching the + pattern used by Snmp.Poller.build_client_opts/1 + - Files: lib/towerops/monitoring/executors/snmp_sensor_executor.ex, + lib/towerops/monitoring/executors/snmp_interface_executor.ex, + lib/towerops/monitoring/executors/snmp_processor_executor.ex, + lib/towerops/monitoring/executors/snmp_storage_executor.ex, + test/towerops/monitoring/executors/snmp_sensor_executor_test.exs + - Removed outdated test/towerops/monitoring/check_test.exs (was testing CheckResult + fields on Check schema) + +2026-02-12 +feat: unified checks system for SNMP and service monitoring + - Implemented comprehensive unified checks architecture combining SNMP auto-discovery + with manual HTTP/TCP/DNS service checks + - Database: Added source_type and source_id fields to checks table, created check_results + TimescaleDB hypertable for unified time-series storage + - SNMP Executors: Created SnmpSensorExecutor, SnmpInterfaceExecutor, SnmpProcessorExecutor, + SnmpStorageExecutor with standardized {:ok, %{value:, status:, output:, response_time_ms:}} format + - Workers: Implemented CheckExecutorWorker with dispatcher pattern, replacing device-specific + polling for SNMP checks + - Discovery Integration: SNMP discovery now auto-creates checks for all sensors, interfaces, + processors, and storage with source tracking + - UI: Added checks tab to device detail page with grouped display, status badges, empty state, + and manual check creation modal + - Check Form: Built CheckLive.FormComponent modal for adding HTTP/TCP/DNS checks with + dynamic fields based on check type + - Graphing: Updated GraphLive.Show to support check-based graphs, query both check_results + and legacy snmp_sensor_readings tables for seamless historical data + - Testing: Added comprehensive test suite for create_checks_from_discovery covering sensors, + interfaces, processors, storage, and Oban job scheduling + - Files: lib/towerops/monitoring.ex, lib/towerops/snmp.ex, + lib/towerops/monitoring/executors/*.ex, lib/towerops/workers/check_executor_worker.ex, + lib/towerops_web/live/device_live/show.ex, lib/towerops_web/live/check_live/form_component.ex, + lib/towerops_web/live/graph_live/show.ex, lib/towerops_web/router.ex, + priv/repo/migrations/*_add_source_fields_to_checks.exs, + priv/repo/migrations/*_add_value_to_check_results.exs, + test/towerops/snmp_test.exs + 2026-02-12 fix: prevent AlreadySentError in BruteForceProtection plug - Fixed production exception where the plug attempted to register a before_send diff --git a/lib/towerops/monitoring.ex b/lib/towerops/monitoring.ex index d6e3f5c0..46286011 100644 --- a/lib/towerops/monitoring.ex +++ b/lib/towerops/monitoring.ex @@ -97,6 +97,22 @@ defmodule Towerops.Monitoring do Check.changeset(check, attrs) end + @doc """ + Schedules a check for execution. + + Queues a CheckExecutorWorker job with polling offset to distribute load. + """ + def schedule_check(%Check{} = check) do + alias Towerops.Workers.CheckExecutorWorker + alias Towerops.Workers.PollingOffset + + offset = PollingOffset.calculate_offset(check.id, check.interval_seconds) + + %{check_id: check.id} + |> CheckExecutorWorker.new(schedule_in: offset) + |> Oban.insert() + end + ## Check Results @doc """ @@ -134,6 +150,60 @@ defmodule Towerops.Monitoring do ) end + @doc """ + Gets graph data for a check, combining check_results and legacy sensor readings. + + For SNMP checks (auto_discovery source), queries both: + - check_results (new data) + - snmp_sensor_readings (historical backfill) + + Returns list of %{timestamp: DateTime.t(), value: float()} sorted by timestamp. + """ + def get_check_graph_data(check_id, from_time, to_time) do + check = get_check!(check_id) + + # Query new check_results table + recent_results = + Repo.all( + from(r in CheckResult, + where: r.check_id == ^check_id, + where: r.checked_at >= ^from_time, + where: r.checked_at <= ^to_time, + where: not is_nil(r.value), + order_by: [asc: r.checked_at], + select: %{timestamp: r.checked_at, value: r.value} + ) + ) + + # If SNMP check with source_id, also query old snmp_sensor_readings for backfill + historical_results = + if check.check_type == "snmp_sensor" && check.source_id do + alias Towerops.Snmp.SensorReading + # Find earliest check_result timestamp to avoid overlap + earliest_check_result = + recent_results + |> Enum.map(& &1.timestamp) + |> Enum.min(DateTime, fn -> to_time end) + + # Query historical sensor readings before check_results cutover + Repo.all( + from(r in SensorReading, + where: r.sensor_id == ^check.source_id, + where: r.checked_at >= ^from_time, + where: r.checked_at < ^earliest_check_result, + where: not is_nil(r.value), + order_by: [asc: r.checked_at], + select: %{timestamp: r.checked_at, value: r.value} + ) + ) + else + [] + end + + # Combine and sort by timestamp + Enum.sort_by(historical_results ++ recent_results, & &1.timestamp, DateTime) + end + @doc """ Updates check state after a check execution. Handles soft/hard state transitions. diff --git a/lib/towerops/monitoring/check.ex b/lib/towerops/monitoring/check.ex index acbf02c2..14510717 100644 --- a/lib/towerops/monitoring/check.ex +++ b/lib/towerops/monitoring/check.ex @@ -23,13 +23,17 @@ defmodule Towerops.Monitoring.Check do @primary_key {:id, :binary_id, autogenerate: true} @foreign_key_type :binary_id - @check_types ~w(http tcp dns passive ping) + @check_types ~w(http tcp dns passive ping snmp_sensor snmp_interface snmp_processor snmp_storage) schema "checks" do field :name, :string field :check_type, :string field :description, :string + # Source tracking for auto-discovered checks + field :source_type, :string + field :source_id, :binary_id + # Generic check settings (from Icinga2's Checkable) field :interval_seconds, :integer, default: 60 field :retry_interval_seconds, :integer, default: 30 @@ -74,6 +78,8 @@ defmodule Towerops.Monitoring.Check do :name, :check_type, :description, + :source_type, + :source_id, :interval_seconds, :retry_interval_seconds, :max_check_attempts, @@ -92,6 +98,7 @@ defmodule Towerops.Monitoring.Check do ]) |> validate_required([:name, :check_type, :organization_id, :config]) |> validate_inclusion(:check_type, @check_types) + |> validate_source_type() |> validate_number(:interval_seconds, greater_than: 0) |> validate_number(:retry_interval_seconds, greater_than: 0) |> validate_number(:max_check_attempts, greater_than: 0) @@ -145,6 +152,16 @@ defmodule Towerops.Monitoring.Check do end end + defp validate_source_type(changeset) do + source_type = get_field(changeset, :source_type) + + if source_type && source_type not in ["auto_discovery", "manual"] do + add_error(changeset, :source_type, "must be auto_discovery or manual") + else + changeset + end + end + defp generate_check_key_if_passive(changeset) do check_type = get_field(changeset, :check_type) check_key = get_field(changeset, :check_key) diff --git a/lib/towerops/monitoring/check_result.ex b/lib/towerops/monitoring/check_result.ex index ff055319..66c5b35d 100644 --- a/lib/towerops/monitoring/check_result.ex +++ b/lib/towerops/monitoring/check_result.ex @@ -19,6 +19,7 @@ defmodule Towerops.Monitoring.CheckResult do field :checked_at, :utc_datetime field :status, :integer field :output, :string + field :value, :float field :response_time_ms, :float belongs_to :organization, Organization @@ -28,7 +29,16 @@ defmodule Towerops.Monitoring.CheckResult do def changeset(result, attrs) do result - |> cast(attrs, [:checked_at, :status, :output, :response_time_ms, :organization_id, :check_id, :agent_token_id]) + |> cast(attrs, [ + :checked_at, + :status, + :output, + :value, + :response_time_ms, + :organization_id, + :check_id, + :agent_token_id + ]) |> validate_required([:checked_at, :status, :organization_id, :check_id]) |> validate_inclusion(:status, 0..3) |> foreign_key_constraint(:organization_id) diff --git a/lib/towerops/monitoring/executors/snmp_interface_executor.ex b/lib/towerops/monitoring/executors/snmp_interface_executor.ex new file mode 100644 index 00000000..67736d2b --- /dev/null +++ b/lib/towerops/monitoring/executors/snmp_interface_executor.ex @@ -0,0 +1,149 @@ +defmodule Towerops.Monitoring.Executors.SnmpInterfaceExecutor do + @moduledoc """ + Executes SNMP interface checks (operational status, bandwidth utilization). + + Polls interface status and traffic counters from IF-MIB. + """ + + alias Towerops.Devices + alias Towerops.Repo + alias Towerops.Snmp + alias Towerops.Snmp.Client + + require Logger + + # IF-MIB OIDs + # ifOperStatus + @if_oper_status_oid "1.3.6.1.2.1.2.2.1.8" + + @doc """ + Executes an SNMP interface check. + + Takes a check struct with: + - source_id: UUID of the interface to poll + - device_id: UUID of the device + - config: Map with interface configuration + + Returns: + - {:ok, %{value:, status:, output:, response_time_ms:}} on success + - {:error, reason} on failure + """ + def execute(check) do + start_time = System.monotonic_time(:millisecond) + + with {:ok, interface} <- get_interface(check.source_id), + {:ok, device} <- get_device_with_snmp(check.device_id), + {:ok, snmp_opts} <- build_snmp_opts(device), + {:ok, oper_status} <- poll_oper_status(snmp_opts, interface.if_index) do + response_time = System.monotonic_time(:millisecond) - start_time + + # Map operational status to check status + status = determine_status(oper_status, interface) + + # Format output with interface name and status + output = format_output(interface, oper_status) + + {:ok, + %{ + value: oper_status, + status: status, + output: output, + response_time_ms: response_time + }} + else + {:error, reason} = error -> + Logger.warning("SNMP interface check failed: #{inspect(reason)}") + error + end + rescue + e -> + Logger.error("SNMP interface check exception: #{inspect(e)}") + {:error, "Exception: #{Exception.message(e)}"} + end + + defp get_interface(interface_id) do + case Snmp.get_interface(interface_id) do + nil -> {:error, "Interface not found: #{interface_id}"} + interface -> {:ok, interface} + end + end + + defp get_device_with_snmp(device_id) do + device = device_id |> Devices.get_device!() |> Repo.preload(:snmp_device) + + if device.snmp_device do + {:ok, device} + else + {:error, "Device #{device_id} does not have SNMP configured"} + end + end + + defp build_snmp_opts(device) do + opts = [ + ip: device.ip_address, + version: device.snmp_version || "2c", + port: device.snmp_port || 161, + timeout: 5000 + ] + + opts = add_snmp_credentials(opts, device) + {:ok, opts} + end + + defp add_snmp_credentials(opts, %{snmp_version: "3"} = device) do + opts + |> Keyword.put(:security_name, device.snmpv3_username || "") + |> Keyword.put(:security_level, device.snmpv3_security_level || "noAuthNoPriv") + |> Keyword.put(:auth_protocol, device.snmpv3_auth_protocol || "MD5") + |> Keyword.put(:auth_password, device.snmpv3_auth_password || "") + |> Keyword.put(:priv_protocol, device.snmpv3_priv_protocol || "DES") + |> Keyword.put(:priv_password, device.snmpv3_priv_password || "") + end + + defp add_snmp_credentials(opts, device) do + Keyword.put(opts, :community, device.snmp_community || "public") + end + + defp poll_oper_status(snmp_opts, if_index) do + oid = "#{@if_oper_status_oid}.#{if_index}" + + case Client.get(snmp_opts, oid) do + {:ok, value} when is_integer(value) -> {:ok, value} + {:ok, value} -> {:error, "Unexpected value type: #{inspect(value)}"} + {:error, _} = error -> error + end + end + + defp determine_status(oper_status, interface) do + # ifOperStatus values from IF-MIB: + # 1 = up, 2 = down, 3 = testing, 4 = unknown, 5 = dormant, 6 = notPresent, 7 = lowerLayerDown + + cond do + # Interface is up - OK + oper_status == 1 -> 0 + # Interface is down - check if this is expected + oper_status == 2 && interface.if_admin_status == "down" -> 0 + # Interface is down but should be up - CRITICAL + oper_status == 2 -> 2 + # Testing, dormant - WARNING + oper_status in [3, 5] -> 1 + # Unknown, notPresent, lowerLayerDown - CRITICAL + true -> 2 + end + end + + defp format_output(interface, oper_status) do + status_str = oper_status_to_string(oper_status) + interface_name = interface.if_alias || interface.if_name || interface.if_descr || "Interface #{interface.if_index}" + "#{interface_name}: #{status_str}" + end + + defp oper_status_to_string(1), do: "Up" + defp oper_status_to_string(2), do: "Down" + defp oper_status_to_string(3), do: "Testing" + defp oper_status_to_string(4), do: "Unknown" + defp oper_status_to_string(5), do: "Dormant" + defp oper_status_to_string(6), do: "Not Present" + defp oper_status_to_string(7), do: "Lower Layer Down" + defp oper_status_to_string(_), do: "Unknown Status" +end diff --git a/lib/towerops/monitoring/executors/snmp_processor_executor.ex b/lib/towerops/monitoring/executors/snmp_processor_executor.ex new file mode 100644 index 00000000..eca6c655 --- /dev/null +++ b/lib/towerops/monitoring/executors/snmp_processor_executor.ex @@ -0,0 +1,148 @@ +defmodule Towerops.Monitoring.Executors.SnmpProcessorExecutor do + @moduledoc """ + Executes SNMP processor/CPU checks. + + Polls CPU load from various MIBs: + - HOST-RESOURCES-MIB hrProcessorLoad + - CISCO-PROCESS-MIB cpmCPUTotal + - UCD-SNMP-MIB system CPU statistics + """ + + alias Towerops.Devices + alias Towerops.Repo + alias Towerops.Snmp + alias Towerops.Snmp.Client + + require Logger + + # OID base paths for different processor types + @hr_processor_load_oid "1.3.6.1.2.1.25.3.3.1.2" + @cisco_cpu_5min_oid "1.3.6.1.4.1.9.9.109.1.1.1.1.5" + + @doc """ + Executes an SNMP processor check. + + Takes a check struct with: + - source_id: UUID of the processor to poll + - device_id: UUID of the device + - config: Map with processor configuration + + Returns: + - {:ok, %{value:, status:, output:, response_time_ms:}} on success + - {:error, reason} on failure + """ + def execute(check) do + start_time = System.monotonic_time(:millisecond) + + with {:ok, processor} <- get_processor(check.source_id), + {:ok, device} <- get_device_with_snmp(check.device_id), + {:ok, snmp_opts} <- build_snmp_opts(device), + {:ok, load_percent} <- poll_processor_load(snmp_opts, processor) do + response_time = System.monotonic_time(:millisecond) - start_time + + # Determine status based on load thresholds + status = determine_status(load_percent) + + # Format output with processor description and load + output = format_output(processor, load_percent) + + {:ok, + %{ + value: load_percent, + status: status, + output: output, + response_time_ms: response_time + }} + else + {:error, reason} = error -> + Logger.warning("SNMP processor check failed: #{inspect(reason)}") + error + end + rescue + e -> + Logger.error("SNMP processor check exception: #{inspect(e)}") + {:error, "Exception: #{Exception.message(e)}"} + end + + defp get_processor(processor_id) do + case Snmp.get_processor(processor_id) do + nil -> {:error, "Processor not found: #{processor_id}"} + processor -> {:ok, processor} + end + end + + defp get_device_with_snmp(device_id) do + device = device_id |> Devices.get_device!() |> Repo.preload(:snmp_device) + + if device.snmp_device do + {:ok, device} + else + {:error, "Device #{device_id} does not have SNMP configured"} + end + end + + defp build_snmp_opts(device) do + opts = [ + ip: device.ip_address, + version: device.snmp_version || "2c", + port: device.snmp_port || 161, + timeout: 5000 + ] + + opts = add_snmp_credentials(opts, device) + {:ok, opts} + end + + defp add_snmp_credentials(opts, %{snmp_version: "3"} = device) do + opts + |> Keyword.put(:security_name, device.snmpv3_username || "") + |> Keyword.put(:security_level, device.snmpv3_security_level || "noAuthNoPriv") + |> Keyword.put(:auth_protocol, device.snmpv3_auth_protocol || "MD5") + |> Keyword.put(:auth_password, device.snmpv3_auth_password || "") + |> Keyword.put(:priv_protocol, device.snmpv3_priv_protocol || "DES") + |> Keyword.put(:priv_password, device.snmpv3_priv_password || "") + end + + defp add_snmp_credentials(opts, device) do + Keyword.put(opts, :community, device.snmp_community || "public") + end + + defp poll_processor_load(snmp_opts, processor) do + oid = build_processor_oid(processor) + + case Client.get(snmp_opts, oid) do + {:ok, value} when is_integer(value) -> {:ok, value * 1.0} + {:ok, value} when is_float(value) -> {:ok, value} + {:ok, value} -> {:error, "Unexpected value type: #{inspect(value)}"} + {:error, _} = error -> error + end + end + + defp build_processor_oid(processor) do + # Extract numeric index from processor_index string + # Examples: "1" -> "1", "cisco_1" -> "1", "hr_1" -> "1" + index = String.replace(processor.processor_index, ~r/[^\d]/, "") + + case processor.processor_type do + "hr_processor" -> "#{@hr_processor_load_oid}.#{index}" + "cisco_cpu" -> "#{@cisco_cpu_5min_oid}.#{index}" + _ -> "#{@hr_processor_load_oid}.#{index}" + end + end + + defp determine_status(load_percent) do + cond do + # CRITICAL + load_percent >= 90 -> 2 + # WARNING + load_percent >= 80 -> 1 + # OK + true -> 0 + end + end + + defp format_output(processor, load_percent) do + description = processor.description || "Processor #{processor.processor_index}" + "#{description}: #{Float.round(load_percent, 1)}% load" + end +end diff --git a/lib/towerops/monitoring/executors/snmp_sensor_executor.ex b/lib/towerops/monitoring/executors/snmp_sensor_executor.ex new file mode 100644 index 00000000..e9740976 --- /dev/null +++ b/lib/towerops/monitoring/executors/snmp_sensor_executor.ex @@ -0,0 +1,166 @@ +defmodule Towerops.Monitoring.Executors.SnmpSensorExecutor do + @moduledoc """ + Executes SNMP sensor checks (temperature, voltage, power, fan speed, etc.). + + Polls a sensor's OID and returns the current value with status determination. + """ + + alias Towerops.Devices + alias Towerops.Repo + alias Towerops.Snmp + alias Towerops.Snmp.Client + + require Logger + + @doc """ + Executes an SNMP sensor check. + + Takes a check struct with: + - source_id: UUID of the sensor to poll + - device_id: UUID of the device + - config: Map with sensor configuration + + Returns: + - {:ok, %{value:, status:, output:, response_time_ms:}} on success + - {:error, reason} on failure + """ + def execute(check) do + start_time = System.monotonic_time(:millisecond) + + with {:ok, sensor} <- get_sensor(check.source_id), + {:ok, device} <- get_device_with_snmp(check.device_id), + {:ok, snmp_opts} <- build_snmp_opts(device), + {:ok, raw_value} <- poll_sensor(snmp_opts, sensor.sensor_oid) do + response_time = System.monotonic_time(:millisecond) - start_time + + # Apply divisor to get actual value + value = raw_value / sensor.sensor_divisor + + # Determine status based on sensor limits (if configured) + status = determine_status(value, sensor) + + # Format output with sensor description and value + output = format_output(sensor, value) + + {:ok, + %{ + value: value, + status: status, + output: output, + response_time_ms: response_time + }} + else + {:error, reason} = error -> + Logger.warning("SNMP sensor check failed: #{inspect(reason)}") + error + end + rescue + e -> + Logger.error("SNMP sensor check exception: #{inspect(e)}") + {:error, "Exception: #{Exception.message(e)}"} + end + + defp get_sensor(sensor_id) do + case Snmp.get_sensor(sensor_id) do + nil -> {:error, "Sensor not found: #{sensor_id}"} + sensor -> {:ok, sensor} + end + end + + defp get_device_with_snmp(device_id) do + device = device_id |> Devices.get_device!() |> Repo.preload(:snmp_device) + + if device.snmp_device do + {:ok, device} + else + {:error, "Device #{device_id} does not have SNMP configured"} + end + end + + defp build_snmp_opts(device) do + opts = [ + ip: device.ip_address, + version: device.snmp_version || "2c", + port: device.snmp_port || 161, + timeout: 5000 + ] + + opts = add_snmp_credentials(opts, device) + {:ok, opts} + end + + defp add_snmp_credentials(opts, %{snmp_version: "3"} = device) do + opts + |> Keyword.put(:security_name, device.snmpv3_username || "") + |> Keyword.put(:security_level, device.snmpv3_security_level || "noAuthNoPriv") + |> Keyword.put(:auth_protocol, device.snmpv3_auth_protocol || "MD5") + |> Keyword.put(:auth_password, device.snmpv3_auth_password || "") + |> Keyword.put(:priv_protocol, device.snmpv3_priv_protocol || "DES") + |> Keyword.put(:priv_password, device.snmpv3_priv_password || "") + end + + defp add_snmp_credentials(opts, device) do + Keyword.put(opts, :community, device.snmp_community || "public") + end + + defp poll_sensor(snmp_opts, oid) do + case Client.get(snmp_opts, oid) do + {:ok, value} when is_number(value) -> {:ok, value} + {:ok, value} -> {:error, "Unexpected value type: #{inspect(value)}"} + {:error, _} = error -> error + end + end + + defp determine_status(value, sensor) do + cond do + critical_high?(value, sensor) -> 2 + critical_low?(value, sensor) -> 2 + warning_high?(value, sensor) -> 1 + warning_low?(value, sensor) -> 1 + true -> 0 + end + end + + defp critical_high?(value, sensor) do + limit = get_in(sensor.metadata, ["limit_high"]) + limit && value >= limit + end + + defp critical_low?(value, sensor) do + limit = get_in(sensor.metadata, ["limit_low"]) + limit && value <= limit + end + + defp warning_high?(value, sensor) do + limit = get_in(sensor.metadata, ["limit_warn_high"]) + limit && value >= limit + end + + defp warning_low?(value, sensor) do + limit = get_in(sensor.metadata, ["limit_warn_low"]) + limit && value <= limit + end + + defp format_output(sensor, value) do + formatted_value = format_value(value, sensor.sensor_type, sensor.sensor_unit) + "#{sensor.sensor_descr}: #{formatted_value}" + end + + @sensor_formats %{ + "temperature" => {1, "°C"}, + "voltage" => {2, "V"}, + "current" => {2, "A"}, + "power" => {1, "W"}, + "frequency" => {0, "Hz"}, + "humidity" => {1, "%"} + } + + defp format_value(value, "fanspeed", unit) do + "#{round(value)}#{unit || " RPM"}" + end + + defp format_value(value, sensor_type, unit) do + {precision, default_unit} = Map.get(@sensor_formats, sensor_type, {2, ""}) + "#{Float.round(value, precision)}#{unit || default_unit}" + end +end diff --git a/lib/towerops/monitoring/executors/snmp_storage_executor.ex b/lib/towerops/monitoring/executors/snmp_storage_executor.ex new file mode 100644 index 00000000..959e37c1 --- /dev/null +++ b/lib/towerops/monitoring/executors/snmp_storage_executor.ex @@ -0,0 +1,180 @@ +defmodule Towerops.Monitoring.Executors.SnmpStorageExecutor do + @moduledoc """ + Executes SNMP storage/disk checks. + + Polls storage usage from HOST-RESOURCES-MIB hrStorageTable: + - Fixed disks (filesystem usage) + - RAM and virtual memory + - Network storage + """ + + alias Towerops.Devices + alias Towerops.Repo + alias Towerops.Snmp + alias Towerops.Snmp.Client + + require Logger + + # HOST-RESOURCES-MIB hrStorageTable OIDs + @hr_storage_allocation_units_oid "1.3.6.1.2.1.25.2.3.1.4" + @hr_storage_size_oid "1.3.6.1.2.1.25.2.3.1.5" + @hr_storage_used_oid "1.3.6.1.2.1.25.2.3.1.6" + + @doc """ + Executes an SNMP storage check. + + Takes a check struct with: + - source_id: UUID of the storage to poll + - device_id: UUID of the device + - config: Map with storage configuration + + Returns: + - {:ok, %{value:, status:, output:, response_time_ms:}} on success + - {:error, reason} on failure + """ + def execute(check) do + start_time = System.monotonic_time(:millisecond) + + with {:ok, storage} <- get_storage(check.source_id), + {:ok, device} <- get_device_with_snmp(check.device_id), + {:ok, snmp_opts} <- build_snmp_opts(device), + {:ok, storage_data} <- poll_storage_usage(snmp_opts, storage.storage_index) do + response_time = System.monotonic_time(:millisecond) - start_time + + # Calculate usage percentage + usage_percent = calculate_usage_percent(storage_data) + + # Determine status based on usage thresholds + status = determine_status(usage_percent) + + # Format output with storage description and usage + output = format_output(storage, usage_percent, storage_data) + + {:ok, + %{ + value: usage_percent, + status: status, + output: output, + response_time_ms: response_time + }} + else + {:error, reason} = error -> + Logger.warning("SNMP storage check failed: #{inspect(reason)}") + error + end + rescue + e -> + Logger.error("SNMP storage check exception: #{inspect(e)}") + {:error, "Exception: #{Exception.message(e)}"} + end + + defp get_storage(storage_id) do + case Snmp.get_storage(storage_id) do + nil -> {:error, "Storage not found: #{storage_id}"} + storage -> {:ok, storage} + end + end + + defp get_device_with_snmp(device_id) do + device = device_id |> Devices.get_device!() |> Repo.preload(:snmp_device) + + if device.snmp_device do + {:ok, device} + else + {:error, "Device #{device_id} does not have SNMP configured"} + end + end + + defp build_snmp_opts(device) do + opts = [ + ip: device.ip_address, + version: device.snmp_version || "2c", + port: device.snmp_port || 161, + timeout: 5000 + ] + + opts = add_snmp_credentials(opts, device) + {:ok, opts} + end + + defp add_snmp_credentials(opts, %{snmp_version: "3"} = device) do + opts + |> Keyword.put(:security_name, device.snmpv3_username || "") + |> Keyword.put(:security_level, device.snmpv3_security_level || "noAuthNoPriv") + |> Keyword.put(:auth_protocol, device.snmpv3_auth_protocol || "MD5") + |> Keyword.put(:auth_password, device.snmpv3_auth_password || "") + |> Keyword.put(:priv_protocol, device.snmpv3_priv_protocol || "DES") + |> Keyword.put(:priv_password, device.snmpv3_priv_password || "") + end + + defp add_snmp_credentials(opts, device) do + Keyword.put(opts, :community, device.snmp_community || "public") + end + + defp poll_storage_usage(snmp_opts, storage_index) do + # Poll allocation units, size, and used from hrStorageTable + alloc_oid = "#{@hr_storage_allocation_units_oid}.#{storage_index}" + size_oid = "#{@hr_storage_size_oid}.#{storage_index}" + used_oid = "#{@hr_storage_used_oid}.#{storage_index}" + + case Client.get_multiple(snmp_opts, [alloc_oid, size_oid, used_oid]) do + {:ok, [alloc_units, size_units, used_units]} -> + {:ok, + %{ + allocation_units: ensure_integer(alloc_units), + size_units: ensure_integer(size_units), + used_units: ensure_integer(used_units) + }} + + {:error, _} = error -> + error + end + end + + defp ensure_integer(value) when is_integer(value), do: value + defp ensure_integer(value) when is_float(value), do: round(value) + defp ensure_integer(_), do: 0 + + defp calculate_usage_percent(%{size_units: 0}), do: 0.0 + + defp calculate_usage_percent(%{size_units: size, used_units: used}) do + used / size * 100.0 + end + + defp determine_status(usage_percent) do + cond do + # CRITICAL + usage_percent >= 95 -> 2 + # WARNING + usage_percent >= 85 -> 1 + # OK + true -> 0 + end + end + + defp format_output(storage, usage_percent, storage_data) do + description = storage.description || storage.device_name || "Storage #{storage.storage_index}" + total_bytes = storage_data.allocation_units * storage_data.size_units + used_bytes = storage_data.allocation_units * storage_data.used_units + + "#{description}: #{Float.round(usage_percent, 1)}% used (#{format_bytes(used_bytes)} / #{format_bytes(total_bytes)})" + end + + defp format_bytes(bytes) when bytes >= 1_099_511_627_776 do + "#{Float.round(bytes / 1_099_511_627_776, 2)} TB" + end + + defp format_bytes(bytes) when bytes >= 1_073_741_824 do + "#{Float.round(bytes / 1_073_741_824, 2)} GB" + end + + defp format_bytes(bytes) when bytes >= 1_048_576 do + "#{Float.round(bytes / 1_048_576, 2)} MB" + end + + defp format_bytes(bytes) when bytes >= 1024 do + "#{Float.round(bytes / 1024, 2)} KB" + end + + defp format_bytes(bytes), do: "#{bytes} B" +end diff --git a/lib/towerops/snmp.ex b/lib/towerops/snmp.ex index e523e67c..d5b7acbd 100644 --- a/lib/towerops/snmp.ex +++ b/lib/towerops/snmp.ex @@ -85,6 +85,197 @@ defmodule Towerops.Snmp do Discovery.discover_all(org_id) end + @doc """ + Creates monitoring checks from discovered SNMP entities. + + After discovery completes, this function creates Check records for: + - Each sensor (temperature, voltage, power, etc.) + - Each interface (operational status monitoring) + - Each processor (CPU load monitoring) + - Each storage volume (disk usage monitoring) + + Checks are automatically enabled and scheduled for execution. + + ## Examples + + iex> create_checks_from_discovery(device, snmp_device) + {:ok, %{sensors: 45, interfaces: 12, processors: 2, storage: 5}} + """ + def create_checks_from_discovery(%DeviceSchema{} = device, %Device{} = snmp_device) do + alias Towerops.Monitoring + + # Preload all associations + snmp_device = Repo.preload(snmp_device, [:sensors, :interfaces, :processors, :storage]) + + results = %{ + sensors: 0, + interfaces: 0, + processors: 0, + storage: 0, + errors: [] + } + + # Create checks for sensors + results = + Enum.reduce(snmp_device.sensors, results, fn sensor, acc -> + case create_sensor_check(device, sensor) do + {:ok, _check} -> Map.update!(acc, :sensors, &(&1 + 1)) + {:error, reason} -> Map.update!(acc, :errors, &[{:sensor, sensor.id, reason} | &1]) + end + end) + + # Create checks for interfaces + results = + Enum.reduce(snmp_device.interfaces, results, fn interface, acc -> + case create_interface_check(device, interface) do + {:ok, _check} -> Map.update!(acc, :interfaces, &(&1 + 1)) + {:error, reason} -> Map.update!(acc, :errors, &[{:interface, interface.id, reason} | &1]) + end + end) + + # Create checks for processors + results = + Enum.reduce(snmp_device.processors, results, fn processor, acc -> + case create_processor_check(device, processor) do + {:ok, _check} -> Map.update!(acc, :processors, &(&1 + 1)) + {:error, reason} -> Map.update!(acc, :errors, &[{:processor, processor.id, reason} | &1]) + end + end) + + # Create checks for storage + results = + Enum.reduce(snmp_device.storage, results, fn storage, acc -> + case create_storage_check(device, storage) do + {:ok, _check} -> Map.update!(acc, :storage, &(&1 + 1)) + {:error, reason} -> Map.update!(acc, :errors, &[{:storage, storage.id, reason} | &1]) + end + end) + + Logger.info( + "Created checks for device #{device.id}: #{results.sensors} sensors, #{results.interfaces} interfaces, #{results.processors} processors, #{results.storage} storage" + ) + + {:ok, results} + end + + defp create_sensor_check(device, sensor) do + alias Towerops.Monitoring + + attrs = %{ + organization_id: device.organization_id, + device_id: device.id, + name: sensor.sensor_descr, + check_type: "snmp_sensor", + source_type: "auto_discovery", + source_id: sensor.id, + interval_seconds: 60, + enabled: sensor.monitored, + config: %{ + "sensor_type" => sensor.sensor_type, + "sensor_oid" => sensor.sensor_oid, + "sensor_unit" => sensor.sensor_unit + } + } + + case Monitoring.create_check(attrs) do + {:ok, check} -> + # Schedule first execution + Monitoring.schedule_check(check) + {:ok, check} + + error -> + error + end + end + + defp create_interface_check(device, interface) do + alias Towerops.Monitoring + + name = interface.if_alias || interface.if_name || interface.if_descr || "Interface #{interface.if_index}" + + attrs = %{ + organization_id: device.organization_id, + device_id: device.id, + name: "#{name} Status", + check_type: "snmp_interface", + source_type: "auto_discovery", + source_id: interface.id, + interval_seconds: 60, + enabled: interface.monitored, + config: %{ + "if_index" => interface.if_index, + "if_descr" => interface.if_descr + } + } + + case Monitoring.create_check(attrs) do + {:ok, check} -> + Monitoring.schedule_check(check) + {:ok, check} + + error -> + error + end + end + + defp create_processor_check(device, processor) do + alias Towerops.Monitoring + + attrs = %{ + organization_id: device.organization_id, + device_id: device.id, + name: processor.description || "Processor #{processor.processor_index}", + check_type: "snmp_processor", + source_type: "auto_discovery", + source_id: processor.id, + interval_seconds: 60, + enabled: true, + config: %{ + "processor_type" => processor.processor_type, + "processor_index" => processor.processor_index + } + } + + case Monitoring.create_check(attrs) do + {:ok, check} -> + Monitoring.schedule_check(check) + {:ok, check} + + error -> + error + end + end + + defp create_storage_check(device, storage) do + alias Towerops.Monitoring + + name = storage.description || storage.device_name || "Storage #{storage.storage_index}" + + attrs = %{ + organization_id: device.organization_id, + device_id: device.id, + name: "#{name} Usage", + check_type: "snmp_storage", + source_type: "auto_discovery", + source_id: storage.id, + interval_seconds: 300, + enabled: true, + config: %{ + "storage_type" => storage.storage_type, + "storage_index" => storage.storage_index + } + } + + case Monitoring.create_check(attrs) do + {:ok, check} -> + Monitoring.schedule_check(check) + {:ok, check} + + error -> + error + end + end + # Device queries @doc """ diff --git a/lib/towerops/snmp/discovery.ex b/lib/towerops/snmp/discovery.ex index 6c3ac759..ece5a8fd 100644 --- a/lib/towerops/snmp/discovery.ex +++ b/lib/towerops/snmp/discovery.ex @@ -270,6 +270,12 @@ defmodule Towerops.Snmp.Discovery do is_rediscovery = device.last_discovery_at != nil _ = log_discovery_event(device, discovered_device, is_rediscovery) + # Create monitoring checks for discovered entities + Logger.info("Creating monitoring checks from discovered entities...", device_id: device.id) + + {:ok, check_counts} = Towerops.Snmp.create_checks_from_discovery(device, discovered_device) + log_check_creation_results(device.id, check_counts) + _ = Phoenix.PubSub.broadcast( Towerops.PubSub, @@ -1185,6 +1191,26 @@ defmodule Towerops.Snmp.Discovery do :ok end + defp log_check_creation_results(device_id, check_counts) do + total_checks = + check_counts.sensors + check_counts.interfaces + check_counts.processors + check_counts.storage + + Logger.info( + "Created #{total_checks} checks: " <> + "#{check_counts.sensors} sensors, #{check_counts.interfaces} interfaces, " <> + "#{check_counts.processors} processors, #{check_counts.storage} storage", + device_id: device_id + ) + + if check_counts.errors != [] do + Logger.warning( + "Failed to create #{length(check_counts.errors)} checks", + device_id: device_id, + errors: check_counts.errors + ) + end + end + defp log_discovery_event(device, discovered_device, is_rediscovery) do event_type = if is_rediscovery, do: "device_rediscovered", else: "device_discovered" diff --git a/lib/towerops/workers/check_executor_worker.ex b/lib/towerops/workers/check_executor_worker.ex new file mode 100644 index 00000000..7727a4eb --- /dev/null +++ b/lib/towerops/workers/check_executor_worker.ex @@ -0,0 +1,165 @@ +defmodule Towerops.Workers.CheckExecutorWorker do + @moduledoc """ + Unified worker for executing all check types. + + Dispatches to appropriate executor based on check_type: + - snmp_sensor → SnmpSensorExecutor + - snmp_interface → SnmpInterfaceExecutor + - snmp_processor → SnmpProcessorExecutor + - snmp_storage → SnmpStorageExecutor + - http → HttpExecutor + - tcp → TcpExecutor + - dns → DnsExecutor + - ping → PingExecutor (future) + + Records results in check_results TimescaleDB hypertable and updates + check state (OK/WARNING/CRITICAL/UNKNOWN). + + Self-schedules next execution based on check interval with distributed + polling offsets to prevent thundering herd. + """ + + use Oban.Worker, + queue: :check_executors, + max_attempts: 3 + + alias Towerops.Monitoring + alias Towerops.Monitoring.Executors.DnsExecutor + alias Towerops.Monitoring.Executors.HttpExecutor + alias Towerops.Monitoring.Executors.SnmpInterfaceExecutor + alias Towerops.Monitoring.Executors.SnmpProcessorExecutor + alias Towerops.Monitoring.Executors.SnmpSensorExecutor + alias Towerops.Monitoring.Executors.SnmpStorageExecutor + alias Towerops.Monitoring.Executors.TcpExecutor + alias Towerops.Workers.PollingOffset + + require Logger + + @impl Oban.Worker + def perform(%Oban.Job{args: %{"check_id" => check_id}}) do + case Monitoring.get_check(check_id) do + nil -> + Logger.debug("Check #{check_id} deleted, skipping execution") + :ok + + check -> + if check.enabled do + execute_and_record(check) + schedule_next_check(check) + else + Logger.debug("Check #{check_id} disabled, skipping execution") + end + + :ok + end + end + + defp execute_and_record(check) do + Logger.debug("Executing check #{check.id} (#{check.check_type})") + + check + |> dispatch_executor() + |> record_result(check) + end + + defp dispatch_executor(%{check_type: "snmp_sensor"} = check), do: SnmpSensorExecutor.execute(check) + defp dispatch_executor(%{check_type: "snmp_interface"} = check), do: SnmpInterfaceExecutor.execute(check) + defp dispatch_executor(%{check_type: "snmp_processor"} = check), do: SnmpProcessorExecutor.execute(check) + defp dispatch_executor(%{check_type: "snmp_storage"} = check), do: SnmpStorageExecutor.execute(check) + defp dispatch_executor(%{check_type: "http"} = check), do: execute_http_check(check) + defp dispatch_executor(%{check_type: "tcp"} = check), do: execute_tcp_check(check) + defp dispatch_executor(%{check_type: "dns"} = check), do: execute_dns_check(check) + defp dispatch_executor(%{check_type: "ping"}), do: {:error, "Ping executor not yet implemented"} + defp dispatch_executor(%{check_type: type}), do: {:error, "Unknown check type: #{type}"} + + defp record_result({:ok, %{value: value, status: status, output: output, response_time_ms: time}}, check) do + Monitoring.create_check_result(%{ + check_id: check.id, + organization_id: check.organization_id, + value: value, + status: status, + output: output, + response_time_ms: time, + checked_at: DateTime.utc_now(), + agent_token_id: check.agent_token_id + }) + + Monitoring.update_check_state(check, status, output) + Logger.debug("Check #{check.id} completed: status=#{status}, value=#{value}, output=#{output}") + end + + defp record_result({:error, reason}, check) do + Logger.warning("Check #{check.id} failed: #{inspect(reason)}") + + Monitoring.create_check_result(%{ + check_id: check.id, + organization_id: check.organization_id, + status: 3, + output: "Error: #{inspect(reason)}", + checked_at: DateTime.utc_now(), + agent_token_id: check.agent_token_id + }) + + Monitoring.update_check_state(check, 3, "Error: #{inspect(reason)}") + end + + # Adapter for HTTP executor which returns different format + defp execute_http_check(check) do + case HttpExecutor.execute(check.config, check.timeout_ms) do + {:ok, response_time, output} -> + {:ok, + %{ + value: nil, + status: 0, + output: output, + response_time_ms: response_time + }} + + {:error, reason} -> + {:error, reason} + end + end + + # Adapter for TCP executor which returns different format + defp execute_tcp_check(check) do + case TcpExecutor.execute(check.config, check.timeout_ms) do + {:ok, response_time, output} -> + {:ok, + %{ + value: nil, + status: 0, + output: output, + response_time_ms: response_time + }} + + {:error, reason} -> + {:error, reason} + end + end + + # Adapter for DNS executor which returns different format + defp execute_dns_check(check) do + case DnsExecutor.execute(check.config, check.timeout_ms) do + {:ok, response_time, output} -> + {:ok, + %{ + value: nil, + status: 0, + output: output, + response_time_ms: response_time + }} + + {:error, reason} -> + {:error, reason} + end + end + + defp schedule_next_check(check) do + # Calculate staggered offset based on check ID + offset = PollingOffset.calculate_offset(check.id, check.interval_seconds) + + %{check_id: check.id} + |> new(schedule_in: offset) + |> Oban.insert() + end +end diff --git a/lib/towerops_web/controllers/api_docs_html/index.html.heex b/lib/towerops_web/controllers/api_docs_html/index.html.heex index 75dd4b59..e4ce3382 100644 --- a/lib/towerops_web/controllers/api_docs_html/index.html.heex +++ b/lib/towerops_web/controllers/api_docs_html/index.html.heex @@ -87,6 +87,22 @@ + + +
  • +

    Tools

    + +
    • @@ -1330,6 +1346,336 @@ Content-Disposition: attachment; filename="towerops-data-{user_id}-{timestamp}.j + +
    + + +
    +

    + Terraform Provider +

    +

    + Manage your Towerops infrastructure as code using our official Terraform provider. Automate site and device provisioning, version control your network configuration, and integrate Towerops into your existing infrastructure workflows. +

    + +
    +
    +
    + + + +
    +
    +

    + Infrastructure as Code +

    +
    +

    + The Terraform provider uses the same REST API documented above. All resource operations require API token authentication (not browser sessions). +

    +
    +
    +
    +
    + + +
    +

    What is Terraform?

    +

    + + Terraform + + is an infrastructure as code tool that lets you define both cloud and on-premise resources in human-readable configuration files that you can version, reuse, and share. With the Towerops Terraform provider, you can: +

    + +
      +
    • Define sites and devices in declarative configuration files
      • +
    • Version control your network monitoring infrastructure
      • +
    • Automate device provisioning and updates
      • +
    • Integrate with CI/CD pipelines for automated deployments
      • +
    • Ensure consistent configuration across environments
      • +
    • Track infrastructure changes with git history
      • +
    +
    + +
    + + +
    +

    Getting Started

    + +
    + +
    +

    Installation

    +

    + The provider is available on the Terraform Registry. Terraform will automatically download it when you run terraform init. +

    + +

    + Requirements +

    +
      +
    • Terraform >= 1.0
      • +
    • Towerops API token (create in Organization Settings)
      • +
    +
    + + +
    +
    +
    +

    Configuration

    +
    + 
    <%= raw(~S"""
+terraform {
+  required_providers {
+    towerops = {
+      source  = "towerops-app/towerops"
+      version = "~> 0.1"
+    }
+  }
+}
+
+variable "towerops_api_token" {
+  type      = string
+  sensitive = true
+}
+
+provider "towerops" {
+  token   = var.towerops_api_token
+  api_url = "https://towerops.net"
+}
+""") %>
    +
    +
    +
    +
    + +
    + + +
    +

    Example Usage

    + +
    +
    +

    + This example creates a site and two devices. Terraform will automatically manage the lifecycle of these resources, creating, updating, or deleting them as needed to match your configuration. +

    + +

    + Key Features +

    +
      +
    • Automatic dependency management (devices depend on sites)
      • +
    • State tracking to detect configuration drift
      • +
    • Plan preview before making changes
      • +
    • Rollback support via version control
      • +
    +
    + +
    +
    +
    +

    main.tf

    +
    + 
    <%= raw(~S"""
+resource "towerops_site" "main_office" {
+  name     = "Main Office"
+  location = "New York, NY"
+}
+
+resource "towerops_device" "core_router" {
+  site_id      = towerops_site.main_office.id
+  name         = "Core Router"
+  ip_address   = "192.168.1.1"
+  description  = "Primary gateway"
+  snmp_enabled = true
+  snmp_version = "2c"
+}
+
+resource "towerops_device" "backup_router" {
+  site_id      = towerops_site.main_office.id
+  name         = "Backup Router"
+  ip_address   = "192.168.1.2"
+  description  = "Failover gateway"
+  snmp_enabled = true
+}
+""") %>
    +
    + +
    +
    +

    Commands

    +
    + 
    <%= raw(~S"""
+# Initialize and download provider
+terraform init
+
+# Preview changes
+terraform plan
+
+# Apply configuration
+terraform apply
+
+# Destroy all resources
+terraform destroy
+""") %>
    +
    +
    +
    +
    + +
    + + +
    +

    Available Resources

    + +
    +
    +
    +
    + towerops_site +
    +
    + Manages a physical site/location. Supports name, location, and SNMP community string configuration. +
    +
    + + View documentation → + +
    +
    + +
    +
    + towerops_device +
    +
    + Manages network equipment at a site. Supports IP address, SNMP configuration, monitoring settings, and more. +
    +
    + + View documentation → + +
    +
    +
    +
    +
    + +
    + + +
    +

    Full Documentation

    + +
    + +
    +
    +

    + Provider Documentation +

    +

    + Complete reference for all resources, data sources, and provider configuration options. +

    +
    + + + +
    +     + + +
    +
    +

    + GitHub Repository +

    +

    + Source code, examples, and issue tracking for the Terraform provider. +

    +
    + + + +
    +     +
    +
    + +
    +

    Need Help?

    +

    + For questions about the Terraform provider, please open an issue on + + GitHub + + or contact us at hi@towerops.net. +

    +
    +