# Firmware Version Tracking System Design **Date:** 2026-02-01 **Status:** Draft **Target:** MikroTik initially, extensible to all vendors ## Overview Implement a firmware version tracking system that: 1. Fetches latest stable firmware versions from vendor sources (MikroTik RSS feed initially) 2. Tracks firmware version changes over time with audit logging 3. Displays update indicators on device detail pages with download links 4. Supports multiple vendors through polymorphic design ## Database Schema ### New Tables #### 1. `firmware_releases` - Latest available firmware versions Stores the most recent stable firmware version for each vendor/product line. ```elixir create table(:firmware_releases, primary_key: false) do add :id, :binary_id, primary_key: true add :vendor, :string, null: false # "mikrotik", "cisco", "ubiquiti" add :product_line, :string # "routeros", "ios-xe", "unifi", null for single product vendors add :version, :string, null: false # "7.14.1", "17.9.4a" add :release_date, :date # When this version was released add :download_url, :string # Official download page URL add :changelog_url, :string # Release notes URL add :metadata, :map, default: %{} # JSON: RSS item data, API response, etc. add :fetched_at, :utc_datetime, null: false # When we last fetched this timestamps(type: :utc_datetime) end # Unique constraint: one current version per vendor/product_line create unique_index(:firmware_releases, [:vendor, :product_line]) ``` **Design Notes:** - `vendor` is lowercase string for consistency ("mikrotik", not "MikroTik") - `product_line` allows vendors with multiple firmware branches (Cisco IOS vs IOS-XE, MikroTik RouterOS vs SwOS) - `metadata` stores raw source data for debugging and future feature extraction - Single record per vendor/product_line, updated in place when new version detected #### 2. `device_firmware_history` - Version change audit trail Tracks when devices change firmware versions. ```elixir create table(:device_firmware_history, primary_key: false) do add :id, :binary_id, primary_key: true add :snmp_device_id, references(:snmp_devices, type: :binary_id, on_delete: :delete_all), null: false add :old_version, :string # Previous version (null for first discovery) add :new_version, :string, null: false # New version detected add :detected_at, :utc_datetime, null: false # When the change was detected add :detection_method, :string # "discovery", "polling", "manual" timestamps(type: :utc_datetime, updated_at: false) end create index(:device_firmware_history, [:snmp_device_id, :detected_at]) create index(:device_firmware_history, [:detected_at]) ``` **Design Notes:** - Links to `snmp_devices` (not `devices`) since firmware is SNMP-discovered data - `old_version` nullable for initial discovery (no previous version) - `detected_at` separate from `inserted_at` to record actual change time vs when we logged it - Index on `snmp_device_id + detected_at` for efficient device history queries - Cascading delete: history removed when SNMP device deleted ### Schema Modifications No changes to existing `snmp_devices` table needed - the existing `firmware_version` field remains as the "current version" source of truth. ## RSS Fetching Implementation ### Oban Worker: `FirmwareVersionFetcherWorker` **Location:** `lib/towerops/workers/firmware_version_fetcher_worker.ex` ```elixir defmodule Towerops.Workers.FirmwareVersionFetcherWorker do use Oban.Worker, queue: :maintenance, max_attempts: 3 require Logger @impl Oban.Worker def perform(%Oban.Job{}) do Logger.info("Starting firmware version fetch") # Fetch each vendor's latest firmware results = [ fetch_mikrotik_routeros(), # Future: fetch_cisco_ios(), # Future: fetch_ubiquiti_unifi() ] case Enum.all?(results, &match?(:ok, &1)) do true -> :ok false -> {:error, "One or more firmware fetches failed"} end end defp fetch_mikrotik_routeros do # Implementation details below end end ``` **Cron Schedule:** Daily at 2:00 AM Add to `config/dev.exs` and `config/runtime.exs` (production): ```elixir config :towerops, Oban, plugins: [ {Oban.Plugins.Cron, crontab: [ # ... existing cron jobs ... {"0 2 * * *", Towerops.Workers.FirmwareVersionFetcherWorker} ]} ] ``` ### MikroTik RSS Parsing **RSS URL:** `https://cdn.mikrotik.com/routeros/latest-stable.rss` **Sample RSS Structure:** ```xml RouterOS latest stable version 7.14.1 https://mikrotik.com/download Wed, 15 Jan 2025 12:00:00 +0000 RouterOS 7.14.1 stable release ``` **Parsing Implementation:** ```elixir defp fetch_mikrotik_routeros do url = "https://cdn.mikrotik.com/routeros/latest-stable.rss" with {:ok, %{status: 200, body: body}} <- Req.get(url), {:ok, parsed} <- parse_mikrotik_rss(body), {:ok, _release} <- upsert_firmware_release(parsed) do Logger.info("Successfully fetched MikroTik RouterOS: #{parsed.version}") :ok else {:error, reason} = error -> Logger.error("Failed to fetch MikroTik firmware: #{inspect(reason)}") error end end defp parse_mikrotik_rss(xml_body) do # Use :xmerl or SweetXml library import SweetXml try do version = xml_body |> xpath(~x"//item/title/text()"s) |> String.trim() link = xml_body |> xpath(~x"//item/link/text()"s) |> String.trim() pub_date_str = xml_body |> xpath(~x"//item/pubDate/text()"s) |> String.trim() release_date = parse_rfc822_date(pub_date_str) {:ok, %{ vendor: "mikrotik", product_line: "routeros", version: version, release_date: release_date, download_url: link, changelog_url: "https://mikrotik.com/download/changelogs", metadata: %{ rss_title: version, rss_description: xml_body |> xpath(~x"//item/description/text()"s) } }} rescue e -> {:error, "XML parsing failed: #{inspect(e)}"} end end defp parse_rfc822_date(date_str) do # "Wed, 15 Jan 2025 12:00:00 +0000" -> ~D[2025-01-15] case Timex.parse(date_str, "{RFC822}") do {:ok, datetime} -> DateTime.to_date(datetime) _ -> Date.utc_today() # Fallback to today if parse fails end end ``` **Dependencies:** - Add `{:sweet_xml, "~> 0.7"}` to `mix.exs` for XML parsing - Use existing `:req` for HTTP (already in project) - Add `{:timex, "~> 3.7"}` for RFC822 date parsing (or use standard library alternative) ### Database Upsert Logic ```elixir defp upsert_firmware_release(attrs) do import Ecto.Query # Use ON CONFLICT to update existing record %Towerops.Devices.FirmwareRelease{} |> Towerops.Devices.FirmwareRelease.changeset(attrs) |> Towerops.Repo.insert( on_conflict: {:replace_all_except, [:id, :inserted_at]}, conflict_target: [:vendor, :product_line] ) end ``` ## Version Change Detection ### Integration Point: Discovery Flow **File:** `lib/towerops/snmp/discovery.ex` **Current Flow:** 1. `run_discovery/1` orchestrates discovery stages 2. `build_device_info/3` extracts firmware version from SNMP 3. `upsert_device/2` updates/inserts SNMP device record **New Logic:** Add version change detection in `upsert_device/2` ```elixir defp upsert_device(device, device_info) do # Fetch current version BEFORE update current_version = get_current_firmware_version(device.id) new_version = device_info[:firmware_version] # Perform existing upsert result = device |> SnmpDevice.changeset(device_info) |> Repo.insert_or_update() # Detect and log version change case result do {:ok, snmp_device} -> if version_changed?(current_version, new_version) do log_firmware_change(snmp_device.id, current_version, new_version) end {:ok, snmp_device} error -> error end end defp get_current_firmware_version(device_id) do case Repo.get_by(SnmpDevice, device_id: device_id) do nil -> nil snmp_device -> snmp_device.firmware_version end end defp version_changed?(nil, new_version) when is_binary(new_version), do: false # Initial discovery defp version_changed?(old, new) when old == new, do: false defp version_changed?(old, new) when is_binary(old) and is_binary(new), do: true defp version_changed?(_, _), do: false ``` ### Firmware Change Logging **File:** `lib/towerops/devices/firmware.ex` (new context module) ```elixir defmodule Towerops.Devices.Firmware do import Ecto.Query alias Towerops.Repo alias Towerops.Devices.DeviceFirmwareHistory def log_firmware_change(snmp_device_id, old_version, new_version) do attrs = %{ snmp_device_id: snmp_device_id, old_version: old_version, new_version: new_version, detected_at: DateTime.utc_now(), detection_method: "discovery" } %DeviceFirmwareHistory{} |> DeviceFirmwareHistory.changeset(attrs) |> Repo.insert() |> case do {:ok, history} -> # Create audit log entry create_audit_log(history) # Broadcast to device topic for real-time updates broadcast_firmware_change(snmp_device_id, old_version, new_version) {:ok, history} error -> error end end defp create_audit_log(history) do # Use existing audit log system if available # Or create device event Logger.info("Firmware changed on device #{history.snmp_device_id}: #{history.old_version} -> #{history.new_version}") end defp broadcast_firmware_change(snmp_device_id, old_version, new_version) do Phoenix.PubSub.broadcast( Towerops.PubSub, "device:#{snmp_device_id}", {:firmware_changed, snmp_device_id, old_version, new_version} ) end end ``` ## Version Comparison Logic ### Semantic Version Parser **File:** `lib/towerops/devices/version_comparator.ex` ```elixir defmodule Towerops.Devices.VersionComparator do @moduledoc """ Semantic version comparison for firmware versions. Handles common formats: - X.Y.Z (7.14.1) - X.Y (7.14) - X.Y.Z-suffix (7.14.1-beta) """ def compare(version1, version2) do parsed1 = parse_version(version1) parsed2 = parse_version(version2) do_compare(parsed1, parsed2) end def newer?(current, available) do compare(current, available) == :lt end defp parse_version(version) when is_binary(version) do # Remove common prefixes cleaned = version |> String.trim() |> String.replace(~r/^v/i, "") # Split on dots and extract numbers parts = cleaned |> String.split(["-", " "], parts: 2) |> List.first() |> String.split(".") |> Enum.map(&String.to_integer/1) # Pad to [major, minor, patch] format case parts do [major, minor, patch] -> {major, minor, patch} [major, minor] -> {major, minor, 0} [major] -> {major, 0, 0} _ -> {0, 0, 0} end rescue _ -> {0, 0, 0} # Invalid version defaults to 0.0.0 end defp do_compare({maj1, min1, patch1}, {maj2, min2, patch2}) do cond do maj1 > maj2 -> :gt maj1 < maj2 -> :lt min1 > min2 -> :gt min1 < min2 -> :lt patch1 > patch2 -> :gt patch1 < patch2 -> :lt true -> :eq end end end ``` **Test Cases:** ```elixir defmodule Towerops.Devices.VersionComparatorTest do use ExUnit.Case, async: true alias Towerops.Devices.VersionComparator describe "compare/2" do test "major version differences" do assert VersionComparator.compare("7.14.1", "8.0.0") == :lt assert VersionComparator.compare("8.0.0", "7.14.1") == :gt end test "minor version differences" do assert VersionComparator.compare("7.12.1", "7.14.1") == :lt assert VersionComparator.compare("7.14.1", "7.12.1") == :gt end test "patch version differences" do assert VersionComparator.compare("7.14.1", "7.14.3") == :lt assert VersionComparator.compare("7.14.3", "7.14.1") == :gt end test "equal versions" do assert VersionComparator.compare("7.14.1", "7.14.1") == :eq end test "handles missing patch versions" do assert VersionComparator.compare("7.14", "7.14.1") == :lt assert VersionComparator.compare("7.14.0", "7.14") == :eq end test "handles version prefixes" do assert VersionComparator.compare("v7.14.1", "7.14.2") == :lt assert VersionComparator.compare("V7.14.1", "7.14.2") == :lt end test "handles beta/rc suffixes" do assert VersionComparator.compare("7.14.1-beta", "7.14.1") == :eq # Ignores suffix assert VersionComparator.compare("7.14.1-rc1", "7.14.2") == :lt end test "handles invalid versions" do assert VersionComparator.compare("invalid", "7.14.1") == :lt assert VersionComparator.compare("7.14.1", "invalid") == :gt end end describe "newer?/2" do test "returns true when available version is newer" do assert VersionComparator.newer?("7.12.1", "7.14.1") end test "returns false when current version is newer or equal" do refute VersionComparator.newer?("7.14.1", "7.12.1") refute VersionComparator.newer?("7.14.1", "7.14.1") end end end ``` ## LiveView Integration ### Device Detail Page Updates **File:** `lib/towerops_web/live/device_live/show.ex` **Add to mount/assigns:** ```elixir def mount(%{"id" => id}, _session, socket) do # ... existing code ... socket = socket # ... existing assigns ... |> assign(:available_firmware, nil) # Will be loaded on first refresh |> load_available_firmware() # New helper {:ok, socket} end defp load_available_firmware(socket) do snmp_device = socket.assigns.snmp_device case get_available_firmware(snmp_device) do {:ok, firmware_release} -> assign(socket, :available_firmware, firmware_release) {:error, _} -> assign(socket, :available_firmware, nil) end end defp get_available_firmware(nil), do: {:error, :no_snmp_device} defp get_available_firmware(snmp_device) do # Determine vendor/product_line from snmp_device.manufacturer vendor = determine_vendor(snmp_device.manufacturer) product_line = determine_product_line(snmp_device.manufacturer, snmp_device.model) case Towerops.Devices.Firmware.get_latest_release(vendor, product_line) do nil -> {:error, :no_release_data} release -> {:ok, release} end end defp determine_vendor("MikroTik"), do: "mikrotik" defp determine_vendor("Cisco"), do: "cisco" defp determine_vendor(_), do: nil defp determine_product_line("MikroTik", _model), do: "routeros" # Future: Cisco IOS vs IOS-XE detection based on model defp determine_product_line(_, _), do: nil ``` **Add to PubSub handler:** ```elixir def handle_info({:firmware_changed, _snmp_device_id, _old, _new}, socket) do # Reload firmware data when change detected {:noreply, load_available_firmware(socket)} end ``` ### UI Component **File:** `lib/towerops_web/live/device_live/show.html.heex` **Add near top of Overview tab (after device name/status):** ```heex <%= if firmware_update_available?(@snmp_device, @available_firmware) do %>
<.icon name="hero-arrow-up-circle" class="h-5 w-5 text-blue-400" />

Firmware Update Available

Current version: <%= @snmp_device.firmware_version %>
Latest version: <%= @available_firmware.version %> (released <%= format_date(@available_firmware.release_date) %>)

<% end %> ``` **Helper functions:** ```elixir defp firmware_update_available?(nil, _), do: false defp firmware_update_available?(_, nil), do: false defp firmware_update_available?(snmp_device, available_firmware) do current = snmp_device.firmware_version available = available_firmware.version current && available && Towerops.Devices.VersionComparator.newer?(current, available) end defp format_date(nil), do: "" defp format_date(date) do Calendar.strftime(date, "%B %-d, %Y") # "January 15, 2026" end ``` ## Context Module: `Towerops.Devices.Firmware` **File:** `lib/towerops/devices/firmware.ex` ```elixir defmodule Towerops.Devices.Firmware do @moduledoc """ Context for firmware version tracking and management. """ import Ecto.Query alias Towerops.Repo alias Towerops.Devices.{FirmwareRelease, DeviceFirmwareHistory} require Logger ## Firmware Releases def get_latest_release(vendor, product_line) do Repo.get_by(FirmwareRelease, vendor: vendor, product_line: product_line) end def upsert_firmware_release(attrs) do %FirmwareRelease{} |> FirmwareRelease.changeset(attrs) |> Repo.insert( on_conflict: {:replace_all_except, [:id, :inserted_at]}, conflict_target: [:vendor, :product_line] ) end ## Firmware History def list_device_firmware_history(snmp_device_id, limit \\ 20) do DeviceFirmwareHistory |> where([h], h.snmp_device_id == ^snmp_device_id) |> order_by([h], desc: h.detected_at) |> limit(^limit) |> Repo.all() end def log_firmware_change(snmp_device_id, old_version, new_version) do attrs = %{ snmp_device_id: snmp_device_id, old_version: old_version, new_version: new_version, detected_at: DateTime.utc_now(), detection_method: "discovery" } %DeviceFirmwareHistory{} |> DeviceFirmwareHistory.changeset(attrs) |> Repo.insert() |> case do {:ok, history} -> log_change_event(history) broadcast_firmware_change(snmp_device_id, old_version, new_version) {:ok, history} error -> error end end defp log_change_event(history) do Logger.info( "Firmware version changed", snmp_device_id: history.snmp_device_id, old_version: history.old_version, new_version: history.new_version, detected_at: history.detected_at ) end defp broadcast_firmware_change(snmp_device_id, old_version, new_version) do Phoenix.PubSub.broadcast( Towerops.PubSub, "device:#{snmp_device_id}", {:firmware_changed, snmp_device_id, old_version, new_version} ) end end ``` ## Extensibility for Future Vendors ### Adding Cisco IOS Support (Example) **1. Add fetch function to worker:** ```elixir # In FirmwareVersionFetcherWorker defp fetch_cisco_ios do # Cisco uses different mechanism - maybe API or web scraping url = "https://software.cisco.com/download/latest/ios-xe" with {:ok, %{status: 200, body: body}} <- Req.get(url, headers: [{"Accept", "application/json"}]), {:ok, parsed} <- parse_cisco_api(body), {:ok, _release} <- Towerops.Devices.Firmware.upsert_firmware_release(parsed) do :ok end end defp parse_cisco_api(json_body) do # Vendor-specific parsing data = Jason.decode!(json_body) {:ok, %{ vendor: "cisco", product_line: "ios-xe", version: data["latestVersion"], download_url: data["downloadUrl"], # ... }} end ``` **2. Update vendor detection in LiveView:** ```elixir defp determine_vendor("Cisco Systems"), do: "cisco" defp determine_product_line("Cisco Systems", model) do cond do String.contains?(model, "Catalyst") -> "ios-xe" String.contains?(model, "ASR") -> "ios-xr" true -> "ios" end end ``` **3. Run migration to add new firmware release record** (happens automatically via worker) ### Configuration-Based Approach (Future Enhancement) Store vendor fetch configurations in database or config file: ```elixir # config/firmware_sources.exs [ %{ vendor: "mikrotik", product_line: "routeros", source_type: :rss, url: "https://cdn.mikrotik.com/routeros/latest-stable.rss", parser: Towerops.Devices.FirmwareParsers.MikroTikRSS }, %{ vendor: "cisco", product_line: "ios-xe", source_type: :api, url: "https://software.cisco.com/api/latest", parser: Towerops.Devices.FirmwareParsers.CiscoAPI, auth: :api_key } ] ``` ## Testing Strategy ### Unit Tests **1. Version Comparator Tests** (see above) **2. RSS Parsing Tests:** ```elixir defmodule Towerops.Workers.FirmwareVersionFetcherWorkerTest do use Towerops.DataCase test "parses MikroTik RSS correctly" do xml = """ 7.14.1 https://mikrotik.com/download Wed, 15 Jan 2025 12:00:00 +0000 """ # Test parsing logic end end ``` **3. Version Change Detection:** ```elixir test "detects firmware version change during discovery" do device = insert(:device) snmp_device = insert(:snmp_device, device: device, firmware_version: "7.12.1") # Run discovery with new version # Assert DeviceFirmwareHistory record created # Assert PubSub broadcast sent end test "does not log change for initial discovery" do device = insert(:device) # No existing snmp_device # Run discovery # Assert no DeviceFirmwareHistory record end ``` ### Integration Tests **1. Worker Execution:** ```elixir test "FirmwareVersionFetcherWorker fetches and stores MikroTik version" do # Mock HTTP response # Execute worker # Assert FirmwareRelease record created/updated end ``` **2. LiveView Display:** ```elixir test "shows firmware update indicator when newer version available" do firmware_release = insert(:firmware_release, vendor: "mikrotik", version: "7.14.1") snmp_device = insert(:snmp_device, manufacturer: "MikroTik", firmware_version: "7.12.1") {:ok, view, _html} = live(conn, ~p"/devices/#{snmp_device.device_id}") assert has_element?(view, "[data-test='firmware-update-indicator']") assert render(view) =~ "7.14.1" end ``` ## Implementation Checklist ### Phase 1: Database Schema - [ ] Create migration for `firmware_releases` table - [ ] Create migration for `device_firmware_history` table - [ ] Create `FirmwareRelease` schema module - [ ] Create `DeviceFirmwareHistory` schema module - [ ] Create `Towerops.Devices.Firmware` context module - [ ] Write tests for schema validations ### Phase 2: Version Comparison - [ ] Create `VersionComparator` module - [ ] Write comprehensive version comparison tests - [ ] Handle edge cases (missing patches, prefixes, suffixes) ### Phase 3: RSS Fetching Worker - [ ] Add `sweet_xml` dependency to mix.exs - [ ] Create `FirmwareVersionFetcherWorker` - [ ] Implement MikroTik RSS parsing - [ ] Add worker to Oban cron schedule - [ ] Write worker tests with mocked HTTP responses - [ ] Test error handling (network failures, malformed XML) ### Phase 4: Version Change Detection - [ ] Modify `Discovery.upsert_device/2` to detect changes - [ ] Implement `Firmware.log_firmware_change/3` - [ ] Add PubSub broadcast for firmware changes - [ ] Write integration tests for change detection - [ ] Test initial discovery (no history created) ### Phase 5: LiveView Integration - [ ] Add `available_firmware` assign to DeviceLive.Show - [ ] Implement `load_available_firmware/1` helper - [ ] Add firmware update indicator component to template - [ ] Add PubSub handler for `:firmware_changed` events - [ ] Write LiveView tests for indicator display - [ ] Test with different vendors (show for MikroTik, hide for others) ### Phase 6: Documentation & Cleanup - [ ] Update CLAUDE.md with firmware tracking info - [ ] Add developer documentation for adding new vendors - [ ] Run `mix format` on all new files - [ ] Run `mix dialyzer` and fix any warnings - [ ] Ensure test coverage >90% for new modules - [ ] Manual testing with real MikroTik device ## Future Enhancements 1. **Version History Display:** - Add "Firmware History" tab to device detail page - Show timeline of version changes with dates 2. **Firmware Update Notifications:** - Email digest of devices with available updates - Dashboard widget showing update summary 3. **Bulk Update Tracking:** - Mark multiple devices as "planned for update" - Track update completion across organization 4. **Multi-Vendor Support:** - Cisco IOS/IOS-XE (requires API or web scraping) - Ubiquiti UniFi (RSS or API) - Juniper JunOS (API) 5. **Release Notes Parsing:** - Extract changelog highlights from vendor sources - Display in-app without external link 6. **Smart Update Recommendations:** - Security bulletin awareness (CVE tracking) - Recommend updates based on severity - "Skip this version" for known problematic releases ## Security Considerations 1. **External HTTP Requests:** - RSS feeds are public, no authentication required - Use HTTPS for all vendor URLs - Set reasonable timeouts (5-10 seconds) - Rate limit to avoid overwhelming vendor servers 2. **Data Validation:** - Sanitize all parsed XML/JSON before storage - Validate version format before comparison - Prevent XML entity expansion attacks (XXE) 3. **PII/Sensitive Data:** - No sensitive data in firmware tracking - Download URLs are public vendor pages - Device firmware versions are operational metadata ## Performance Considerations 1. **RSS Fetching:** - Daily schedule minimizes external requests - Async worker doesn't block web requests - Failed fetches retry (Oban max_attempts: 3) 2. **LiveView Loading:** - Firmware lookup is single query by vendor/product_line - Indexed unique constraint makes lookup fast - Version comparison is in-memory (no DB query) 3. **Version History:** - Indexed on `snmp_device_id + detected_at` - Limit queries to recent N records (default 20) - Cascading delete prevents orphaned records ## Rollout Plan 1. **Deploy schema migrations** (production downtime: ~5 seconds) 2. **Deploy worker code** (no immediate execution) 3. **Manually trigger worker** to seed initial firmware data: `Oban.insert(Towerops.Workers.FirmwareVersionFetcherWorker.new(%{}))` 4. **Verify firmware release record** created 5. **Monitor cron execution** daily at 2am 6. **Enable LiveView indicator** after confirming data accuracy ## Monitoring & Alerts 1. **Worker Failures:** - Oban dashboard shows failed jobs - Email alert if worker fails 3 consecutive days - Log warning if RSS format changes unexpectedly 2. **Data Freshness:** - Alert if `firmware_releases.fetched_at` >48 hours old - Dashboard shows last successful fetch time 3. **Version Comparison Issues:** - Log warning for unparseable version strings - Store raw version in history for manual review