Graham McIntire e64df28906 fix: correct login URL path in markdown negotiation content

2026-04-17 15:50:31 -05:00

22 KiB

Raw Blame History

Agent Readiness Implementation Plan

For Claude: REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

Goal: Make towerops.net pass all isitagentready.com checks by adding sitemap, well-known endpoints, Link headers, markdown negotiation, and WebMCP support.

Architecture: Add a WellKnownController for all /.well-known/* JSON/stub endpoints, a SitemapController for /sitemap.xml, patch PageController for Link headers + markdown content negotiation, and add WebMCP JS to app.js. All endpoints are unauthenticated.

Tech Stack: Phoenix controllers, Elixir, plain JSON (no extra libraries), standard XML generation, vanilla JS in app.js.

Task 1: Sitemap XML + robots.txt

Files:

Create: lib/towerops_web/controllers/sitemap_controller.ex
Modify: lib/towerops_web/router.ex
Modify: priv/static/robots.txt
Test: test/towerops_web/controllers/sitemap_controller_test.exs

Step 1: Write the failing test

# test/towerops_web/controllers/sitemap_controller_test.exs
defmodule ToweropsWeb.SitemapControllerTest do
  use ToweropsWeb.ConnCase, async: true

  describe "GET /sitemap.xml" do
    test "returns valid XML with public pages", %{conn: conn} do
      conn = get(conn, "/sitemap.xml")

      assert response(conn, 200)
      assert get_resp_header(conn, "content-type") == ["text/xml; charset=utf-8"]
      body = response(conn, 200)
      assert body =~ ~r/<\?xml/
      assert body =~ "<urlset"
      assert body =~ "<loc>https://towerops.net/</loc>"
      assert body =~ "<loc>https://towerops.net/privacy</loc>"
      assert body =~ "<loc>https://towerops.net/terms</loc>"
    end
  end
end

Step 2: Run test to verify it fails

cd /Users/graham/dev/towerops/towerops-web
mix test test/towerops_web/controllers/sitemap_controller_test.exs

Expected: FAIL with no route/action error.

Step 3: Add the route to router.ex

In lib/towerops_web/router.ex, after the health check scope:

# Sitemap (no pipeline — public, no CSRF)
scope "/", ToweropsWeb do
  get "/sitemap.xml", SitemapController, :index
end

Step 4: Create the controller

# lib/towerops_web/controllers/sitemap_controller.ex
defmodule ToweropsWeb.SitemapController do
  @moduledoc false

  use ToweropsWeb, :controller

  @public_urls ["/", "/privacy", "/terms", "/docs/api", "/docs/graphql", "/help"]

  def index(conn, _params) do
    base_url = ToweropsWeb.Endpoint.url()

    xml = build_sitemap(base_url)

    conn
    |> put_resp_content_type("text/xml")
    |> send_resp(200, xml)
  end

  defp build_sitemap(base_url) do
    urls =
      @public_urls
      |> Enum.map(fn path ->
        "<url><loc>#{base_url}#{path}</loc></url>"
      end)
      |> Enum.join("\n  ")

    """
    <?xml version="1.0" encoding="UTF-8"?>
    <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
      #{urls}
    </urlset>
    """
    |> String.trim()
  end
end

Step 5: Update robots.txt

Replace priv/static/robots.txt content:

# See https://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
User-agent: *
Allow: /

Sitemap: https://towerops.net/sitemap.xml

Step 6: Run tests to verify they pass

mix test test/towerops_web/controllers/sitemap_controller_test.exs

Expected: All tests PASS.

Step 7: Commit

git add lib/towerops_web/controllers/sitemap_controller.ex \
        test/towerops_web/controllers/sitemap_controller_test.exs \
        lib/towerops_web/router.ex \
        priv/static/robots.txt
git commit -m "feat: add /sitemap.xml and update robots.txt for agent discovery"

Task 2: Well-Known JSON Endpoints

Serve all /.well-known/* discovery endpoints from a single controller.

Files:

Create: lib/towerops_web/controllers/well_known_controller.ex
Modify: lib/towerops_web/router.ex
Test: test/towerops_web/controllers/well_known_controller_test.exs

Step 1: Write the failing tests

# test/towerops_web/controllers/well_known_controller_test.exs
defmodule ToweropsWeb.WellKnownControllerTest do
  use ToweropsWeb.ConnCase, async: true

  describe "GET /.well-known/api-catalog" do
    test "returns linkset+json with API entries", %{conn: conn} do
      conn = get(conn, "/.well-known/api-catalog")
      assert response(conn, 200)
      assert get_resp_header(conn, "content-type") == ["application/linkset+json; charset=utf-8"]
      body = json_response(conn, 200)
      assert is_list(body["linkset"])
      [entry] = body["linkset"]
      assert entry["anchor"] =~ "/api/v1"
      assert Map.has_key?(entry, "service-doc")
    end
  end

  describe "GET /.well-known/openid-configuration" do
    test "returns OIDC metadata", %{conn: conn} do
      conn = get(conn, "/.well-known/openid-configuration")
      assert response(conn, 200)
      body = json_response(conn, 200)
      assert Map.has_key?(body, "issuer")
      assert Map.has_key?(body, "authorization_endpoint")
      assert Map.has_key?(body, "token_endpoint")
      assert Map.has_key?(body, "jwks_uri")
    end
  end

  describe "GET /.well-known/oauth-authorization-server" do
    test "returns OAuth 2.0 metadata", %{conn: conn} do
      conn = get(conn, "/.well-known/oauth-authorization-server")
      assert response(conn, 200)
      body = json_response(conn, 200)
      assert Map.has_key?(body, "issuer")
    end
  end

  describe "GET /.well-known/oauth-protected-resource" do
    test "returns protected resource metadata", %{conn: conn} do
      conn = get(conn, "/.well-known/oauth-protected-resource")
      assert response(conn, 200)
      body = json_response(conn, 200)
      assert Map.has_key?(body, "resource")
      assert Map.has_key?(body, "authorization_servers")
    end
  end

  describe "GET /.well-known/mcp/server-card.json" do
    test "returns MCP server card", %{conn: conn} do
      conn = get(conn, "/.well-known/mcp/server-card.json")
      assert response(conn, 200)
      body = json_response(conn, 200)
      assert get_in(body, ["serverInfo", "name"]) == "TowerOps"
      assert Map.has_key?(body, "transport")
    end
  end

  describe "GET /.well-known/agent-skills/index.json" do
    test "returns agent skills index", %{conn: conn} do
      conn = get(conn, "/.well-known/agent-skills/index.json")
      assert response(conn, 200)
      body = json_response(conn, 200)
      assert Map.has_key?(body, "$schema")
      assert is_list(body["skills"])
    end
  end
end

Step 2: Run tests to verify they fail

mix test test/towerops_web/controllers/well_known_controller_test.exs

Expected: All FAIL with no route error.

Step 3: Add routes to router.ex

After the sitemap scope, add a scope for well-known endpoints. Note the nested path for MCP:

# Well-known discovery endpoints (no pipeline — public, no CSRF)
scope "/.well-known", ToweropsWeb do
  get "/api-catalog", WellKnownController, :api_catalog
  get "/openid-configuration", WellKnownController, :openid_configuration
  get "/oauth-authorization-server", WellKnownController, :oauth_authorization_server
  get "/oauth-protected-resource", WellKnownController, :oauth_protected_resource
  get "/agent-skills/index.json", WellKnownController, :agent_skills
end

scope "/.well-known/mcp", ToweropsWeb do
  get "/server-card.json", WellKnownController, :mcp_server_card
end

Step 4: Create the controller

# lib/towerops_web/controllers/well_known_controller.ex
defmodule ToweropsWeb.WellKnownController do
  @moduledoc false

  use ToweropsWeb, :controller

  def api_catalog(conn, _params) do
    base_url = ToweropsWeb.Endpoint.url()

    conn
    |> put_resp_content_type("application/linkset+json")
    |> json(%{
      "linkset" => [
        %{
          "anchor" => "#{base_url}/api/v1",
          "service-doc" => [%{"href" => "#{base_url}/docs/api"}],
          "service-desc" => [
            %{"href" => "#{base_url}/docs/api", "type" => "text/html"}
          ],
          "status" => [%{"href" => "#{base_url}/health"}]
        }
      ]
    })
  end

  def openid_configuration(conn, _params) do
    base_url = ToweropsWeb.Endpoint.url()

    json(conn, %{
      "issuer" => base_url,
      "authorization_endpoint" => "#{base_url}/oauth/authorize",
      "token_endpoint" => "#{base_url}/oauth/token",
      "jwks_uri" => "#{base_url}/.well-known/jwks.json",
      "grant_types_supported" => ["authorization_code"],
      "response_types_supported" => ["code"]
    })
  end

  def oauth_authorization_server(conn, _params) do
    base_url = ToweropsWeb.Endpoint.url()

    json(conn, %{
      "issuer" => base_url,
      "authorization_endpoint" => "#{base_url}/oauth/authorize",
      "token_endpoint" => "#{base_url}/oauth/token",
      "jwks_uri" => "#{base_url}/.well-known/jwks.json",
      "grant_types_supported" => ["authorization_code"],
      "response_types_supported" => ["code"]
    })
  end

  def oauth_protected_resource(conn, _params) do
    base_url = ToweropsWeb.Endpoint.url()

    json(conn, %{
      "resource" => "#{base_url}/api/v1",
      "authorization_servers" => [base_url],
      "scopes_supported" => ["read", "write"]
    })
  end

  def mcp_server_card(conn, _params) do
    base_url = ToweropsWeb.Endpoint.url()

    json(conn, %{
      "serverInfo" => %{
        "name" => "TowerOps",
        "version" => "1.0.0"
      },
      "transport" => %{
        "endpoint" => "#{base_url}/mcp"
      },
      "capabilities" => %{
        "tools" => %{},
        "resources" => %{},
        "prompts" => %{}
      }
    })
  end

  def agent_skills(conn, _params) do
    json(conn, %{
      "$schema" => "https://schemas.agentskills.io/discovery/0.2.0/schema.json",
      "skills" => []
    })
  end
end

Step 5: Run tests

mix test test/towerops_web/controllers/well_known_controller_test.exs

Expected: All PASS.

Step 6: Commit

git add lib/towerops_web/controllers/well_known_controller.ex \
        test/towerops_web/controllers/well_known_controller_test.exs \
        lib/towerops_web/router.ex
git commit -m "feat: add /.well-known discovery endpoints for agent readiness"

Task 3: Link Response Headers on Homepage (RFC 8288)

Files:

Modify: lib/towerops_web/controllers/page_controller.ex
Modify: test/towerops_web/controllers/page_controller_test.exs

Step 1: Write the failing test

Add to the existing page_controller_test.exs (find the GET / section and add):

describe "GET / link headers" do
  test "homepage includes RFC 8288 Link headers", %{conn: conn} do
    conn = get(conn, ~p"/")
    link_headers = get_resp_header(conn, "link")
    assert Enum.any?(link_headers, &String.contains?(&1, ~s(rel="api-catalog")))
    assert Enum.any?(link_headers, &String.contains?(&1, ~s(rel="service-doc")))
  end
end

Step 2: Run tests to verify they fail

mix test test/towerops_web/controllers/page_controller_test.exs

Expected: New test FAILS (no Link headers on current response).

Step 3: Update PageController

Modify home/2 in lib/towerops_web/controllers/page_controller.ex:

def home(conn, _params) do
  conn = put_link_headers(conn)

  if conn.assigns.current_scope && conn.assigns.current_scope.user do
    redirect(conn, to: ~p"/dashboard")
  else
    render(conn, :home, layout: false)
  end
end

defp put_link_headers(conn) do
  conn
  |> put_resp_header("link", ~s(</.well-known/api-catalog>; rel="api-catalog"))
  |> put_resp_header("link", ~s(</docs/api>; rel="service-doc"))
  |> put_resp_header("link", ~s(</health>; rel="status"))
end

Note: Phoenix put_resp_header/3 replaces a header. To emit multiple Link headers, use Plug.Conn.put_resp_header/3 only once with a comma-joined value instead:

defp put_link_headers(conn) do
  link_value =
    [
      ~s(</.well-known/api-catalog>; rel="api-catalog"),
      ~s(</docs/api>; rel="service-doc"),
      ~s(</health>; rel="status")
    ]
    |> Enum.join(", ")

  put_resp_header(conn, "link", link_value)
end

Step 4: Run tests

mix test test/towerops_web/controllers/page_controller_test.exs

Expected: All PASS.

Step 5: Commit

git add lib/towerops_web/controllers/page_controller.ex \
        test/towerops_web/controllers/page_controller_test.exs
git commit -m "feat: add RFC 8288 Link response headers to homepage"

Task 4: Markdown Content Negotiation

When a request has Accept: text/markdown, return a markdown representation of the page with Content-Type: text/markdown.

Files:

Modify: lib/towerops_web/controllers/page_controller.ex
Create: lib/towerops_web/controllers/page_html/home.md.eex (or inline in controller)
Test: test/towerops_web/controllers/page_controller_test.exs

Step 1: Write the failing test

Add to page_controller_test.exs:

describe "GET / markdown negotiation" do
  test "returns markdown when Accept: text/markdown", %{conn: conn} do
    conn =
      conn
      |> put_req_header("accept", "text/markdown")
      |> get(~p"/")

    assert response(conn, 200)
    assert get_resp_header(conn, "content-type") == ["text/markdown; charset=utf-8"]
    body = response(conn, 200)
    assert body =~ "# TowerOps"
  end
end

Step 2: Run to verify it fails

mix test test/towerops_web/controllers/page_controller_test.exs

Expected: FAIL — response returns HTML, not markdown.

Step 3: Update PageController

Note: The browser pipeline has plug :accepts, ["html"] which will reject non-HTML requests before they reach the controller. To work around this, check the accept header before format negotiation, or add the markdown check early.

The cleanest approach: add "md" to accepted formats and register the MIME type. But the simpler approach is to check the header directly at the top of home/2 before the pipeline can reject it. Actually, the :accepts plug runs in the pipeline before the controller. We need a different route.

Solution: Add a separate scope for markdown negotiation that bypasses the html-only browser pipeline:

In router.ex, add before the browser-pipeline scope:

# Markdown content negotiation (must come before :browser scope)
pipeline :markdown do
  plug :accepts, ["html", "text/markdown"]
  plug :fetch_session
  plug :fetch_live_flash
  plug :put_root_layout, html: {ToweropsWeb.Layouts, :root}
  plug :protect_from_forgery
  plug :put_secure_browser_headers, %{}
  plug ToweropsWeb.Plugs.CaptureTimezone
  plug :fetch_current_scope_for_user
  plug :set_user_locale
  plug ToweropsWeb.Plugs.UpdateSessionActivity
  plug :store_return_to_for_liveview
  plug ToweropsWeb.Plugs.DetectEUUser
  plug ToweropsWeb.Plugs.CheckPolicyConsent
end

Actually, the simplest approach is to not use a pipeline at all for the markdown route, and instead add a dedicated action that handles markdown and sits outside the :accepts restriction. We can match on a different path...

Actually simplest: use the :accepts plug but modify the browser pipeline to also accept "text/markdown" mapped to a "md" format, then check format in the controller.

Even simpler: Add the markdown check before the :accepts plug runs, via a custom plug or by checking get_req_header in an early plug.

Recommended approach: Create a ToweropsWeb.Plugs.MarkdownNegotiation plug that runs early (before :accepts), detects Accept: text/markdown, intercepts only the homepage, and returns markdown directly.

# lib/towerops_web/plugs/markdown_negotiation.ex
defmodule ToweropsWeb.Plugs.MarkdownNegotiation do
  @moduledoc false
  import Plug.Conn

  @markdown_pages %{
    "/" => &__MODULE__.home_markdown/0,
    "/privacy" => &__MODULE__.privacy_markdown/0,
    "/terms" => &__MODULE__.terms_markdown/0
  }

  def init(opts), do: opts

  def call(conn, _opts) do
    if accepts_markdown?(conn) && Map.has_key?(@markdown_pages, conn.request_path) do
      markdown_fn = @markdown_pages[conn.request_path]
      markdown = markdown_fn.()

      conn
      |> put_resp_content_type("text/markdown")
      |> send_resp(200, markdown)
      |> halt()
    else
      conn
    end
  end

  defp accepts_markdown?(conn) do
    conn
    |> get_req_header("accept")
    |> Enum.any?(&String.contains?(&1, "text/markdown"))
  end

  def home_markdown do
    """
    # TowerOps

    Network monitoring and alerting platform built for WISP and ISP operators.

    See the business impact of every network event.

    ## Features

    - Real-time network monitoring
    - Subscriber impact analysis
    - Revenue impact tracking
    - Equipment management
    - Multi-site support

    ## Get Started

    - [Register](/users/register)
    - [Log In](/users/log_in)
    - [API Documentation](/docs/api)
    - [Privacy Policy](/privacy)
    - [Terms of Service](/terms)

    ## API

    TowerOps provides a REST API for programmatic access.
    See [API documentation](/docs/api) for details.

    ## Contact

    Visit [towerops.net](https://towerops.net) for more information.
    """
  end

  def privacy_markdown do
    "# Privacy Policy\n\nSee [towerops.net/privacy](https://towerops.net/privacy) for the full privacy policy.\n"
  end

  def terms_markdown do
    "# Terms of Service\n\nSee [towerops.net/terms](https://towerops.net/terms) for the full terms of service.\n"
  end
end

Add this plug to the endpoint (lib/towerops_web/endpoint.ex) before the router, so it runs before the :accepts pipeline check:

# Add before plug ToweropsWeb.Router
plug ToweropsWeb.Plugs.MarkdownNegotiation
plug ToweropsWeb.Router

Step 4: Update the test

The test in page_controller_test.exs will work with ConnCase which bypasses the endpoint. Modify the test to use build_conn() and the endpoint dispatch:

Actually in ConnCase, tests go through the full router pipeline. The plug being at the endpoint level means it runs before the router, but ConnCase dispatches through ToweropsWeb.Endpoint. Verify with:

describe "GET / markdown negotiation" do
  test "returns markdown when Accept: text/markdown", %{conn: conn} do
    conn =
      conn
      |> put_req_header("accept", "text/markdown")
      |> get(~p"/")

    assert response(conn, 200)
    assert get_resp_header(conn, "content-type") == ["text/markdown; charset=utf-8"]
    body = response(conn, 200)
    assert body =~ "# TowerOps"
  end
end

Step 5: Run tests

mix test test/towerops_web/controllers/page_controller_test.exs

Expected: All PASS including the markdown test.

Step 6: Commit

git add lib/towerops_web/plugs/markdown_negotiation.ex \
        lib/towerops_web/endpoint.ex \
        test/towerops_web/controllers/page_controller_test.exs
git commit -m "feat: add markdown content negotiation for agent compatibility"

Task 5: WebMCP JavaScript

Expose site tools to AI agents via navigator.modelContext.provideContext() on page load.

Files:

Modify: assets/js/app.js
Modify: test/towerops_web/controllers/page_html_test.exs (or create a JS test if preferred)

Since we cannot write inline <script> tags in templates, add WebMCP initialization to assets/js/app.js. The navigator.modelContext API is browser-experimental; guard against missing support.

Step 1: Append to assets/js/app.js

Find the end of assets/js/app.js and add:

// WebMCP: expose site tools to AI agents (https://webmachinelearning.github.io/webmcp/)
if (typeof navigator !== "undefined" && navigator.modelContext) {
  navigator.modelContext.provideContext({
    tools: [
      {
        name: "navigate",
        description: "Navigate to a page on TowerOps",
        inputSchema: {
          type: "object",
          properties: {
            path: {
              type: "string",
              description: "The path to navigate to, e.g. /dashboard or /users/register"
            }
          },
          required: ["path"]
        },
        execute({ path }) {
          window.location.href = path;
        }
      },
      {
        name: "search_docs",
        description: "Open the TowerOps API documentation",
        inputSchema: {
          type: "object",
          properties: {}
        },
        execute() {
          window.location.href = "/docs/api";
        }
      },
      {
        name: "register",
        description: "Start the registration flow for a new TowerOps account",
        inputSchema: {
          type: "object",
          properties: {}
        },
        execute() {
          window.location.href = "/users/register";
        }
      }
    ]
  });
}

Step 2: Verify it compiles

mix assets.build

Expected: No JS errors.

Step 3: Run full test suite to ensure nothing broken

mix test

Expected: All PASS.

Step 4: Commit

git add assets/js/app.js
git commit -m "feat: add WebMCP tool definitions for AI agent browser interaction"

Task 6: Run precommit and fix any issues

Step 1: Run precommit

mix precommit

Expected: Compiles, formats, all tests pass.

If any Credo issues: fix them and run again.

Step 2: Final commit if any format fixes

git add -p  # stage only format changes
git commit -m "chore: run mix format"

Verification Checklist

After all tasks complete, these endpoints must all return HTTP 200:

GET /sitemap.xml → text/xml, lists canonical URLs
GET /.well-known/api-catalog → application/linkset+json
GET /.well-known/openid-configuration → JSON with issuer
GET /.well-known/oauth-authorization-server → JSON with issuer
GET /.well-known/oauth-protected-resource → JSON with resource
GET /.well-known/mcp/server-card.json → JSON with serverInfo
GET /.well-known/agent-skills/index.json → JSON with $schema
GET / → includes Link header with rel="api-catalog"
GET / with Accept: text/markdown → text/markdown response
priv/static/robots.txt → contains Sitemap: https://towerops.net/sitemap.xml

22 KiB Raw Blame History

Agent Readiness Implementation Plan

Task 1: Sitemap XML + robots.txt

Task 2: Well-Known JSON Endpoints

Task 3: Link Response Headers on Homepage (RFC 8288)

Task 4: Markdown Content Negotiation

Task 5: WebMCP JavaScript

Task 6: Run precommit and fix any issues

Verification Checklist

22 KiB

Raw Blame History