resolvers: serve DoH via Caddy, add IPv6 records and PQ SSH

- unbound: add loopback DoH listener (127.0.0.1@8080, http-notls-downstream);
  interface-automatic-ports lists 8080 so interface-automatic does not silently
  drop the non-53 listener; access-control driven by resolver_allowed_netblocks
- caddy: add RedHat/COPR install path, caddy_template var, /var/log/caddy dir;
  new Caddyfile-resolver.j2 terminates LE TLS and h2c-proxies to unbound,
  enforcing the client allow-list via remote_ip
- resolvers: open 80/443 in firewall; prefer post-quantum SSH key exchange
  (sntrup761x25519-sha512) via validated sshd_config.d drop-in
- bind9: zone.j2 emits AAAA records; add ipv6 for resolver1/2 (2606:1c80::240/250)
- playbook: run caddy role on the resolvers group

ansible/docs/plans/2026-05-14-resolver-doh-design.md

@@ -0,0 +1,72 @@
+# DoH on resolver1/resolver2 with Caddy-managed TLS
+
+**Date:** 2026-05-14
+**Status:** In Progress
+
+## Goal
+
+Serve DNS-over-HTTPS (RFC 8484) from `resolver1.vntx.net` and `resolver2.vntx.net`,
+with Let's Encrypt certificate lifecycle handled by Caddy. Also add AAAA records for
+the resolvers so they are reachable over IPv6.
+
+## Architecture
+
+Each resolver host runs two cooperating services:
+
+```
+        Internet (DoH client)
+              | HTTPS :443  (resolverN.vntx.net/dns-query)
+              v
+        Caddy (new on host)        <- obtains/renews LE cert via HTTP-01 on :80
+              | HTTP/2 cleartext (h2c), 127.0.0.1:8080/dns-query
+              v
+        Unbound (existing)         :53 plain DNS (unchanged)
+                                   :8080 DoH, http-notls-downstream
+```
+
+- Caddy owns :443 and :80, manages the LE cert for `resolverN.vntx.net`.
+- Unbound serves DoH as cleartext HTTP/2 on loopback only — no certs touch Unbound.
+- Plain DNS on :53 is untouched; this change is purely additive.
+
+## Access control
+
+DoH clients reach Caddy, which forwards from 127.0.0.1 — so Unbound's source-IP
+`access-control` cannot see the real client. Caddy therefore enforces the client
+allow-list with a `remote_ip` matcher; non-allowed clients get 403. The allow-list
+lives in one place: `resolver_allowed_netblocks` in `group_vars/resolvers/main.yml`,
+consumed by both the Caddy matcher and Unbound's `access-control` (for :53).
+
+## Changes
+
+### IPv6 / DNS
+- `roles/ns/templates/zone.j2` — emit AAAA records when a host has `ipv6:`.
+- `group_vars/bind9_servers/vntx.net.yml` — add `ipv6:` to resolver1/2 + resolver-01/02
+  (`2606:1c80::240`, `2606:1c80::250`).
+
+### Resolver group vars
+- `group_vars/resolvers/main.yml` — add `resolver_allowed_netblocks`, add 80/443 to
+  `firewall_allow_rules`, set `caddy_template: Caddyfile-resolver.j2`.
+
+### Unbound
+- `roles/resolvers/templates/unbound.conf.j2` — add loopback DoH listener
+  (`interface: 127.0.0.1@8080`, `https-port: 8080`, `http-endpoint`,
+  `http-notls-downstream: yes`); drive `access-control` from
+  `resolver_allowed_netblocks`.
+
+### Caddy role
+- `roles/caddy/tasks/main.yml` — add RedHat (COPR) install path; support a
+  `caddy_template` variable; ensure `/var/log/caddy` exists.
+- `roles/caddy/templates/Caddyfile-resolver.j2` — new shared template for resolvers.
+
+### Playbook
+- `playbook.yml` — add the `caddy` role to the `resolvers` play.
+
+## Verification
+
+- `ansible-playbook --syntax-check playbook.yml`
+- After apply: `unbound -V | grep nghttp2` (DoH build support), `ss -tlnp` shows
+  unbound on 127.0.0.1:8080 and caddy on :443/:80.
+- `curl -H 'accept: application/dns-message' https://resolver1.vntx.net/dns-query?dns=...`
+  from an allowed netblock; 403 from outside.
+- `dig AAAA resolver1.vntx.net` returns `2606:1c80::240`.
+</invoke>

ansible/group_vars/bind9_servers/vntx.net.yml

@@ -44,12 +44,16 @@ vntx_net_zone:
       ip: 204.110.191.225
     - name: resolver-01
       ip: 204.110.191.240
+      ipv6: 2606:1c80::240
     - name: resolver-02
       ip: 204.110.191.250
+      ipv6: 2606:1c80::250
     - name: resolver1
       ip: 204.110.191.240
+      ipv6: 2606:1c80::240
     - name: resolver2
       ip: 204.110.191.250
+      ipv6: 2606:1c80::250
     - name: unimus
       ip: 204.110.191.238
     - name: verona

ansible/group_vars/resolvers/main.yml

@@ -1,4 +1,16 @@
 ---
+# Netblocks permitted to use the resolvers (plain DNS access-control and DoH).
+# Single source of truth: consumed by unbound.conf.j2 and Caddyfile-resolver.j2.
+resolver_allowed_netblocks:
+  - 127.0.0.0/8
+  - 204.110.188.0/22
+  - 10.0.0.0/8
+  - 172.1.0.0/15
+  - 100.64.0.0/10
+  - 216.180.128.0/20
+  - ::1
+  - 2606:1c80::/32
+
 firewall_allow_rules:
   - port: 53
     proto: udp
@@ -6,3 +18,12 @@ firewall_allow_rules:
   - port: 53
     proto: tcp
     comment: DNS
+  - port: 80
+    proto: tcp
+    comment: HTTP (Let's Encrypt HTTP-01 challenge)
+  - port: 443
+    proto: tcp
+    comment: HTTPS (DNS-over-HTTPS)
+
+# Use the shared resolver Caddyfile instead of the global default.
+caddy_template: Caddyfile-resolver.j2

ansible/playbook.yml

@@ -32,6 +32,7 @@
     - resolvers
   roles:
     - resolvers
+    - caddy
 
 - name: Install and configure NetBox
   hosts: netbox_servers

ansible/roles/caddy/tasks/main.yml

@@ -1,5 +1,5 @@
 ---
-# Debian/Ubuntu tasks
+# --- Install: Debian/Ubuntu ---
 - name: Check if Caddy is already installed
   command: which caddy
   register: caddy_installed
@@ -33,7 +33,7 @@
     - /etc/apt/sources.list.d/caddy-stable.list
     - /etc/apt/sources.list.d/caddy.list
     - /usr/share/keyrings/caddy-stable-archive-keyring.gpg
-  when: 
+  when:
     - ansible_os_family == "Debian"
     - old_keyring.stat.exists | default(false)
   notify: update apt cache
@@ -71,18 +71,57 @@
     - ansible_os_family == "Debian"
     - caddy_installed.rc != 0
 
+# --- Install: RedHat/AlmaLinux ---
+- name: Install dnf copr plugin (RedHat)
+  ansible.builtin.dnf:
+    name: dnf-plugins-core
+    state: present
+  when: ansible_os_family == "RedHat"
+
+- name: Enable Caddy COPR repository (RedHat)
+  ansible.builtin.command:
+    cmd: dnf -y copr enable @caddy/caddy
+    creates: /etc/yum.repos.d/_copr:copr.fedorainfracloud.org:group_caddy:caddy.repo
+  when: ansible_os_family == "RedHat"
+
+- name: Install Caddy (RedHat)
+  ansible.builtin.dnf:
+    name: caddy
+    state: present
+  when: ansible_os_family == "RedHat"
+
+# --- Common configuration ---
+- name: Ensure Caddy log directory exists
+  ansible.builtin.file:
+    path: /var/log/caddy
+    state: directory
+    owner: "{{ caddy_user }}"
+    group: "{{ caddy_group }}"
+    mode: '0755'
+
 - name: Ensure Caddy service is enabled and started
   service:
     name: caddy
     state: started
     enabled: true
 
+- name: Configure Caddy (shared template via caddy_template)
+  template:
+    src: "{{ caddy_template }}"
+    dest: /etc/caddy/Caddyfile
+    owner: root
+    group: root
+    mode: '0644'
+  notify: restart caddy
+  when: caddy_template is defined
+
 - name: Check if host-specific Caddyfile template exists
   stat:
     path: "{{ playbook_dir }}/roles/caddy/templates/Caddyfile-{{ inventory_hostname }}.j2"
   register: host_specific_template
   delegate_to: localhost
   become: false
+  when: caddy_template is not defined
 
 - name: Configure Caddy (host-specific template)
   template:
@@ -92,7 +131,9 @@
     group: root
     mode: '0644'
   notify: restart caddy
-  when: host_specific_template.stat.exists
+  when:
+    - caddy_template is not defined
+    - host_specific_template.stat.exists | default(false)
 
 - name: Configure Caddy (default template)
   template:
@@ -102,4 +143,6 @@
     group: root
     mode: '0644'
   notify: restart caddy
-  when: not host_specific_template.stat.exists
+  when:
+    - caddy_template is not defined
+    - not (host_specific_template.stat.exists | default(false))

ansible/roles/caddy/templates/Caddyfile-resolver.j2

@@ -0,0 +1,35 @@
+# Managed by Ansible (roles/caddy) - shared by the [resolvers] group.
+# Terminates TLS for the DNS-over-HTTPS endpoint and reverse-proxies to the
+# local Unbound DoH listener (cleartext HTTP/2 on 127.0.0.1:8080).
+{
+    admin off
+    persist_config off
+}
+
+{{ inventory_hostname }} {
+    # Restrict DoH to the same netblocks Unbound allows for plain DNS.
+    # Caddy forwards from 127.0.0.1, so Unbound cannot see the real client;
+    # the allow-list is enforced here instead.
+    @allowed remote_ip {% for nb in resolver_allowed_netblocks %}{{ nb }} {% endfor %}
+
+
+    handle /dns-query {
+        handle @allowed {
+            reverse_proxy 127.0.0.1:8080 {
+                transport http {
+                    versions h2c 2
+                }
+            }
+        }
+        respond "Forbidden" 403
+    }
+
+    handle {
+        respond "DNS-over-HTTPS endpoint: /dns-query" 200
+    }
+
+    log {
+        output file /var/log/caddy/{{ inventory_hostname }}.log
+        format json
+    }
+}

ansible/roles/ns/templates/zone.j2

@@ -28,6 +28,13 @@ $TTL 604800
 {% endfor %}
 {% endif %}
 
+; AAAA Records
+{% if item.hosts is defined %}
+{% for host in item.hosts if host.ipv6 is defined %}
+{{ host.name }}    IN      AAAA    {{ host.ipv6 }}
+{% endfor %}
+{% endif %}
+
 ; TXT Records
 {% if item.text is defined %}
 {% for txt in item.text %}

ansible/roles/resolvers/tasks/main.yml

@@ -53,3 +53,41 @@
   when: ansible_os_family == "RedHat"
   notify:
     - Restart firewalld
+
+# EL9's crypto-policy default KexAlgorithms omits the post-quantum
+# sntrup761x25519-sha512@openssh.com (the binary supports it). This drop-in
+# prepends it (^ = place at head of default set) so it is preferred,
+# mitigating "store now, decrypt later" attacks. Validated before restart;
+# reverted if sshd rejects it, so a broken config can never lock out SSH.
+- name: Prefer post-quantum SSH key exchange
+  block:
+    - name: Install post-quantum SSH key-exchange drop-in
+      ansible.builtin.copy:
+        dest: /etc/ssh/sshd_config.d/10-postquantum.conf
+        content: |
+          # Managed by Ansible (roles/resolvers) - prefer post-quantum key exchange
+          KexAlgorithms ^sntrup761x25519-sha512@openssh.com
+        owner: root
+        group: root
+        mode: '0600'
+      register: pq_sshd_dropin
+
+    - name: Validate sshd configuration
+      ansible.builtin.command: sshd -t
+      changed_when: false
+
+    - name: Restart sshd to apply key-exchange change
+      ansible.builtin.systemd:
+        name: sshd
+        state: restarted
+      when: pq_sshd_dropin is changed
+  rescue:
+    - name: Revert post-quantum drop-in after failed validation
+      ansible.builtin.file:
+        path: /etc/ssh/sshd_config.d/10-postquantum.conf
+        state: absent
+    - name: Fail because sshd rejected the post-quantum drop-in
+      ansible.builtin.fail:
+        msg: >-
+          sshd -t rejected the post-quantum KexAlgorithms drop-in; it has been
+          removed and sshd was not restarted. Existing config is untouched.

ansible/roles/resolvers/templates/unbound.conf.j2

@@ -61,10 +61,22 @@ server:
 	interface: 0.0.0.0
 	interface: ::0
 
+	# DNS-over-HTTPS endpoint. Cleartext HTTP/2 on loopback only; Caddy
+	# terminates TLS on :443 and reverse-proxies here.
+	interface: 127.0.0.1@8080
+	https-port: 8080
+	http-endpoint: "/dns-query"
+	http-notls-downstream: yes
+
 	# enable this feature to copy the source address of queries to reply.
 	# Socket options are not supported on all platforms. experimental.
 	interface-automatic: yes
 
+	# interface-automatic ignores manually-configured interface: lines for
+	# non-default ports, so the DoH listener port (8080) must be listed here
+	# or the 127.0.0.1@8080 socket is silently never bound.
+	interface-automatic-ports: "53 8080"
+
 	# instead of the default port, open additional ports separated by
 	# spaces when interface-automatic is enabled, by listing them here.
 	# interface-automatic-ports: "53,443"
@@ -321,15 +333,10 @@ server:
 	# access-control: ::1 allow
 	# access-control: ::ffff:127.0.0.1 allow
     access-control: 0.0.0.0/0 refuse
-    access-control: 127.0.0.0/8 allow
-    access-control: 204.110.188.0/22 allow
-    access-control: 10.0.0.0/8 allow
-    access-control: 172.1.0.0/15 allow
-    access-control: 100.64.0.0/10 allow
-    access-control: 216.180.128.0/20 allow
     access-control: ::0/0 refuse
-    access-control: ::1 allow
-    access-control: 2606:1c80::/32 allow
+{% for netblock in resolver_allowed_netblocks %}
+    access-control: {{ netblock }} allow
+{% endfor %}
 
 	# tag access-control with list of tags (in "" with spaces between)
 	# Clients using this access control element use localzones that