- New notification script (incidentio-notification.sh) that POSTs JSON
payloads to incident.io HTTP alert sources via curl/jq
- New Icinga2 config template defining User, NotificationCommand (host
+ service), and apply Notification rules — follows the PagerDuty
pattern with opt-in via vars.enable_incidentio
- Notification type mapping: PROBLEM/DOWN/CRITICAL/WARNING/UNKNOWN →
firing, RECOVERY/UP/OK → resolved, ACKNOWLEDGEMENT → firing with
metadata
- Deduplication keys: host-{name} for hosts, service-{host}-{service}
for services
- Deployed via Ansible: script copied to /etc/icinga2/scripts/,
config rendered to conf.d/; gated on icinga2_incidentio_token being
defined
Add an RPZ blocklist on the Unbound resolvers that returns NXDOMAIN for a
set of domains (and subdomains) and logs only those matches under rpz-log
tag "blocklist". Domains are templated from resolver_blocked_domains
(single source of truth) into /etc/unbound/blocklist.rpz; respip module
enabled for RPZ support.
Fix deprecation warnings across the Ansible tree:
- apt_repository -> deb822_repository (monitor, aprsc, uisp, grafana),
removing stale .list files and adding update_cache where deb822 drops it
- community.mysql.* -> ansible.mysql.* (monitor) and requirements.yml
- top-level facts -> ansible_facts[...] (ansible_os_family,
distribution_release, fqdn, architecture, default_ipv4) repo-wide
- Convert ~290 short module names to FQCNs across all roles
- Change become: yes/no to become: true/false, gather_facts: no to false
- Remove deprecated inject_facts_as_vars from ansible.cfg
- Add missing task names to import_tasks/include_role calls
- Add become: yes to librenms clone task (partial-become fix)
- Add pipefail to risky shell command in udp-gro-fix.yml
- Quote all octal mode values
- Harden .ansible-lint with production profile and empty skip_list
- Update findings.md with all fixes
Codifies the bare-metal Icinga2 master + Icinga Web 2 stack on
monitor.vntx.net (Ubuntu 22.04, Apache + mod_php, MariaDB 10.6, Icinga
2.16, IDO-MySQL backend). Re-running the role against the live host
should produce a near-identical configuration.
Role includes:
- Icinga apt repo + full package set (icinga2, ido-mysql, icingaweb2,
monitoring-plugins, nagios-plugins-contrib, php + apache deps).
- MariaDB DB + user provisioning for both icinga2 (IDO daemon) and
icingaweb2 (UI), plus a separate read-only `icinga` user that the UI
uses to read the IDO. Schema bootstrapped only on first install.
- Templated icinga2 config (constants/zones/icinga2.conf), features-
enabled symlinks (api/checker/command/ido-mysql/mainlog), and the
three secret-bearing conf.d files (api-users, pagerduty x2) wired to
vault vars.
- 23 dormant conf.d/* host + service definitions copied verbatim under
files/. The live install has `include_recursive "conf.d"` commented
out (a node-setup CLI artifact); role mirrors that. Set
`icinga2_load_confd: true` in host_vars to rehydrate them.
- Icinga Web 2 ini files (config/auth/groups/roles/modules) + templated
resources.ini for DB credentials. monitoring module enabled via
symlink. icingaweb2 admin user bootstrapped on first install via a
one-shot insert into icingaweb_user (gated on schema-not-yet-imported).
- Apache rewrite + ssl modules. Vhost itself is left to certbot --apache
(Let's Encrypt cert is operator-managed, role doesn't manage it).
Cluster-wide cleanup:
- Gut roles/general/tasks/debian/icinga2.yml down to the uninstall
safety net (kept the sweep that removes icinga2 from non-monitoring
hosts). The previous master-side install + cert-distribution tasks
referenced templates that never existed in the role; the new monitor
role replaces them properly.
Operator action before first apply:
- Create host_vars/monitor.vntx.net/vault.yml with the seven required
secrets (see roles/monitor/README.md). Live values can be pulled from
/etc/icinga2/conf.d/{api-users,pagerduty-*,ido-mysql}.conf and
/etc/icingaweb2/resources.ini on the live host.
- Generate the icingaweb2 admin password hash via
`php -r 'echo password_hash("plaintext", PASSWORD_DEFAULT);'`.
- Run --check --diff before applying; the role is meant to be idempotent
but hasn't been verified against the live install.
