Add incident.io alert integration for Icinga2

- New notification script (incidentio-notification.sh) that POSTs JSON
  payloads to incident.io HTTP alert sources via curl/jq
- New Icinga2 config template defining User, NotificationCommand (host
  + service), and apply Notification rules — follows the PagerDuty
  pattern with opt-in via vars.enable_incidentio
- Notification type mapping: PROBLEM/DOWN/CRITICAL/WARNING/UNKNOWN →
  firing, RECOVERY/UP/OK → resolved, ACKNOWLEDGEMENT → firing with
  metadata
- Deduplication keys: host-{name} for hosts, service-{host}-{service}
  for services
- Deployed via Ansible: script copied to /etc/icinga2/scripts/,
  config rendered to conf.d/; gated on icinga2_incidentio_token being
  defined

2026-07-24 16:36:49 -05:00

2.3 KiB

Raw Blame History

monitor role

Builds an Icinga2 master + Icinga Web 2 stack matching the live install on monitor.vntx.net:

Ubuntu 22.04 + Apache 2.4 + mod_php + MariaDB 10.6 + Icinga 2.16
Single zone, master-only (no satellites configured but zones.conf is templated to add them later)
IDO-MySQL backend; icinga2 daemon writes status, icinga user read-only-selects for the Web 2 UI, icingaweb2 user owns the UI's own DB
25 dormant conf.d/* host + service files shipped under files/ and copied verbatim. The live install has include_recursive "conf.d" commented out (a node-setup CLI artifact); this role mirrors that. Set icinga2_load_confd: true in host_vars to rehydrate them.

Required vault variables

Encrypt these with ansible-vault (e.g. host_vars/monitor.vntx.net/vault.yml):

# Icinga API users (api-users.conf)
icinga2_api_user_root_password: "..."
icinga2_api_user_icingaweb2_password: "..."

# Icinga2 daemon → IDO database
icinga2_ido_db_password: "..."

# icingaweb2 UI → its own database
icingaweb2_db_password: "..."

# icingaweb2 UI → IDO database (separate read-only user)
icingaweb2_ido_db_password: "..."

# Constants
icinga2_iftraffic_snmp_community: "..."
icinga2_ticket_salt: ""        # only needed for distributed setups

# PagerDuty integration keys
icinga2_pagerduty_key: "..."
icinga2_pagerduty_servers_key: "..."

# incident.io alert integration token (Bearer token for the HTTP alert source)
icinga2_incidentio_token: "..."

# icingaweb2 admin user (web UI password). Generate the hash on any host
# with PHP installed:  php -r 'echo password_hash("plaintext", PASSWORD_DEFAULT);'
icingaweb2_admin_password_hash: "$2y$10$..."

What the role does NOT manage

Let's Encrypt certificate provisioning. The live install used certbot --apache; this role assumes the cert already exists at /etc/letsencrypt/live/<host>/. Run certbot once by hand.
The icingaweb2 schema is imported once on first install. Schema upgrades after Icinga Web 2 minor version bumps are not handled here — see /usr/share/icingaweb2/schema/upgrades/ and run the matching upgrade SQL by hand.
Distributed monitoring satellites. The role provisions a single-node master; add Endpoint/Zone objects in zones.d/ (or extend zones.conf.j2) when you wire satellites in.

2.3 KiB Raw Blame History

monitor role

Required vault variables

What the role does NOT manage

2.3 KiB

Raw Blame History