Require meaningful equality labels in Alertmanager inhibition rules

Two absent labels can match each other and suppress more notifications than intended.

Source facts

Alertmanager inhibition requires the source and target alerts to share values for labels listed in equal. Its configuration documentation treats an absent label and an empty-valued label as equivalent. If all equality labels are missing from both sides, inhibition can apply. The guidance also recommends source and target matchers that avoid an alert matching both sides.

Source: Prometheus documentation — Configuration.

Applicability

Review Alertmanager inhibition equality labels and source/target matchers before enabling suppression.

DSE recommendation

DSE recommends selecting equality labels that reliably identify the same site, cluster, or service boundary and verifying that every relevant alert supplies them. Keep the causal explanation with the inhibition rule. Do not use an optional label as the sole guard against cross-customer or cross-site suppression. Review newly introduced alert rules against existing inhibition because a missing label can change notification behavior without editing the suppression rule itself. Assign an owner to investigate suppressed alerts during major incidents.

Verification

Using synthetic alerts, test matching values, different values, empty values, and labels missing on one or both sides. Inspect which target notifications are inhibited while the source is active and after it clears. Confirm the result stays inside the intended operational boundary before enabling the rule. Retain the exact source and target label sets with each expected notification result. Have the incident owner approve the missing-label cases explicitly and identify the rule change required if suppression crosses the chosen site or service boundary.

Official reference

Prometheus documentation — Configuration. Reviewed October 20, 2026.