kubernetes - 如何使用 Prometheus Operator 将 PrometheusRule 与 AlertmanagerConfig 匹配

Question

我有多个prometheusRules(rule a, rule b) ，并且每个规则定义了不同的exp来约束警报；那么，我有不同的AlertmanagerConfig (一个接收器是松弛的，那么另一个接收器是 opsgenie )；我们如何在规则和alertmanagerconfig之间建立联系？例如:如果规则a被触发，我想发送消息到slack；如果规则b被触发，我想发送消息到opsgenie .

这是我尝试过的方法，但是不起作用。我错过了什么吗？

这是prometheuisRule文件

apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  labels:
    prometheus: service-prometheus
    role: alert-rules
    app: kube-prometheus-stack
    release: monitoring-prom
  name: rule_a
  namespace: monitoring
spec:
  groups:
    - name: rule_a_alert
      rules:
        - alert: usage_exceed
          expr: salesforce_api_usage > 100000
          labels:
            severity: urgent

这是alertManagerConfig

apiVersion: monitoring.coreos.com/v1alpha1
kind: AlertmanagerConfig
metadata:
  labels:
    alertmanagerConfig: slack
  name: slack
  namespace: monitoring
  resourceVersion: "25842935"
  selfLink: /apis/monitoring.coreos.com/v1alpha1/namespaces/monitoring/alertmanagerconfigs/opsgenie-and-slack
  uid: fbb74924-5186-4929-b363-8c056e401921
spec:
  receivers:
  - name: slack-receiver
    slackConfigs:
    - apiURL:
        key: apiURL
        name: slack-config
  route:
    groupBy:
    - job
    groupInterval: 60s
    groupWait: 60s
    receiver: slack-receiver
    repeatInterval: 1m
    routes:
    - matchers:
      - name: job
        value: service_a
      receiver: slack-receiver

Answer 1

您需要匹配警报的标签，在您的情况下，您尝试将标签 job 与不存在的值 service_a 进行匹配。您可以通过更改alertManagerConfig文件中的match来匹配prometheuisRule文件中确实存在的标签，例如severity:

route:
  routes:
  - match:
      severity: urgent
    receiver: slack-receiver

或者您可以向 prometheuisRule 文件添加另一个标签:

spec:
  groups:
    - name: rule_a_alert
      rules:
        - alert: usage_exceed
          expr: salesforce_api_usage > 100000
          labels:
            severity: urgent
            job: service_a