elasticsearch - Metricbeat Beat无法连接es(已启用xpack)

Question

Helm版本:“v3.1.1”

helm get release输出

Helm 状态度量
名称:metricbeat
最后部署:2020年4月30日星期四10:05:32
NAMESPACE:默认
状态:已部署
修订版:1

描述错误:无法连接到elasticsearch，部署的configmap不包含用户名:和密码输入。

重现步骤:

在elasticsearch中启用xpack

运行- Helm 安装metricbeat elastic / metricbeat --set imageTag = 7.6.2 --valuesmetrics.yaml

metrics.yaml

daemonset:
  extraEnvs:
    - name: 'ES_USERNAME'
      valueFrom:
        secretKeyRef:
          name: elastic-credentials
          key: username
    - name: 'ES_PASSWORD'
      valueFrom:
        secretKeyRef:
          name: elastic-credentials
          key: password
  # Allows you to add any config files in /usr/share/metricbeat
  # such as metricbeat.yml for daemonset
  metricbeatConfig:
    metricbeat.yml: |
      metricbeat.modules:
      - module: kubernetes
        metricsets:
          - container
          - node
          - pod
          - system
          - volume
        period: 10s
        host: "${NODE_NAME}"
        hosts: ["${NODE_NAME}:10250"]
        # bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
        # ssl.verification_mode: "none"
        # If using Red Hat OpenShift remove ssl.verification_mode entry and
        # uncomment these settings:
        #ssl.certificate_authorities:
          #- /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt
        processors:
        - add_kubernetes_metadata: ~
      - module: kubernetes
        enabled: true
        metricsets:
          - event
      - module: system
        period: 10s
        metricsets:
          - cpu
          - load
          - memory
          - network
          - process
          - process_summary
        processes: ['.*']
        process.include_top_n:
          by_cpu: 5
          by_memory: 5
      - module: system
        period: 1m
        metricsets:
          - filesystem
          - fsstat
        processors:
        - drop_event.when.regexp:
            system.filesystem.mount_point: '^/(sys|cgroup|proc|dev|etc|host|lib)($|/)'
      output.elasticsearch:
        username: '${ES_USERNAME}'
        password: '${ES_PASSWORD}'
        hosts: ["elasticsearch-master:9200"]


deployment:
  extraEnvs:
    - name: 'ES_USERNAME'
      valueFrom:
        secretKeyRef:
          name: elastic-credentials
          key: username
    - name: ES_PASSWORD'
      valueFrom:
        secretKeyRef:
          name: elastic-credentials
          key: password
  # Allows you to add any config files in /usr/share/metricbeat
  # such as metricbeat.yml for deployment
  metricbeatConfig:
    metricbeat.yml: |
      metricbeat.modules:
      - module: kubernetes
        enabled: true
        metricsets:
          - state_node
          - state_deployment
          - state_replicaset
          - state_pod
          - state_container
        period: 10s
        hosts: ["${KUBE_STATE_METRICS_HOSTS}"]
      output.elasticsearch:
        username: '${ES_USERNAME}'
        password: '${ES_PASSWORD}'
        hosts: ["elasticsearch-master:9200"]

部署后的 metricbeat.yml不包含用户名和密码输入:

 metricbeat.modules:
    - module: kubernetes
      metricsets:
        - container
        - node
        - pod
        - system
        - volume
      period: 10s
      host: "${NODE_NAME}"
      hosts: ["${NODE_NAME}:10255"]
      processors:
      - add_kubernetes_metadata:
          in_cluster: true
    - module: kubernetes
      enabled: true
      metricsets:
        - event
    - module: system
      period: 10s
      metricsets:
        - cpu
        - load
        - memory
        - network
        - process
        - process_summary
      processes: ['.*']
      process.include_top_n:
        by_cpu: 5
        by_memory: 5
    - module: system
      period: 1m
      metricsets:
        - filesystem
        - fsstat
      processors:
      - drop_event.when.regexp:
          system.filesystem.mount_point: '^/(sys|cgroup|proc|dev|etc|host|lib)($|/)'
    output.elasticsearch:
      hosts: '${ELASTICSEARCH_HOSTS:elasticsearch-master:9200}'

预期的行为:应该使用用户名和密码连接到es，而不会出现问题。

提供日志和/或服务器输出(如果有):

   pipeline/output.go:100 Failed to connect to backoff(elasticsearch(http://elasticsearch-master:9200)): 401 Unauthorized: {"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}

任何其他上下文:

我能够使用相同的凭据连接 Elasticsearch ，并且也可以与Kibana登录一起正常工作。

Answer 1

我是Elastic Helm charts的维护者。

由于最近进行了重构，Git存储库主分支上记录的daemonset.xxx和deployment.xxx值与当前发行的图表版本不兼容。

如果要安装当前发布的版本(7.6.2)，则可以找到要在README from 7.6.2 release上使用的值。

如果要部署下一个即将发布的版本，并将使用daemonset.xxx和deployment.xxx值，则可以克隆存储库并使用installation instructions from 7.7 branch。

PS:如评论中所述，请注意 flex 图表尚不支持Helm v3，我们仍然建议您使用Helm v2。