kubernetes - Metricbeat kubernetes 模块无法连接到 kubelet

Question

我们有一个设置，其中 Metricbeat 作为 DaemonSet 部署在 Kubernetes 集群(特别是 AWS EKS)上。

一切似乎都正常运行，但 kubelet 联系。

为了澄清，以下模块:

- module: kubernetes
  enabled: true
  metricsets:
    - state_pod
  period: 10s
  hosts: ["kube-state-metrics.system:8080"]

工作正常(事件流入logstash/elastic)。

但是，此模块配置不适用于主机值的任何变体(localhost/kubernetes.default/whatever):

- module: kubernetes
  period: 10s
  metricsets:
    - pod
  hosts: ["localhost:10255"]
  enabled: true
  add_metadata: true
  in_cluster: true

NOTE: using cluster IP instead of localhost (so that it goes to control plane) also works (although doesn't retrieve the needed information, of course).

The configuration above was taken directly from the Metricbeat documentation and immediately struck me as odd -- how does localhost get translated (from within Metricbeat docker) to corresponding kubelet?

正如人们所预料的那样，鉴于上述情况，该错误是:

error making http request: Get http://localhost:10255/stats/summary: 
dial tcp [::1]:10255: connect: cannot assign requested address

这表明某种连接问题。

但是，当 SSH-ing 到任何部署 Metricbeat 的节点时，http://localhost:10255/stats/summary提供正确的输出:

{
  "node": {
   "nodeName": "...",
   "systemContainers": [
    {
     "name": "pods",
     "startTime": "2018-12-06T11:22:07Z",
     "cpu": {
      "time": "2018-12-23T06:54:06Z",
      ...
     },
     "memory": {
      "time": "2018-12-23T06:54:06Z",
      "availableBytes": 17882275840,
      ....

我一定遗漏了一些非常明显的东西。任何建议都可以。

注意:我在 Elasticsearch Forums 上交叉发布(并且几天没有回复)相同的内容

Answer 1

通过 valueFrom 注入(inject) Pod 的节点 IP env: 中的提供者列表:

env:
- name: HOST_IP
  valueFrom:
    fieldRef: status.hostIP

然后更新 metricbeat 配置文件以使用主机的 IP:

hosts: ["${HOST_IP}:10255"]

哪个 metricbeat 将通过其 environment variable config injection 解析