kubernetes - Traefik 无法读取 k8s api

Question

这已经是我第四次搭建kubernetes集群了。它始终是相同的设置:基本的 k8s、作为反向代理的 traefik、仪表板、prometheus、elk-stack。但是这次 traefik 部署有些奇怪......

所以对于所有其他集群，我只是部署了我的默认设置，其中包含一些 rbac 条目、包含 toml 文件的配置映射、实际部署、服务和 web-ui:

RBAC:

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller
  namespace: infra
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
rules:
- apiGroups:
  - ""
  resources:
  - services
  - endpoints
  - secrets
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - extensions
  resources:
  - ingresses
  verbs:
  - get
  - list
  - watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
  name: traefik-ingress-controller
  namespace: infra

配置图:

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: traefik-toml
  labels:
    name: traefik-toml
  namespace: infra
data:
  traefik.toml: |-
    defaultEntryPoints = ["http","https"]
    [entryPoints]
      [entryPoints.http]
      address = ":80"
        [entryPoints.http.redirect]
          entryPoint = "https"
      [entryPoints.https]
      address = ":443"
        [entryPoints.https.tls]
          [[entryPoints.https.tls.certificates]]
          CertFile = "/ssl/external/<EXTERNAL_URL>.crt"
          KeyFile = "/ssl/external/<EXTERNAL_URL>.key"
          [[entryPoints.https.tls.certificates]]
          CertFile = "/ssl/internal/<INTERNAL_URL>.crt"
          KeyFile = "/ssl/internal/<INTERNAL_URL>.key"
    [accessLog]

部署

---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: traefik-ingress-controller
  namespace: infra
  labels:
    k8s-app: traefik-ingress-lb
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: traefik-ingress-lb
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
      containers:
      - image: traefik:v1.6.5
        name: traefik-ingress-lb
        volumeMounts:
        - mountPath: /ssl/external
          name: ssl-external
        - mountPath: /ssl/internal
          name: ssl-internal
        - name: traefik-toml
          subPath: traefik.toml
          mountPath: /config/traefik.toml
        ports:
        - name: http
          containerPort: 80
        - name: https
          containerPort: 443
        - name: admin
          containerPort: 8080
        args:
        - --configfile=/config/traefik.toml
        - --api
        - --kubernetes
        - --logLevel=INFO
      volumes:
      - name: ssl-external
        secret:
          secretName: <EXTERNAL_URL>.cert
      - name: ssl-internal
        secret:
          secretName: <INTERNAL_URL>.cert
      - name: traefik-toml
        configMap:
          name: traefik-toml

服务:

---
kind: Service
apiVersion: v1
metadata:
  name: traefik-ingress-service
  namespace: infra
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
    - protocol: TCP
      port: 80
      name: web
    - protocol: TCP
      port: 443
      name: sweb
  externalIPs:
    - <WORKER IP 1>
    - <WORKER IP 2>

这对其他人来说效果很好，但在新的人身上(我自己没有设置 kubernetes)，日志中每 30 秒就会出现以下错误(错误检查新版本不经常!):

E0827 14:29:49.566294       1 reflector.go:205] github.com/containous/traefik/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Service: Get https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout                                                       
E0827 14:29:49.572633       1 reflector.go:205] github.com/containous/traefik/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Endpoints: Get https://10.96.0.1:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout                                                    
E0827 14:29:49.592844       1 reflector.go:205] github.com/containous/traefik/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.Ingress: Get https://10.96.0.1:443/apis/extensions/v1beta1/ingresses?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout                                
time="2018-08-27T14:30:00Z" level=warning msg="Error checking new version: Get https://update.traefik.io/repos/containous/traefik/releases: dial tcp: i/o timeout"

有人知道吗？这是一个已知的问题？我找不到关于此主题的任何已知问题..

提前致谢!

Answer 1

我设法解决了这个问题:

问题是错误的 iptables FORWARD 策略，由较新的 docker 引擎设置:https://github.com/moby/moby/issues/35777

目前我们有一个变通办法，正在稳步将政策设置回接受。

如果我们有一个真正的修复，我希望记得回到这里发布它:)