kubernetes - 具有自定义 docker 镜像的 Jenkins 代理未与 minikube 中的 Controller 连接

Question

摘要: K8s minikkube 中的 Jenkins 在默认 jnlp 代理的情况下工作正常并且扩展良好，但在自定义 jnlp 图像的情况下坚持“等待代理连接”。

详细说明:

我正在使用 Jenkins 设置运行本地 minikube。

Jenkins Controller dockerfile:

from jenkins/jenkins:alpine

# Distributed Builds plugins
RUN /usr/local/bin/install-plugins.sh ssh-slaves

# install Notifications and Publishing plugins
RUN /usr/local/bin/install-plugins.sh email-ext
RUN /usr/local/bin/install-plugins.sh mailer
RUN /usr/local/bin/install-plugins.sh slack

# Artifacts
RUN /usr/local/bin/install-plugins.sh htmlpublisher

# UI
RUN /usr/local/bin/install-plugins.sh greenballs
RUN /usr/local/bin/install-plugins.sh simple-theme-plugin

# Scaling
RUN /usr/local/bin/install-plugins.sh kubernetes

# install Maven
USER root
RUN apk update && \
    apk upgrade && \
    apk add maven
USER jenkins

部署:

 apiVersion: extensions/v1beta1
    kind: Deployment
    metadata:
      name: jenkins
    spec:
      replicas: 1
      template:
        metadata:
          labels:
            app: jenkins
        spec:
          containers:
            - name: jenkins
              image: ybushnev/my-jenkins-image:1.3
              env:
                - name: JAVA_OPTS
                  value: -Djenkins.install.runSetupWizard=false
              ports:
                - name: http-port
                  containerPort: 8080
                - name: jnlp-port
                  containerPort: 50000
              volumeMounts:
                - name: jenkins-home
                  mountPath: /var/jenkins_home
          volumes:
            - name: jenkins-home
              emptyDir: {}

服务:

apiVersion: v1
kind: Service
metadata:
  name: jenkins
spec:
  type: NodePort
  ports:
    - port: 8080
      name: "http"
      targetPort: 8080
    - port: 50000
      name: "slave"
      targetPort: 50000
  selector:
    app: jenkins

部署后我有这样的服务:

Yuris-MBP-2% kubectl get services       
NAME         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                          AGE
jenkins      NodePort    10.108.30.10   <none>        8080:30267/TCP,50000:31588/TCP   1h
kubernetes   ClusterIP   10.96.0.1      <none>        443/TCP                          1h

Kubernetes master 运行于:

Yuris-MBP-2%  kubectl cluster-info | grep master
Kubernetes master is running at https://192.168.99.100:8443

根据上面的配置，我在 Jenkins 中指定云配置:

最后我为 Slave Pod 模板进行了这样的配置:

因此，通过 k8s 日志，我在 master 上看到了这样的日志:

Waiting for agent to connect (41/100): kubernetes-agent-tgskx
Waiting for agent to connect (42/100): kubernetes-agent-tgskx
Waiting for agent to connect (43/100): kubernetes-agent-tgskx
Waiting for agent to connect (44/100): kubernetes-agent-tgskx
Waiting for agent to connect (45/100): kubernetes-agent-tgskx

Jenkins 容器似乎是绿色的。 K8s没有日志，但是有这样的事件发生:

Successfully assigned kubernetes-agent-517tl to minikube
MountVolume.SetUp succeeded for volume "workspace-volume"
MountVolume.SetUp succeeded for volume "default-token-8sgh6"

重要 如果我没有将“jnlp”放在容器名称中(我想这很重要，因为在另一种情况下它需要一些默认的 jnlp 代理镜像)代理正在旋转并连接到 Controller 就好，但即使我有自定义的 docker 镜像在“Docker 镜像”字段内，它不会将其作为引用，因为我可以看到 Jenkins 代理没有它认为基于提供的镜像的此类工具/文件。 上次我尝试使用此图像:“gcr.io/cloud-solutions-images/jenkins-k8s-slave”，但对我来说，如果我将“jnlp”作为容器模板名称，则任何图像都会失败。 我尝试玩很多图像但没有运气...将非常高兴任何提示!

Answer 1

我认为您应该为主 Jenkins 设置凭据以启动新的 Pod。

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: jenkins
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/exec"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/log"]
  verbs: ["get","list","watch"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: jenkins
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: jenkins
subjects:
- kind: ServiceAccount
  name: jenkins

然后在您的部署中使用该帐户:

spec:           
      serviceAccountName: jenkins

查看我之前的回答 https://stackoverflow.com/a/47874390/2718151

我希望这会有所帮助。