docker - 是否可以在 Kubernetes 中启动自签名 Docker 注册表，并让其他服务使用它作为注册表来获取其镜像？

Question

问题陈述

• 我想提供一个私有(private)注册表，其中捆绑了我的产品所需的所有图像(是的，这将是胖图像，但我对此很满意)
• 我会以某种方式手动上传此图片
• 我会将 docker 私有(private)注册表作为 Kubernetes 中的服务运行(可能在某个命名空间中)
• 当其他服务/部署(与注册表在同一命名空间中)发生在 Kubernetes 中时，它们应该使用一致的名称引用此注册表

约束

• 我们希望注册表仅暴露给集群而不是外部
• 我们希望使用自签名证书而不是由 CA 签名

我按照这些链接中的一些说明进行操作(不知道这是否正确)

• https://kubernetes.io/docs/concepts/cluster-administration/certificates/
• https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster/#create-a-certificate-signing-request-object-to-send-to-the-kubernetes-api

创建通过 Kubernetes 签名的证书

1. 创建 server.key

2. 创建 csr.info

[ req ]
default_bits = 2048
prompt = no
default_md = sha256
req_extensions = req_ext
distinguished_name = dn

[ dn ]
C = US
ST = oh
L = cincinnati
O = engg
OU = prod
CN = prateek.svc.cluster.local

[ req_ext ]
subjectAltName = @alt_names

[ alt_names ]
DNS.1 = registry.prateek.svc.cluster.local

[ v3_ext ]
authorityKeyIdentifier=keyid,issuer:always
basicConstraints=CA:FALSE
keyUsage=keyEncipherment,dataEncipherment
extendedKeyUsage=serverAuth,clientAuth
subjectAltName=@alt_names

创建了 server.csr(openssl req -new -key server.key -out server.csr -config csr.conf)

在 K8s 中创建 CertificateSigningRequest

cat <<EOF | kubectl apply -f -
apiVersion: certificates.k8s.io/v1beta1
kind: CertificateSigningRequest
metadata:
name: registry.prateek
spec:
groups:
- system:authenticated
request: $(cat server.csr | base64 | tr -d '\n')
usages:
- digital signature
- key encipherment
- server auth
EOF

检查 CSR 是否存在

kubectl describe csr registry.prateek
Name: registry.prateek
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"certificates.k8s.io/v1beta1","kind":"CertificateSigningRequest","metadata":{"annotations":{},"name":"registry.prateek","namespace":""},"spec":{"groups":["system:authenticated"],"request":"LS0sdfsfsdsfd=","usages":["digital signature","key encipherment","server auth"]}}

CreationTimestamp: Thu, 11 Apr 2019 11:15:42 -0400
Requesting User: docker-for-desktop
Status: Pending
Subject:
Common Name: prateek.svc.cluster.local
Serial Number:
Organization: engg
Organizational Unit: prod
Country: US
Locality: cincinnati
Province: oh
Subject Alternative Names:
DNS Names: registry.prateek.svc.cluster.local
Events: <none>

批准了 CSR:kubectl 证书批准registry.prateek

启动registry内部服务

添加了证书和 key 类型:Secret

注册表 secret .yml

apiVersion: v1
kind: Secret
metadata:
  name: registry-credentials
data:
  certificate: <CERTIFICATE in base64>
  key: <KEY in base64>

创建注册表部署和服务(使用这些 secret ) 注册表部署.yml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: registry
  namespace: prateek
  labels:
      app: registry
spec:
  replicas: 1
  selector:
    matchLabels:
      app: registry
  template:
    metadata:
      labels:
        app: registry
    spec:
      containers:
        - name: registry
          image: prateek/registry
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 443
          env:
            - name: REGISTRY_HTTP_ADDR
              value: "0.0.0.0:443"
            - name: REGISTRY_HTTP_TLS_CERTIFICATE
              value: "/certs/certificate"
            - name: REGISTRY_HTTP_TLS_KEY
              value: "/certs/key"
          volumeMounts:
            - name: cert-files
              mountPath: /certs
      volumes:
        - name: cert-files
          secret:
            secretName: registry-credentials

注册表服务.yml

apiVersion: v1
kind: Service
metadata:
  name: registry
  namespace: prateek
spec:
  selector:
    app: registry
  ports:
  - protocol: TCP
    port: 443
    targetPort: 443
  type: LoadBalancer

测试注册服务已启动

尝试通过测试 Pod 访问注册表端点。我已经在 docker 中加载了这个测试容器的图像。

curl https://registry.prateek.svc.cluster.local/v2/_catalog -k
{"repositories":["prateek/echo"]}

使用注册表服务中的镜像进行部署

尝试使用镜像进行部署:registry.prateek/prateek/echo:latest

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello
  namespace: cequence
  labels:
      app: hello
spec:
  replicas: 1
  selector:
    matchLabels:
      app: hello
  template:
    metadata:
      labels:
        app: hello
    spec:
      containers:
      - name: hello
        image: registry.prateek/prateek/echo:latest
        imagePullPolicy: IfNotPresent
        ports:
         - containerPort: 5678
        args: ["-text=hello"]

部署出现错误

Normal Pulling 10s (x2 over 25s) kubelet, docker-for-desktop pulling image "registry.prateek/prateek/echo:latest"
Warning Failed 10s (x2 over 25s) kubelet, docker-for-desktop Failed to pull image "registry.prateek/prateek/echo:latest": rpc error: code = Unknown desc = Error response from daemon: Get https://registry.prateek/v2/: Service Unavailable 

更改了部署以包含镜像:registry.prateek.svc.cluster.local/prateek/echo:latest

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello
  namespace: cequence
  labels:
      app: hello
spec:
  replicas: 1
  selector:
    matchLabels:
      app: hello
  template:
    metadata:
      labels:
        app: hello
    spec:
      containers:
      - name: hello
        image: registry.prateek.svc.cluster.local/prateek/echo:latest
        imagePullPolicy: IfNotPresent
        ports:
         - containerPort: 5678
        args: ["-text=hello"]

得到类似的错误

Warning Failed 1s kubelet, docker-for-desktop Failed to pull image "registry.prateek.svc.cluster.local/prateek/echo:latest": rpc error: code = Unknown desc = Error response from daemon: Get https://registry.prateek.svc.cluster.local/v2/: Service Unavailable

我不认为这是可能的。将 Docker 注册表作为服务运行，并指向命名空间中的其他服务以在集群中使用该注册表部署。欢迎任何建议

Answer 1

容器守护进程在 kubernetes 之外运行。

因此，如果要拉取镜像，需要确保可以从节点直接访问registry，而不是使用像服务这样的kubernetes机制。 (与您在步骤 9 中通过 pod 进行测试不同，您必须能够直接在节点上工作!)

通常的选项是创建一个 DNS 条目或 ports.txt 条目来指向一个节点，该节点通过 hostPort(容器)或 nodePort(服务)注册表可访问或者您使用适当的入口。