docker - Kubernetes 挂载的卷具有 root 所有权

Question

我的 Kubernetes 集群遇到问题。目前我正在 kubernetes 中成功运行我的 Laravel 应用程序。现在我试图使我的应用程序中的存储文件夹成为持久卷，因为它可用于存储图像和内容。我的部署现在如下所示:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: laravel-api-app
  namespace: my-project
  labels:
    app.kubernetes.io/name: laravel-api-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: laravel-api-app
  template:
    metadata:
      labels:
        app: laravel-api-app
    spec:
      containers:
        - name: laravel-api-app
          image: me/laravel-api:v1.0.0
          ports:
            - name: laravel
              containerPort: 8080
          imagePullPolicy: Always
          envFrom:
            - secretRef:
                name: laravel-api-secret
            - configMapRef:
                name: laravel-api-config
          volumeMounts:
            - name: storage
              mountPath: /var/www/html/storage
      imagePullSecrets:
        - name: regcred
      volumes:
        - name: storage
          persistentVolumeClaim:
            claimName: laravel-api-persistant-volume-claim

如您所见，我的声明已安装到 /var/www/html/storage 文件夹中。现在，在我的 Dockerfile 中，我将所有文件夹设置为用户 nobody，如下所示:

USER nobody
COPY --chown=nobody . /var/www/html

但是，使用此命令会在我的 pod 中产生以下文件夹权限 (ls -la):

drwxrwxrwx    1 www-data www-data      4096 Mar 14 18:24 .
drwxr-xr-x    1 root     root          4096 Feb 26 17:43 ..
-rw-rw-rw-    1 nobody   nobody          48 Mar 12 22:27 .dockerignore
-rw-rw-rw-    1 nobody   nobody         220 Mar 12 22:27 .editorconfig
-rw-r--r--    1 nobody   nobody         718 Mar 14 18:22 .env
-rw-rw-rw-    1 nobody   nobody         660 Mar 14 18:22 .env.example
-rw-rw-rw-    1 nobody   nobody         718 Mar 14 12:10 .env.pipeline
-rw-rw-rw-    1 nobody   nobody         111 Mar 12 22:27 .gitattributes
-rw-rw-rw-    1 nobody   nobody         171 Mar 14 12:10 .gitignore
drwxrwxrwx    2 nobody   nobody        4096 Mar 14 12:30 .gitlab-ci-scripts
-rw-rw-rw-    1 nobody   nobody        2336 Mar 14 01:13 .gitlab-ci.yml
-rw-rw-rw-    1 nobody   nobody         174 Mar 12 22:27 .styleci.yml
-rw-rw-rw-    1 nobody   nobody         691 Mar 14 10:02 Makefile
drwxrwxrwx    6 nobody   nobody        4096 Mar 12 22:27 app
-rwxrwxrwx    1 nobody   nobody        1686 Mar 12 22:27 artisan
drwxrwxrwx    1 nobody   nobody        4096 Mar 12 22:27 bootstrap
-rw-rw-rw-    1 nobody   nobody        1476 Mar 12 22:27 composer.json
-rw-rw-rw-    1 nobody   nobody      261287 Mar 12 22:27 composer.lock
drwxrwxrwx    2 nobody   nobody        4096 Mar 14 12:10 config
drwxrwxrwx    5 nobody   nobody        4096 Mar 12 22:27 database
drwxrwxrwx    5 nobody   nobody        4096 Mar 13 09:45 docker
-rw-rw-rw-    1 nobody   nobody         569 Mar 14 12:27 docker-compose-test.yml
-rw-rw-rw-    1 nobody   nobody         584 Mar 14 12:27 docker-compose.yml
-rw-rw-rw-    1 nobody   nobody        1013 Mar 14 18:24 package.json
-rw-rw-rw-    1 nobody   nobody        1405 Mar 12 22:27 phpunit.xml
drwxrwxrwx    5 nobody   nobody        4096 Mar 14 18:23 public
-rw-rw-rw-    1 nobody   nobody        3496 Mar 12 22:27 readme.md
drwxrwxrwx    6 nobody   nobody        4096 Mar 12 22:27 resources
drwxrwxrwx    2 nobody   nobody        4096 Mar 12 22:27 routes
drwxrwxrwx    2 nobody   nobody        4096 Mar 12 22:27 scripts
-rw-rw-rw-    1 nobody   nobody         563 Mar 12 22:27 server.php
drwxr-xr-x    2 root     root          4096 Mar 14 18:18 storage
drwxrwxrwx    4 nobody   nobody        4096 Mar 12 22:27 tests
drwxr-xr-x   38 nobody   nobody        4096 Mar 14 18:22 vendor
-rw-rw-rw-    1 nobody   nobody         538 Mar 12 22:27 webpack.mix.js

如您所见，我的存储文件夹有 root/root，我也希望它是 nobody/nobody。我考虑过创建一个像这样的 initContainer :

initContainers:
  - name: setup-storage
    image: busybox
    command: ['sh', '-c', '/path/to/setup-script.sh']
    volumeMounts:
      - name: storage
        mountPath: /path/to/storage/directory

setup-script.sh 包含:

#!/bin/sh

chown -R nobody:nobody /path/to/storage/directory
chmod -R 755 /path/to/storage/directory

但我有一种感觉，应该(或者是)有更简单的东西来获得我想要的结果。

我已经尝试添加 ID 为 65534 的 securityContext，如下所示:

安全上下文: 运行用户:65534 运行组:65534 fs组:65534

但这导致了相同的root/root owner/group。我尝试的最后一件事是创建一个像这样的 initContainer :

initContainers:
  - name: laravel-api-init
    image: me/laravel-api:v1.0.0
    args:
      - /bin/bash
      - -c
      - cp -Rnp /var/www/html/storage/* /mnt
    imagePullPolicy: Always
    envFrom:
      - secretRef:
          name: laravel-api-secret
      - configMapRef:
          name: laravel-api-config
    volumeMounts:
      - name: storage
        mountPath: /mnt

此“应该”将所有内容复制到 /mnt 这是存储的安装位置，然后启动实际部署，将复制的数据安装到应用程序中。不幸的是，这会返回错误:Init:ExitCode:127 kubernetes，这很奇怪，因为这两个位置确实存在。这种方法不应该发生的另一件事(我不知道是否会发生)是，一旦卷包含来自先前 session 的数据(可能在服务器重新启动后)，它就不会篡改已经存在的数据该应用程序。

简而言之

所以经过这个解释和我的尝试，这就是我想要实现的目标。我希望我的 Laravel 应用程序有一个持久卷(存储文件夹)，以便我将该 Laravel 应用程序的开发人员限制在给定的存储范围内。例如，当我创建 5GB 的 PV 时，他们无法为其应用程序存储超过 5GB 的数据。该存储必须是持久的，以便服务器重新启动后，存储仍然存在!

更新

这是带有安全上下文的更新后的 yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: laravel-api-app
  namespace: my-project
  labels:
    app.kubernetes.io/name: laravel-api-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: laravel-api-app
  template:
    metadata:
      labels:
        app: laravel-api-app
    spec:
      containers:
        - name: laravel-api-init
          image: docker.argoplan.nl/clients/opus-volvere/laravel-api/production:v1.0.0
          args:
            - /bin/sh
            - -c
            - cp -Rnp /var/www/html/storage/* /mnt
          imagePullPolicy: Always
          envFrom:
            - secretRef:
                name: laravel-api-secret
            - configMapRef:
                name: laravel-api-config
          volumeMounts:
            - name: storage
              mountPath: /mnt
          securityContext:
            fsGroup: 65534
            fsGroupChangePolicy: "OnRootMismatch"
      imagePullSecrets:
        - name: regcred
      volumes:
        - name: storage
          persistentVolumeClaim:
            claimName: laravel-api-persistant-volume-claim

出于调试目的，我将 initContainer 复制为实际容器，这样我就可以在 ArgoCD 中看到我的容器日志。如果是 initContainer，我看不到任何日志。使用上面的 yaml，我在日志中看到了这一点:

cp: can't create directory '/mnt/app': Permission denied
cp: can't create directory '/mnt/framework': Permission denied

这是实时 list ，它显然不包含新的安全上下文，而我刚刚生成了应用程序:

apiVersion: v1
kind: Pod
metadata:
  annotations:
    cni.projectcalico.org/containerID: 0a4ce0e873c92442fdaf1ac8a1313966bd995ae65471b34f70b9de2634edecf9
    cni.projectcalico.org/podIP: 10.1.10.55/32
    cni.projectcalico.org/podIPs: 10.1.10.55/32
  creationTimestamp: '2023-03-17T09:17:58Z'
  generateName: laravel-api-app-74b7d9584c-
  labels:
    app: laravel-api-app
    pod-template-hash: 74b7d9584c
  name: laravel-api-app-74b7d9584c-4dc9h
  namespace: my-project
  ownerReferences:
    - apiVersion: apps/v1
      blockOwnerDeletion: true
      controller: true
      kind: ReplicaSet
      name: laravel-api-app-74b7d9584c
      uid: d2e2ab4d-0916-43fc-b294-3e5eb2778c0d
  resourceVersion: '4954636'
  uid: 12327d67-cdf9-4387-afe8-3cf536531dd2
spec:
  containers:
    - args:
        - /bin/sh
        - '-c'
        - cp -Rnp /var/www/html/storage/* /mnt
      envFrom:
        - secretRef:
            name: laravel-api-secret
        - configMapRef:
            name: laravel-api-config
      image: 'me/laravel-api:v1.0.0'
      imagePullPolicy: Always
      name: laravel-api-init
      resources: {}
      securityContext: {}
      terminationMessagePath: /dev/termination-log
      terminationMessagePolicy: File
      volumeMounts:
        - mountPath: /mnt
          name: storage
        - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
          name: kube-api-access-8cfg8
          readOnly: true
  dnsPolicy: ClusterFirst
  enableServiceLinks: true
  imagePullSecrets:
    - name: regcred
  nodeName: tohatsu
  preemptionPolicy: PreemptLowerPriority
  priority: 0
  restartPolicy: Always
  schedulerName: default-scheduler
  securityContext: {}
  serviceAccount: default
  serviceAccountName: default
  terminationGracePeriodSeconds: 30
  tolerations:
    - effect: NoExecute
      key: node.kubernetes.io/not-ready
      operator: Exists
      tolerationSeconds: 300
    - effect: NoExecute
      key: node.kubernetes.io/unreachable
      operator: Exists
      tolerationSeconds: 300
  volumes:
    - name: storage
      persistentVolumeClaim:
        claimName: laravel-api-persistant-volume-claim
    - name: kube-api-access-8cfg8
      projected:
        defaultMode: 420
        sources:
          - serviceAccountToken:
              expirationSeconds: 3607
              path: token
          - configMap:
              items:
                - key: ca.crt
                  path: ca.crt
              name: kube-root-ca.crt
          - downwardAPI:
              items:
                - fieldRef:
                    apiVersion: v1
                    fieldPath: metadata.namespace
                  path: namespace
status:
  conditions:
    - lastProbeTime: null
      lastTransitionTime: '2023-03-17T09:17:58Z'
      status: 'True'
      type: Initialized
    - lastProbeTime: null
      lastTransitionTime: '2023-03-17T09:17:58Z'
      message: 'containers with unready status: [laravel-api-init]'
      reason: ContainersNotReady
      status: 'False'
      type: Ready
    - lastProbeTime: null
      lastTransitionTime: '2023-03-17T09:17:58Z'
      message: 'containers with unready status: [laravel-api-init]'
      reason: ContainersNotReady
      status: 'False'
      type: ContainersReady
    - lastProbeTime: null
      lastTransitionTime: '2023-03-17T09:17:58Z'
      status: 'True'
      type: PodScheduled
  containerStatuses:
    - containerID: >-
        containerd://eaf8e09f0e2aceec6cb26e09406518a5d9851f94dfb8f8be3ce3e65ee47e282c
      image: 'me/laravel-api:v1.0.0'
      imageID: >-
        me/laravel-api@secret
      lastState:
        terminated:
          containerID: >-
            containerd://eaf8e09f0e2aceec6cb26e09406518a5d9851f94dfb8f8be3ce3e65ee47e282c
          exitCode: 1
          finishedAt: '2023-03-17T09:20:53Z'
          reason: Error
          startedAt: '2023-03-17T09:20:53Z'
      name: laravel-api-init
      ready: false
      restartCount: 5
      started: false
      state:
        waiting:
          message: >-
            back-off 2m40s restarting failed container=laravel-api-init
            pod=laravel-api-app-74b7d9584c-4dc9h_my-project(12327d67-cdf9-4387-afe8-3cf536531dd2)
          reason: CrashLoopBackOff
  hostIP: 192.168.1.8
  phase: Running
  podIP: 10.1.10.55
  podIPs:
    - ip: 10.1.10.55
  qosClass: BestEffort
  startTime: '2023-03-17T09:17:58Z'

Answer 1

您没有提及您的 k8s 版本。当您使用 v1.23 以下的 k8s 时，我的答案可能不适合您。

Kubernetes 可以为您设置权限。使用 fsGroup 和 fsGroupChangePolicy，k8s 将为您接管这项工作。

containers:
  - name: laravel-api-app
    image: me/laravel-api:v1.0.0
    ports:
      - name: laravel
        containerPort: 8080
    imagePullPolicy: Always
    envFrom:
      - secretRef:
        name: laravel-api-secret
      - configMapRef:
          name: laravel-api-config
    volumeMounts:
      - name: storage
        mountPath: /var/www/html/storage
    # this part is new
    securityContext:
      # user/group of nobody should have the highest possible id
      fsGroup: 65534
      fsGroupChangePolicy: "OnRootMismatch"

Related configuration specs from k8s