docker - 具有持久卷的 kube 上的 Elasticsearch

Question

我正在尝试使用持久卷 (EBS) 在 Kubernetes (EKS) 集群上部署 elasticsearch，引用 https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: elasticsearch
spec:
  replicas: 1
  template:
    metadata:
      labels:
        run: elasticsearch
    spec:
      initContainers:
        - name: init-sysctl
          image: busybox
          imagePullPolicy: IfNotPresent
          command: ["sysctl", "-w", "vm.max_map_count=262144"]
          securityContext:
            privileged: true
      containers:
      - name: elasticsearch
        image: elasticsearch:6.6.1
        imagePullPolicy: IfNotPresent
        securityContext:
          privileged: true
          capabilities:
            add:
              - IPC_LOCK
        ports:
        - containerPort: 9300
          protocol: TCP
        resources:
          limits:
            cpu: 1
        env:
        - name: CLUSTER_NAME
          value: elasticsearch
        - name: ES_JAVA_OPTS
          value: "-Xms256m -Xmx512m"
        volumeMounts:
          - name: elasticsearch-storage
            mountPath: /usr/share/elasticsearch/data
      volumes:
      - name: elasticsearch-storage
        persistentVolumeClaim:
          claimName: elasticsearch

但是 pod 失败了

OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[2019-03-13T18:03:08,459][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [unknown] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: Failed to create node environment
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.6.1.jar:6.6.1]
    at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.6.1.jar:6.6.1]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) ~[elasticsearch-6.6.1.jar:6.6.1]
Caused by: java.lang.IllegalStateException: Failed to create node environment
    at org.elasticsearch.node.Node.<init>(Node.java:298) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.node.Node.<init>(Node.java:265) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.6.1.jar:6.6.1]
    ... 6 more
Caused by: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/nodes
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:90) ~[?:?]
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) ~[?:?]
    at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:385) ~[?:?]
    at java.nio.file.Files.createDirectory(Files.java:689) ~[?:?]
    at java.nio.file.Files.createAndCheckIsDirectory(Files.java:796) ~[?:?]
    at java.nio.file.Files.createDirectories(Files.java:782) ~[?:?]
    at org.elasticsearch.env.NodeEnvironment.lambda$new$0(NodeEnvironment.java:270) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:203) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:267) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.node.Node.<init>(Node.java:295) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.node.Node.<init>(Node.java:265) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.6.1.jar:6.6.1]
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.6.1.jar:6.6.1]
    ... 6 more

从此discussion ，看起来像是 /usr/share/elasticsearch/data 目录上的权限问题。

以下initContainers帮助我解决了这个问题。

initContainers:
  - name: init-sysctl
    image: busybox
    imagePullPolicy: IfNotPresent
    #command: ["sysctl", "-w", "vm.max_map_count=262144"]
    command: ["/bin/sh","-c"]
    args: ["sysctl -w vm.max_map_count=262144; chown -R 1000:1000 /usr/share/elasticsearch/data"]
    securityContext:
      privileged: true
    volumeMounts:
      - name: elasticsearch-storage
        mountPath: /usr/share/elasticsearch/data

解决此问题的正确方法是什么？

Answer 1

您使用的解决方案是一种解决方法，但您可以用 kubernetes 的方式来实现，这里有解释:

Permission denied when chown on elasticsearch data directory in kubernetes statefulset

所以在你的情况下应该是这样的:

....

spec:
  ...
  securityContext:
    fsGroup: 1000