kubernetes - 如何更改 RKE 中的 kubelet 选项，例如 Rancher 中的 eviction-hard(nodefs.available、imagefs.available 等)？

Question

我仔细阅读了 Kubernetes 文档 here关于扩展 imagefs.available 的默认 15%和其他参数，但没有说明如何设置，我已经安装了具有以下配置的 RKE(Rancher Kubernetes Engine)。

Kubernets Version:v1.17.5
OS: Ubuntu 18.04.4 LTS
Container Runtime: 19.3.9

顺便说一句，文档没有清楚地解释如何扩展默认驱逐，所以我来这里是想知道是否有人可以告诉我如何扩展它。
我也尝试过 kubelet 命令行，但我明白了

I0903 09:59:13.143727   31472 plugins.go:100] No cloud provider specified.
W0903 09:59:13.143760   31472 server.go:560] standalone mode, no API client
W0903 09:59:13.148508   31472 container_manager_linux.go:912] CPUAccounting not enabled for pid: 31472
W0903 09:59:13.148527   31472 container_manager_linux.go:915] MemoryAccounting not enabled for pid: 31472
W0903 09:59:13.579594   31472 server.go:474] No api server defined - no events will be sent to API server.
I0903 09:59:13.579620   31472 server.go:647] --cgroups-per-qos enabled, but --cgroup-root was not specified.  defaulting to /
F0903 09:59:13.581308   31472 server.go:274] failed to run Kubelet: running with swap on is not supported, please disable swap! or set --fail-swap-on flag to false. /proc/swaps contained: [Filename                           Type            Size    Used    Priority /dev/md0                                partition      16758780       9846712  -2]

附加信息
服务器(云、裸机等):裸机
Kubernetes 安装使用:rancher
当我通过 config.yaml 使用 Rancher 提供的 kuberconfig 文件执行此操作时，它会尝试在 docker 中而不是在 kubernets 容器中驱逐我的容器。
所以我的命令是这样的
kubelet --kubeconfig=/root/.kube/config --config=/root/.kube/config.yaml

E0903 12:29:25.934582   26731 remote_image.go:135] RemoveImage "sha256:8c8e0533fa439319ae0ada18d3a7380d33f14f313ea5dd750ab68a1fde65d909" from image service failed: rpc error: code = Unknown desc = Error response from daemon: conflict: unable to remove repository reference "rancher/rke-tools:v0.1.56" (must force) - container 2e31fd3c915d is using its referenced image 8c8e0533fa43
E0903 12:29:25.934602   26731 kuberuntime_image.go:120] Remove image "sha256:8c8e0533fa439319ae0ada18d3a7380d33f14f313ea5dd750ab68a1fde65d909" failed: rpc error: code = Unknown desc = Error response from daemon: conflict: unable to remove repository reference "rancher/rke-tools:v0.1.56" (must force) - container 2e31fd3c915d is using its referenced image 8c8e0533fa43
I0903 12:29:25.934614   26731 image_gc_manager.go:376] [imageGCManager]: Removing image "sha256:2b5329557072f7cd9591670db3814bb54a72aa474ca301f00c8a29597cd32c94" to free 18234790 bytes
E0903 12:29:25.935726   26731 remote_image.go:135] RemoveImage "sha256:2b5329557072f7cd9591670db3814bb54a72aa474ca301f00c8a29597cd32c94" from image service failed: rpc error: code = Unknown desc = Error response from daemon: conflict: unable to remove repository reference "willfarrell/autoheal:latest" (must force) - container 5fb28a06d30d is using its referenced image 2b5329557072
E0903 12:29:25.935746   26731 kuberuntime_image.go:120] Remove image "sha256:2b5329557072f7cd9591670db3814bb54a72aa474ca301f00c8a29597cd32c94" failed: rpc error: code = Unknown desc = Error response from daemon: conflict: unable to remove repository reference "willfarrell/autoheal:latest" (must force) - container 5fb28a06d30d is using its referenced image 2b5329557072
I0903 12:29:25.935758   26731 image_gc_manager.go:376] [imageGCManager]: Removing image "sha256:0a3db21bc2b49b5cddea7ead8b99387ab37a332b71f0b016e1983587bb7d24af" to free 421267928 bytes
E0903 12:29:25.937276   26731 remote_image.go:135] RemoveImage "sha256:0a3db21bc2b49b5cddea7ead8b99387ab37a332b71f0b016e1983587bb7d24af" from image service failed: rpc error: code = Unknown desc = Error response from daemon: conflict: unable to remove repository reference "linuxserver/plex:latest" (must force) - container 8ca7a3b6202b is using its referenced image 0a3db21bc2b4
E0903 12:29:25.937295   26731 kuberuntime_image.go:120] Remove image "sha256:0a3db21bc2b49b5cddea7ead8b99387ab37a332b71f0b016e1983587bb7d24af" failed: rpc error: code = Unknown desc = Error response from daemon: conflict: unable to remove repository reference "linuxserver/plex:latest" (must force) - container 8ca7a3b6202b is using its referenced image 0a3db21bc2b4
I0903 12:29:25.937307   26731 image_gc_manager.go:376] [imageGCManager]: Removing image "sha256:7aab9b2c0cd6d39e03815bf3429be845257a016c3a3f5fd9e22b1ec5c3e9f329" to free 291526564 bytes
E0903 12:29:25.938673   26731 remote_image.go:135] RemoveImage "sha256:7aab9b2c0cd6d39e03815bf3429be845257a016c3a3f5fd9e22b1ec5c3e9f329" from image service failed: rpc error: code = Unknown desc = Error response from daemon: conflict: unable to remove repository reference "rancher/rancher-agent:v2.4.3" (must force) - container 9eb5c8663b7d is using its referenced image 7aab9b2c0cd6
E0903 12:29:25.938693   26731 kuberuntime_image.go:120] Remove image "sha256:7aab9b2c0cd6d39e03815bf3429be845257a016c3a3f5fd9e22b1ec5c3e9f329" failed: rpc error: code = Unknown desc = Error response from daemon: conflict: unable to remove repository reference "rancher/rancher-agent:v2.4.3" (must force) - container 9eb5c8663b7d is using its referenced image 7aab9b2c0cd6
W0903 12:29:25.938733   26731 eviction_manager.go:430] eviction manager: unexpected error when attempting to reduce inodes pressure: wanted to free 9223372036854775807 bytes, but freed 0 bytes space with errors in image deletion: [rpc error: code = Unknown desc = Error response from daemon: conflict: unable to remove repository reference "portainer/portainer:latest" (must force) - container 6193548811df is using its referenced image 2869fc110bf7, rpc error: code = Unknown desc = Error response from daemon: conflict: unable to remove repository reference "rancher/coreos-etcd:v3.4.3-rancher1" (must force) - container fd5c341853c0 is using its referenced image a0b920cf970d, rpc error: code = Unknown desc = Error response from daemon: conflict: unable to remove repository reference "rancher/hyperkube:v1.17.5-rancher1" (must force) - container 640e1d162e1e is using its referenced image ff99d966b0ee, rpc error: code = Unknown desc = Error response from daemon: conflict: unable to remove repository reference "hawkinzzz/dngele:latest" (must force) - container c16369f9385b is using its referenced image f3eb38637bfa, rpc error: code = Unknown desc = Error response from daemon: conflict: unable to remove repository reference "pyouroboros/ouroboros:latest" (must force) - container c0e2abfb3104 is using its referenced image 6bde18e83f41, rpc error: code = Unknown desc = Error response from daemon: conflict: unable to remove repository reference "linuxserver/emby:beta" (must force) - container 3def488e8543 is using its referenced image 95d10324f024, rpc error: code = Unknown desc = Error response from daemon: conflict: unable to remove repository reference "physk/rclone-mergerfs:latest" (must force) - container 9affc85735a1 is using its referenced image 80e6bae8490b, rpc error: code = Unknown desc = Error response from daemon: conflict: unable to remove repository reference "rancher/rancher:latest" (must force) - container 549b67e38726 is using its referenced image ddb8c914dee4, rpc error: code = Unknown desc = Error response from daemon: conflict: unable to remove repository reference "rancher/rke-tools:v0.1.56" (must force) - container 2e31fd3c915d is using its referenced image 8c8e0533fa43, rpc error: code = Unknown desc = Error response from daemon: conflict: unable to remove repository reference "willfarrell/autoheal:latest" (must force) - container 5fb28a06d30d is using its referenced image 2b5329557072, rpc error: code = Unknown desc = Error response from daemon: conflict: unable to remove repository reference "linuxserver/plex:latest" (must force) - container 8ca7a3b6202b is using its referenced image 0a3db21bc2b4, rpc error: code = Unknown desc = Error response from daemon: conflict: unable to remove repository reference "rancher/rancher-agent:v2.4.3" (must force) - container 9eb5c8663b7d is using its referenced image 7aab9b2c0cd6]
I0903 12:29:25.951254   26731 eviction_manager.go:355] eviction manager: must evict pod(s) to reclaim inodes
I0903 12:29:25.951274   26731 eviction_manager.go:373] eviction manager: pods ranked for eviction: canal-l29wx_kube-system(421796dc-24b3-4bcb-97f3-f61e36665c05)
E0903 12:29:25.951289   26731 eviction_manager.go:575] eviction manager: cannot evict a critical pod canal-l29wx_kube-system(421796dc-24b3-4bcb-97f3-f61e36665c05)
I0903 12:29:25.951297   26731 eviction_manager.go:396] eviction manager: unable to evict any pods from the node

亲切的问候

Answer 1

The kubelet has the following default hard eviction threshold: memory.available<100Mi nodefs.available<10% nodefs.inodesFree<5% imagefs.available<15%

根据 official Rancher page :
您可以通过 Config File option 添加其他参数/​​绑定(bind)/环境变量在集群选项中。如需更多信息，请参阅 Extra Args, Extra Binds, and Extra Environment Variables在 RKE 文档中或浏览 Example Cluster.ymls .
查看 full example如何配置 kubelet 选项:

kubelet:
      # Base domain for the cluster
      cluster_domain: cluster.local
      # IP address for the DNS service endpoint
      cluster_dns_server: 10.43.0.10
      # Fail if swap is on
      fail_swap_on: false
      # Set max pods to 250 instead of default 110
      extra_args:
        max-pods: 250

在这里，您可以添加如下所示的内容或您真正需要的任何内容。

eviction-hard: "nodefs.available<1%,imagefs.available<1%"
eviction-soft-grace-period: "nodefs.available=30s,imagefs.available=30s"
eviction-soft: "nodefs.available<1%,imagefs.available<1%"

坦率地说，我不确定您是否可以在飞行中更改 RKE 的 kubelet，但可以肯定的是，您可以使用所需的选项从头开始创建集群。