docker - kubectl 端口转发 : "pod does not exist" at the first time running?

Question

关注this指导，我正在尝试启动 minikube 并在启动时转发端口。

我的脚本:

#!/bin/bash

set -eux

export PATH=/usr/local/bin:$PATH

minikube status || minikube start
minikube ssh 'grep docker.for.mac.localhost /etc/hosts || echo -e "127.0.0.1\tdocker.for.mac.localhost" | sudo tee -a /etc/hosts'
minikube ssh 'test -f wait-for-it.sh || curl -O https://raw.githubusercontent.com/vishnubob/wait-for-it/master/wait-for-it.sh'
minikube ssh 'chmod +x wait-for-it.sh && ./wait-for-it.sh 127.0.1.1:10250'
POD=$(kubectl get po --namespace kube-system | awk '/kube-registry-v0/ { print $1 }')
kubectl port-forward --namespace kube-system $POD 5000:5000

一切正常，除了 kubectl port-forward 说 pod 在第一次运行时不存在:

++ kubectl get po --namespace kube-system
++ awk '/kube-registry-v0/ { print $1 }'
+ POD=kube-registry-v0-qr2ml
+ kubectl port-forward --namespace kube-system kube-registry-v0-qr2ml 5000:5000
error: error upgrading connection: unable to upgrade connection: pod does not exist

如果我重新运行:

+ minikube status
minikube: Running
cluster: Running
kubectl: Correctly Configured: pointing to minikube-vm at 192.168.99.100
+ minikube ssh 'grep docker.for.mac.localhost /etc/hosts || echo -e "127.0.0.1\tdocker.for.mac.localhost" | sudo tee -a /etc/hosts'
127.0.0.1   docker.for.mac.localhost
+ minikube ssh 'test -f wait-for-it.sh || curl -O https://raw.githubusercontent.com/vishnubob/wait-for-it/master/wait-for-it.sh'
+ minikube ssh 'chmod +x wait-for-it.sh && ./wait-for-it.sh 127.0.1.1:10250'
wait-for-it.sh: waiting 15 seconds for 127.0.1.1:10250
wait-for-it.sh: 127.0.1.1:10250 is available after 0 seconds
++ kubectl get po --namespace kube-system
++ awk '/kube-registry-v0/ { print $1 }'
+ POD=kube-registry-v0-qr2ml
+ kubectl port-forward --namespace kube-system kube-registry-v0-qr2ml 5000:5000
Forwarding from 127.0.0.1:5000 -> 5000
Forwarding from [::1]:5000 -> 5000

我在转发前添加了调试行:

kubectl describe pod --namespace kube-system $POD

看到这个:

+ POD=kube-registry-v0-qr2ml
+ kubectl describe pod --namespace kube-system kube-registry-v0-qr2ml
Name:       kube-registry-v0-qr2ml
Namespace:  kube-system
Node:       minikube/192.168.99.100
Start Time: Thu, 28 Dec 2017 10:00:00 +0700
Labels:     k8s-app=kube-registry
        version=v0
Annotations:    kubernetes.io/created-by={"kind":"SerializedReference","apiVersion":"v1","reference":{"kind":"ReplicationController","namespace":"kube-system","name":"kube-registry-v0","uid":"317ecc42-eb7b-11e7-a8ce-...
Status:     Running
IP:     172.17.0.6
Controllers:    ReplicationController/kube-registry-v0
Containers:
  registry:
    Container ID:   docker://6e8f3f33399605758354f3f546996067d834459781235d51eef3ffa9c6589947
    Image:      registry:2.5.1
    Image ID:       docker-pullable://registry@sha256:946480a23b33480b8e7cdb89b82c1bd6accae91a8e66d017e21e8b56551f6209
    Port:       5000/TCP
    State:      Running
      Started:      Thu, 28 Dec 2017 13:22:44 +0700

为什么kubectl说不存在？

---

2017 年 12 月 29 日星期五 04:58:06 +07

仔细查看事件，我发现了一些东西:

Events:
  FirstSeen     LastSeen        Count   From                    SubObjectPath                   Type            Reason                  Message
  ---------     --------        -----   ----                    -------------                   --------        ------                  -------
  20m           20m             1       kubelet, minikube                                       Normal          SuccessfulMountVolume   MountVolume.SetUp succ
eeded for volume "image-store"
  20m           20m             1       kubelet, minikube                                       Normal          SuccessfulMountVolume   MountVolume.SetUp succ
eeded for volume "default-token-fs7kr"
  20m           20m             1       kubelet, minikube                                       Normal          SandboxChanged          Pod sandbox changed, it will be killed and re-created.
  20m           20m             1       kubelet, minikube       spec.containers{registry}       Normal          Pulled                  Container image "registry:2.5.1" already present on machine
  20m           20m             1       kubelet, minikube       spec.containers{registry}       Normal          Created                 Created container
  20m           20m             1       kubelet, minikube       spec.containers{registry}       Normal          Started                 Started container

Pod sandbox changed, it will be killed and re-created.

之前:

Containers:
  registry:
    Container ID:       docker://47c510dce00c6c2c29c9fe69665e1241c457d0666174a7723062c534e7229c58
    Image:              registry:2.5.1
    Image ID:           docker-pullable://registry@sha256:946480a23b33480b8e7cdb89b82c1bd6accae91a8e66d017e21e8b56551f6209
    Port:               5000/TCP
    State:              Running
      Started:          Thu, 28 Dec 2017 13:47:02 +0700
    Last State:         Terminated
      Reason:           Error
      Exit Code:        2
      Started:          Thu, 28 Dec 2017 13:22:44 +0700
      Finished:         Thu, 28 Dec 2017 13:45:18 +0700
    Ready:              True
    Restart Count:      14

之后:

Containers:
  registry:
    Container ID:       docker://3a7da784d3d596796111348757725f5af22b47c5edd0fc29a4ffbb84f3f08956
    Image:              registry:2.5.1
    Image ID:           docker-pullable://registry@sha256:946480a23b33480b8e7cdb89b82c1bd6accae91a8e66d017e21e8b56551f6209
    Port:               5000/TCP
    State:              Running
      Started:          Thu, 28 Dec 2017 19:03:04 +0700
    Last State:         Terminated
      Reason:           Error
      Exit Code:        2
      Started:          Thu, 28 Dec 2017 13:47:02 +0700
      Finished:         Thu, 28 Dec 2017 19:00:48 +0700
    Ready:              True
    Restart Count:      15

minikube 日志:

Dec 28 22:15:41 minikube localkube[3250]: W1228 22:15:41.102038
3250 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/kube-registry-v0-qr2ml through plugin: invalid network status for

Answer 1

POD=$(kubectl get po --namespace kube-system | awk '/kube-registry-v0/ { print $1 }')

请注意，使用 selector几乎肯定比使用文本实用程序更好，尤其是 kubectl 的“非结构化”输出。我不知道他们对默认输出的格式做出任何 promise ，这就是 --output=json 和 friend 存在的原因。但是，在您只需要名称的情况下，有一个特殊的 --output=name 可以按其说明进行操作，但要注意 Resource 前缀将位于名称前面( pods/kube-registry-v0-qr2ml 在你的情况下)

另外，我看到您“等待”，但仅仅因为端口正在接受连接并不意味着 Pod 已就绪。您实际上需要使用 --output=json(或者我猜是更多的 awk 脚本)来确保 Pod 正在运行和就绪，并达到后者状态当 kubernetes 和 Pod 一致认为一切都很酷时。

我怀疑，但必须进行实验才能确定错误消息只是误导；并不是说 kubernetes 对您的 Pod 一无所知，而只是它无法在它所处的状态下转发到它。

您还可以通过创建 type: NodePort 的 Service 来体验更好的成功，然后在分配的端口上与节点的 IP 通信；这完全避开了这个 kubectl-shell 困惑，但没有避开就绪部分——只有处于就绪状态的 Pod 才会从服务接收流量

---

作为一个次要的、迂腐的注释，--namespace 是 kubectl 的参数，而不是 port-forward 的参数，因此最正确的调用是 kubectl --namespace=kube-system port-forward kube-registry-v0-qr2ml 5000:5000 以确保参数没有被错误解析