作为kubernetes的新手，我按照以下教程使用“packstack”集群。本教程使用的Kubernetes版本1.2.1相当老:
http://kubernetes.io/docs/getting-started-guides/coreos/coreos_multinode_cluster/
一切似乎都很好，我能够启动Pods，因此尽管我会尝试将DNS插件安装为kubernetes服务/ rc。我四处搜寻，发现附加的DNS需要kubernetes 1.3或更高版本。
我将主节点和节点云配置中的Kubernetes版本提高到1.4.5，然后重试。同样，一切似乎都可以正常工作-除非我尝试启动DNS复制 Controller (和服务)，但我在日志中看到以下错误:
Expected to load root CA config from /var/run/secrets/kubernetes.io/serviceaccount/ca.crt, but got err: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory
我不确定如何进行。我试图创建另一个服务帐户，但密码中没有证书。我的云配置没有引用任何根证书，所以我想知道这是问题吗？
我已经附加了主云和节点云配置，DNS附加yaml文件和DNS服务器日志。
主云配置:
#cloud-config
---
write-files:
- path: /etc/conf.d/nfs
permissions: '0644'
content: |
OPTS_RPC_MOUNTD=""
- path: /opt/bin/wupiao
permissions: '0755'
content: |
#!/bin/bash
# [w]ait [u]ntil [p]ort [i]s [a]ctually [o]pen
[ -n "$1" ] && \
until curl -o /dev/null -sIf http://${1}; do \
sleep 1 && echo .;
done;
exit $?
hostname: master
coreos:
etcd2:
name: master
listen-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001
advertise-client-urls: http://$private_ipv4:2379,http://$private_ipv4:4001
initial-cluster-token: k8s_etcd
listen-peer-urls: http://$private_ipv4:2380,http://$private_ipv4:7001
initial-advertise-peer-urls: http://$private_ipv4:2380
initial-cluster: master=http://$private_ipv4:2380
initial-cluster-state: new
fleet:
metadata: "role=master"
units:
- name: etcd2.service
command: start
- name: generate-serviceaccount-key.service
command: start
content: |
[Unit]
Description=Generate service-account key file
[Service]
ExecStartPre=-/usr/bin/mkdir -p /opt/bin
ExecStart=/bin/openssl genrsa -out /opt/bin/kube-serviceaccount.key 2048 2>/dev/null
RemainAfterExit=yes
Type=oneshot
- name: setup-network-environment.service
command: start
content: |
[Unit]
Description=Setup Network Environment
Documentation=https://github.com/kelseyhightower/setup-network-environment
Requires=network-online.target
After=network-online.target
[Service]
ExecStartPre=-/usr/bin/mkdir -p /opt/bin
ExecStartPre=/usr/bin/curl -L -o /opt/bin/setup-network-environment -z /opt/bin/setup-network-environment https://github.com/kelseyhightower/setup-network-environment/releases/download/v1.0.0/setup-network-environment
ExecStartPre=/usr/bin/chmod +x /opt/bin/setup-network-environment
ExecStart=/opt/bin/setup-network-environment
RemainAfterExit=yes
Type=oneshot
- name: fleet.service
command: start
- name: flanneld.service
command: start
drop-ins:
- name: 50-network-config.conf
content: |
[Unit]
Requires=etcd2.service
[Service]
ExecStartPre=/usr/bin/etcdctl set /coreos.com/network/config '{"Network":"10.244.0.0/16", "Backend": {"Type": "vxlan"}}'
- name: docker.service
command: start
- name: kube-apiserver.service
command: start
content: |
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/kubernetes/kubernetes
Requires=setup-network-environment.service etcd2.service generate-serviceaccount-key.service
After=setup-network-environment.service etcd2.service generate-serviceaccount-key.service
[Service]
EnvironmentFile=/etc/network-environment
ExecStartPre=-/usr/bin/mkdir -p /opt/bin
ExecStartPre=/usr/bin/curl -L -o /opt/bin/kube-apiserver -z /opt/bin/kube-apiserver https://storage.googleapis.com/kubernetes-release/release/v1.4.5/bin/linux/amd64/kube-apiserver
ExecStartPre=/usr/bin/chmod +x /opt/bin/kube-apiserver
ExecStartPre=/opt/bin/wupiao 127.0.0.1:2379/v2/machines
ExecStart=/opt/bin/kube-apiserver \
--service-account-key-file=/opt/bin/kube-serviceaccount.key \
--service-account-lookup=false \
--admission-control=NamespaceLifecycle,NamespaceAutoProvision,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota \
--runtime-config=api/v1 \
--allow-privileged=true \
--insecure-bind-address=0.0.0.0 \
--insecure-port=8080 \
--kubelet-https=true \
--secure-port=6443 \
--service-cluster-ip-range=10.244.0.0/16 \
--etcd-servers=http://127.0.0.1:2379 \
--public-address-override=${DEFAULT_IPV4} \
--logtostderr=true
Restart=always
RestartSec=10
- name: kube-controller-manager.service
command: start
content: |
[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/kubernetes/kubernetes
Requires=kube-apiserver.service
After=kube-apiserver.service
[Service]
ExecStartPre=/usr/bin/curl -L -o /opt/bin/kube-controller-manager -z /opt/bin/kube-controller-manager https://storage.googleapis.com/kubernetes-release/release/v1.4.5/bin/linux/amd64/kube-controller-manager
ExecStartPre=/usr/bin/chmod +x /opt/bin/kube-controller-manager
ExecStart=/opt/bin/kube-controller-manager \
--service-account-private-key-file=/opt/bin/kube-serviceaccount.key \
--master=127.0.0.1:8080 \
--logtostderr=true
Restart=always
RestartSec=10
- name: kube-scheduler.service
command: start
content: |
[Unit]
Description=Kubernetes Scheduler
Documentation=https://github.com/kubernetes/kubernetes
Requires=kube-apiserver.service
After=kube-apiserver.service
[Service]
ExecStartPre=/usr/bin/curl -L -o /opt/bin/kube-scheduler -z /opt/bin/kube-scheduler https://storage.googleapis.com/kubernetes-release/release/v1.4.5/bin/linux/amd64/kube-scheduler
ExecStartPre=/usr/bin/chmod +x /opt/bin/kube-scheduler
ExecStart=/opt/bin/kube-scheduler --master=127.0.0.1:8080
Restart=always
RestartSec=10
update:
group: alpha
reboot-strategy: off
节点云配置
#cloud-config
write-files:
- path: /opt/bin/wupiao
permissions: '0755'
content: |
#!/bin/bash
# [w]ait [u]ntil [p]ort [i]s [a]ctually [o]pen
[ -n "$1" ] && [ -n "$2" ] && while ! curl --output /dev/null \
--silent --head --fail \
http://${1}:${2}; do sleep 1 && echo -n .; done;
exit $?
coreos:
etcd2:
listen-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001
advertise-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001
initial-cluster: master=http://10.10.1.31:2380
proxy: on
fleet:
metadata: "role=node"
units:
- name: etcd2.service
command: start
- name: fleet.service
command: start
- name: flanneld.service
command: start
- name: docker.service
command: start
- name: setup-network-environment.service
command: start
content: |
[Unit]
Description=Setup Network Environment
Documentation=https://github.com/kelseyhightower/setup-network-environment
Requires=network-online.target
After=network-online.target
[Service]
ExecStartPre=-/usr/bin/mkdir -p /opt/bin
ExecStartPre=/usr/bin/curl -L -o /opt/bin/setup-network-environment -z /opt/bin/setup-network-environment https://github.com/kelseyhightower/setup-network-environment/releases/download/v1.0.0/setup-network-environment
ExecStartPre=/usr/bin/chmod +x /opt/bin/setup-network-environment
ExecStart=/opt/bin/setup-network-environment
RemainAfterExit=yes
Type=oneshot
- name: kube-proxy.service
command: start
content: |
[Unit]
Description=Kubernetes Proxy
Documentation=https://github.com/kubernetes/kubernetes
Requires=setup-network-environment.service
After=setup-network-environment.service
[Service]
ExecStartPre=/usr/bin/curl -L -o /opt/bin/kube-proxy -z /opt/bin/kube-proxy https://storage.googleapis.com/kubernetes-release/release/v1.4.5/bin/linux/amd64/kube-proxy
ExecStartPre=/usr/bin/chmod +x /opt/bin/kube-proxy
# wait for kubernetes master to be up and ready
ExecStartPre=/opt/bin/wupiao 10.10.1.31 8080
ExecStart=/opt/bin/kube-proxy \
--master=10.10.1.31:8080 \
--logtostderr=true
Restart=always
RestartSec=10
- name: kube-kubelet.service
command: start
content: |
[Unit]
Description=Kubernetes Kubelet
Documentation=https://github.com/kubernetes/kubernetes
Requires=setup-network-environment.service
After=setup-network-environment.service
[Service]
EnvironmentFile=/etc/network-environment
ExecStartPre=/usr/bin/curl -L -o /opt/bin/kubelet -z /opt/bin/kubelet https://storage.googleapis.com/kubernetes-release/release/v1.4.5/bin/linux/amd64/kubelet
ExecStartPre=/usr/bin/chmod +x /opt/bin/kubelet
# wait for kubernetes master to be up and ready
ExecStartPre=/opt/bin/wupiao 10.10.1.31 8080
ExecStart=/opt/bin/kubelet \
--address=0.0.0.0 \
--port=10250 \
--hostname-override=${DEFAULT_IPV4} \
--api-servers=10.10.1.31:8080 \
--allow-privileged=true \
--logtostderr=true \
--cadvisor-port=4194 \
--healthz-bind-address=0.0.0.0 \
--healthz-port=10248
Restart=always
RestartSec=10
update:
group: alpha
reboot-strategy: off
DNS附加程式码Yaml
apiVersion: v1
kind: Service
metadata:
name: kube-dns
namespace: kube-system
labels:
k8s-app: kube-dns
kubernetes.io/cluster-service: "true"
kubernetes.io/name: "KubeDNS"
spec:
selector:
k8s-app: kube-dns
clusterIP: 10.244.0.5
ports:
- name: dns
port: 53
protocol: UDP
- name: dns-tcp
port: 53
protocol: TCP
---
apiVersion: v1
kind: ReplicationController
metadata:
name: kube-dns-v20
namespace: kube-system
labels:
k8s-app: kube-dns
version: v20
kubernetes.io/cluster-service: "true"
spec:
replicas: 1
selector:
k8s-app: kube-dns
version: v20
template:
metadata:
labels:
k8s-app: kube-dns
version: v20
annotations:
scheduler.alpha.kubernetes.io/critical-pod: ''
scheduler.alpha.kubernetes.io/tolerations: '[{"key":"CriticalAddonsOnly", "operator":"Exists"}]'
spec:
containers:
- name: kubedns
image: gcr.io/google_containers/kubedns-amd64:1.8
resources:
limits:
memory: 170Mi
requests:
cpu: 100m
memory: 70Mi
livenessProbe:
httpGet:
path: /healthz-kubedns
port: 8080
scheme: HTTP
initialDelaySeconds: 60
timeoutSeconds: 5
successThreshold: 1
failureThreshold: 5
readinessProbe:
httpGet:
path: /readiness
port: 8081
scheme: HTTP
initialDelaySeconds: 3
timeoutSeconds: 5
args:
- --domain=cluster.local.
- --dns-port=10053
ports:
- containerPort: 10053
name: dns-local
protocol: UDP
- containerPort: 10053
name: dns-tcp-local
protocol: TCP
- name: dnsmasq
image: gcr.io/google_containers/kube-dnsmasq-amd64:1.4
livenessProbe:
httpGet:
path: /healthz-dnsmasq
port: 8080
scheme: HTTP
initialDelaySeconds: 60
timeoutSeconds: 5
successThreshold: 1
failureThreshold: 5
args:
- --cache-size=1000
- --no-resolv
- --server=127.0.0.1#10053
- --log-facility=-
ports:
- containerPort: 53
name: dns
protocol: UDP
- containerPort: 53
name: dns-tcp
protocol: TCP
- name: healthz
image: gcr.io/google_containers/exechealthz-amd64:1.2
resources:
limits:
memory: 50Mi
requests:
cpu: 10m
memory: 50Mi
args:
- --cmd=nslookup kubernetes.default.svc.cluster.local 127.0.0.1 >/dev/null
- --url=/healthz-dnsmasq
- --cmd=nslookup kubernetes.default.svc.cluster.local 127.0.0.1:10053 >/dev/null
- --url=/healthz-kubedns
- --port=8080
- --quiet
ports:
- containerPort: 8080
protocol: TCP
dnsPolicy: Default
DNS附加日志
E1118 17:33:10.140677 1 config.go:265] Expected to load root CA config from /var/run/secrets/kubernetes.io/serviceaccount/ca.crt, but got err: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory
I1118 17:33:10.141079 1 server.go:94] Using https://10.244.0.1:443 for kubernetes master, kubernetes API: <nil>
I1118 17:33:10.141596 1 server.go:99] v1.5.0-alpha.0.1651+7dcae5edd84f06-dirty
I1118 17:33:10.141728 1 server.go:101] FLAG: --alsologtostderr="false"
I1118 17:33:10.141840 1 server.go:101] FLAG: --dns-port="10053"
I1118 17:33:10.141931 1 server.go:101] FLAG: --domain="cluster.local."
I1118 17:33:10.142073 1 server.go:101] FLAG: --federations=""
I1118 17:33:10.142171 1 server.go:101] FLAG: --healthz-port="8081"
I1118 17:33:10.142260 1 server.go:101] FLAG: --kube-master-url=""
I1118 17:33:10.142345 1 server.go:101] FLAG: --kubecfg-file=""
I1118 17:33:10.142433 1 server.go:101] FLAG: --log-backtrace-at=":0"
I1118 17:33:10.142522 1 server.go:101] FLAG: --log-dir=""
I1118 17:33:10.142605 1 server.go:101] FLAG: --log-flush-frequency="5s"
I1118 17:33:10.142688 1 server.go:101] FLAG: --logtostderr="true"
I1118 17:33:10.142771 1 server.go:101] FLAG: --stderrthreshold="2"
I1118 17:33:10.142853 1 server.go:101] FLAG: --v="0"
I1118 17:33:10.142932 1 server.go:101] FLAG: --version="false"
I1118 17:33:10.143056 1 server.go:101] FLAG: --vmodule=""
I1118 17:33:10.143247 1 server.go:138] Starting SkyDNS server. Listening on port:10053
I1118 17:33:10.143455 1 server.go:145] skydns: metrics enabled on : /metrics:
I1118 17:33:10.143556 1 dns.go:166] Waiting for service: default/kubernetes
I1118 17:33:10.144214 1 logs.go:41] skydns: ready for queries on cluster.local. for tcp://0.0.0.0:10053 [rcache 0]
I1118 17:33:10.144358 1 logs.go:41] skydns: ready for queries on cluster.local. for udp://0.0.0.0:10053 [rcache 0]
I1118 17:33:10.154429 1 dns.go:172] Ignoring error while waiting for service default/kubernetes: Get https://10.244.0.1:443/api/v1/namespaces/default/services/kubernetes: x509: failed to load system roots and no roots provided. Sleeping 1s before retrying.
E1118 17:33:10.159852 1 reflector.go:214] pkg/dns/dns.go:155: Failed to list *api.Service: Get https://10.244.0.1:443/api/v1/services?resourceVersion=0: x509: failed to load system roots and no roots provided
E1118 17:33:10.171051 1 reflector.go:214] pkg/dns/dns.go:154: Failed to list *api.Endpoints: Get https://10.244.0.1:443/api/v1/endpoints?resourceVersion=0: x509: failed to load system roots and no roots provided
I1118 17:33:11.157527 1 dns.go:172] Ignoring error while waiting for service default/kubernetes: Get https://10.244.0.1:443/api/v1/namespaces/default/services/kubernetes: x509: failed to load system roots and no roots provided. Sleeping 1s before retrying.
E1118 17:33:11.166217 1 reflector.go:214] pkg/dns/dns.go:155: Failed to list *api.Service: Get https://10.244.0.1:443/api/v1/services?resourceVersion=0: x509: failed to load system roots and no roots provided
E1118 17:33:11.181840 1 reflector.go:214] pkg/dns/dns.go:154: Failed to list *api.Endpoints: Get https://10.244.0.1:443/api/v1/endpoints?resourceVersion=0: x509: failed to load system roots and no roots provided
最佳答案
出于礼貌，我发布了解决方案，这也可能会帮助处于相同情况的某人。我正在使用cloud-init启动K8S服务并在Pod中运行DNS。我意识到pod在不同的网络中运行，因此修改了dns-addon.yaml以将额外的arg传递给具有正确网络地址的kubedns容器:“---kube-master-url = http://10.10.1.31:8080”成功了。错误消息有点误导!
关于dns - DNS插件在Kubernetes 1.4.5中进入crashloopbackoff，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/40683027/