我用这个软件
Kubernetes 1.11.5
Haproxy: last
Nginx: 1.15.7
我从我购买的由 comodo CA 制造的证书中制作 default/tls-secret
并得到这个错误:
Error code: SSL_ERROR_RX_RECORD_TOO_LONG
这是我的配置 Haproxy 入口
---
apiVersion: v1
kind: Namespace
metadata:
name: ingress-controller
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: ingress-controller
namespace: ingress-controller
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: ingress-controller
rules:
- apiGroups:
- ""
resources:
- configmaps
- endpoints
- nodes
- pods
- secrets
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- "extensions"
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- "extensions"
resources:
- ingresses/status
verbs:
- update
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
name: ingress-controller
namespace: ingress-controller
rules:
- apiGroups:
- ""
resources:
- configmaps
- pods
- secrets
- namespaces
verbs:
- get
- apiGroups:
- ""
resources:
- configmaps
verbs:
- get
- update
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- apiGroups:
- ""
resources:
- endpoints
verbs:
- get
- create
- update
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: ingress-controller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-controller
subjects:
- kind: ServiceAccount
name: ingress-controller
namespace: ingress-controller
- apiGroup: rbac.authorization.k8s.io
kind: User
name: ingress-controller
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: ingress-controller
namespace: ingress-controller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-controller
subjects:
- kind: ServiceAccount
name: ingress-controller
namespace: ingress-controller
- apiGroup: rbac.authorization.k8s.io
kind: User
name: ingress-controller
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
labels:
run: ingress-default-backend
name: ingress-default-backend
namespace: ingress-controller
spec:
selector:
matchLabels:
run: ingress-default-backend
template:
metadata:
labels:
run: ingress-default-backend
spec:
containers:
- name: ingress-default-backend
image: gcr.io/google_containers/defaultbackend:1.0
ports:
- containerPort: 8080
resources:
limits:
cpu: 10m
memory: 20Mi
---
apiVersion: v1
kind: Service
metadata:
name: ingress-default-backend
namespace: ingress-controller
spec:
ports:
- port: 8080
selector:
run: ingress-default-backend
---
apiVersion: v1
kind: ConfigMap
metadata:
name: haproxy-ingress
namespace: ingress-controller
data:
ssl-options: force-tlsv12
---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
labels:
run: haproxy-ingress
name: haproxy-ingress
namespace: ingress-controller
spec:
updateStrategy:
type: RollingUpdate
selector:
matchLabels:
run: haproxy-ingress
template:
metadata:
labels:
run: haproxy-ingress
spec:
hostNetwork: true
nodeSelector:
role: edge-router
serviceAccountName: ingress-controller
containers:
- name: haproxy-ingress
image: quay.io/jcmoraisjr/haproxy-ingress
args:
- --default-backend-service=$(POD_NAMESPACE)/ingress-default-backend
- --default-ssl-certificate=default/tls-secret
- --configmap=$(POD_NAMESPACE)/haproxy-ingress
- --sort-backends
ports:
- name: http
containerPort: 80
- name: https
containerPort: 443
- name: stat
containerPort: 1936
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
这是我的应用程序和它的入口配置
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: meteo
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: meteo
template:
metadata:
labels:
app: meteo
spec:
containers:
- name: meteo
image: devprofi/meteo:v39
ports:
- containerPort: 443
imagePullSecrets:
- name: meteo-secret
---
apiVersion: v1
kind: Service
metadata:
name: meteo-svc
namespace: default
spec:
type: NodePort
ports:
# - port: 80
# targetPort: 80
# protocol: TCP
# name: http
- port: 443
targetPort: 443
protocol: TCP
name: https
selector:
app: meteo
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
ingress.kubernetes.io/ssl-passthrough: "true"
kubernetes.io/ingress.class: "haproxy"
ingress.kubernetes.io/secure-backends: "true"
ingress.kubernetes.io/backend-protocol: "HTTPS"
name: meteo-ingress
namespace: default
spec:
tls:
- hosts:
- meteotravel.ru
secretName: cafe-secret # this is another copy of secret made from my buyed cert and key
rules:
- host: meteotravel.ru
http:
paths:
- path: /
backend:
serviceName: meteo-svc
servicePort: 443
我尝试这个命令并得到错误
openssl s_client -connect meteotravel.ru:443
-----END CERTIFICATE-----
subject=/OU=Domain Control Validated/OU=PositiveSSL/CN=meteotravel.ru
issuer=/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Domain Validation Secure Server CA
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 6055 bytes and written 312 bytes
Verification error: self signed certificate in certificate chain
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES128-GCM-SHA256
Session-ID: BDD996AF8814404E3E385A6FBE49F56CA2668C54FD157FD3FB28F38DB64F771E
Session-ID-ctx:
Master-Key: F8EB4A4DA674F286E44C71605DF1D7DE4A6FE58D249162B086CE17E899FAC88CFA213018F89B8A9939CB842639D2B68A
PSK identity: None
PSK identity hint: None
SRP username: None
TLS session ticket lifetime hint: 300 (seconds)
TLS session ticket:
0000 - 5e 9a 46 20 a8 60 30 88-fa 2e c5 37 b7 29 0b 4e ^.F .`0....7.).N
0010 - 41 67 2b b6 e7 8e 2e 12-8b 55 0c ad 59 80 f7 d5 Ag+......U..Y...
0020 - d1 07 8e fc 92 a1 2e 01-59 cf 00 2d d5 39 11 10 ........Y..-.9..
0030 - bf f3 89 af 2d 7a 02 59-49 54 3a bf e4 8b 97 f3 ....-z.YIT:.....
0040 - 55 da 4b 6f 9b 86 c4 85-eb e4 f9 a1 e3 74 76 be U.Ko.........tv.
0050 - 65 57 76 ec e3 76 c9 c8-5a 47 c6 c2 ee eb bd ec eWv..v..ZG......
0060 - 61 88 7c 35 8c a6 c0 b3-25 b5 79 06 99 df 66 75 a.|5....%.y...fu
0070 - 8e 9d 3a 17 61 40 7c 1c-09 e3 07 aa 49 b9 c3 cf ..:.a@|.....I...
0080 - d7 ff 7d 1b cc 3f b9 3f-c7 bd ad 4d f9 4f eb 6c ..}..?.?...M.O.l
0090 - 6f 42 2e c8 30 75 a9 07-d4 9e f0 12 6b 9c ca ac oB..0u......k...
Start Time: 1544706461
Timeout : 7200 (sec)
Verify return code: 19 (self signed certificate in certificate chain)
Extended master secret: no
---
HTTP/1.0 408 Request Time-out
Cache-Control: no-cache
Connection: close
Content-Type: text/html
<html><body><h1>408 Request Time-out</h1>
Your browser didn't send a complete request in time.
</body></html>
closed
我也试过这个命令
curl -vL https://meteotravel.ru >/dev/null
* Rebuilt URL to: https://meteotravel.ru/
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 212.26.248.233...
* TCP_NODELAY set
* Connected to meteotravel.ru (212.26.248.233) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
} [5 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [222 bytes data]
* error:1408F10B:SSL routines:ssl3_get_record:wrong version number
* stopped the pause stream!
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
* Closing connection 0
curl: (35) error:1408F10B:SSL routines:ssl3_get_record:wrong version number
我的 nginx 可以独立工作 有配置
server {
access_log /var/log/nginx/default.access.log;
error_log /var/log/nginx/default.error.log warn;
listen 443 ssl default;
#listen 443 ssl http2 default reuseport;
# Redirect HTTP to HTTPS
if ($scheme = http) {
return 301 https://$host$request_uri;
}
ssl_certificate /etc/nginx/ssl/meteotravel.ru/mt.crt;
ssl_certificate_key /etc/nginx/ssl/meteotravel.ru/pk;
server_name meteotravel.ru;
root /var/www/html;
# Add index.php to the list if you are using PHP
index index.html index.htm index.nginx-debian.html;
# server_name _;
location /fop2{
# First attempt to serve request as file, then
# as directory, then fall back to displaying a 404.
# Add index.php to the list if you are using PHP
try_files $uri $uri/ =404;
}
location /{
try_files $uri $uri/ =404;
}
}
哪里出错了？？
这是我在服务器上的 haproxy 配置
我还尝试为 nginx 设置 proxy_protocol:listen 443 ssl proxy_protocol; 和 haproxy-ingress 的选项:代理协议(protocol):[v1|v2|v2-ssl|v2-ssl-cn] 所有这一切都是分开的 并在 nginx 后端的日志中获取错误
����kjih9876������2��.��*��&����=5"while reading PROXY protocol, client: 10.244.5.0, server: 0.0.0.0:443 2018 年 12 月 13 日 19:55:24 [错误] 7#7:2271 损坏的 header :“�6�v+\��w �?����3�Alm9i��L$&��h ��0��,��(��$�� ����kjih9876������2��..��&����=5"while reading PROXY protocol, client: 10.244.6.0, server: 0.0.0.0:443
最佳答案
这个问题是因为注释
ingress.kubernetes.io/secure-backends: "true" 。
它不需要，因为我们已经从 haproxy 中获取了 TCP 流中的安全数据。有了这个注解我们做了两次加密，nginx 不能正确解密
关于ssl - Kubernetes Haproxy Ingress + Nginx 作为后端 Https，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/53762751/