救命啊!我们在 Azure 上的 AKS 中运行 Kubernetes 1.24.10。我们通过 Helm 从 https://kubernetes.github.io/ingress-nginx 安装了一个 Nginx 入口 Controller ，但是，对此 IP 的所有外部请求(包括 ping)都会挂起并最终超时。
Nginx 通过负载均衡器公开，将所有 Web(80、443)流量转发到 K8 中 nginx 服务的外部 IP。负载均衡器还转发其他几个端口(例如 AMQP 端口 5671、5672 和 CouchDB 端口 5984)。它还在其他端口上转发到 K8 中的其他几个服务。
nginx Ingress 设置本身似乎完全没问题。这一点没有改变。 Ingress 指向的所有 Pod 均正常运行并已准备就绪。
据我们所知，负载均衡器设置很好，VM 规模集附加到的 NSG 也很好。作为引用，NSG规则如下(20.26.34.172为Nginx服务的外部IP)
我们没有运行 WAF 或应用程序网关。与另一个正在运行的 K8 集群相比，所有配置似乎都是一致的。与该集群唯一的区别是 K8 的版本是 1.23.12。
请注意，从集群内部(即从 K8 内部运行的 POD)实际上可以成功连接到该 IP。毫无疑问，仅受影响的是外部访问。另请注意，与其他公开 IP 的连接已成功，因此它仅对 nginx IP 是唯一的。
当我们在此集群上将 K8 从 1.23 升级到 1.24 时，实际上就开始发生这种情况。 我们查看了 K8 的文档，没有发现升级会导致此问题的任何原因。
我们正在从头开始重新创建此集群，但是，我确实需要了解出了什么问题，这样我在升级生产集群时就不会遇到同样的问题!
来自 nginx pod 的示例日志。请注意，在此日志期间发出了外部请求。没有提及请求。我认为所有这些记录的请求都是集群本地的:
10.244.1.1 - development [31/Jul/2023:10:11:47 +0000] "POST /realms/clarity-go/protocol/openid-connect/token HTTP/2.0" 200 2289 "-" "okhttp/4.9.0" 279 0.194 [default-keycloak-8080] [] 10.244.1.38:8080 2289 0.194 200 3eeafa82ee9d3f5a7c6e36bdae4a793c
10.244.1.1 - - [31/Jul/2023:10:11:49 +0000] "GET /featuretoggle-service/v1/development/accounts/522/features/PACKAGE_ALLOCATION HTTP/2.0" 204 0 "-" "okhttp/4.9.0" 1290 2.336 [default-gateway-service-3011] [] 10.244.1.35:3011 0 2.337 204 fc9638892b735f7bdd5266dcbd0c1b85
10.244.1.1 - admin [31/Jul/2023:10:11:49 +0000] "GET /tenant-522/_design/views/_view/by_not_done_and_completed HTTP/1.1" 404 58 "-" "-" 264 0.007 [default-go-couchdb-server-svc-couchdb-5984] [] 10.244.1.49:5984 58 0.006 404 62a0857ef8d05ea6ae643624468ffadd
10.244.1.1 - admin [31/Jul/2023:10:11:49 +0000] "GET /tenant-522-4212/_design/views/_view/by_not_done_and_completed HTTP/1.1" 200 53 "-" "-" 269 0.021 [default-go-couchdb-server-svc-couchdb-5984] [] 10.244.1.49:5984 58 0.021 200 92271aa19c98de58ddb62db6485faf4f
10.244.1.1 - admin [31/Jul/2023:10:11:49 +0000] "GET /tenant-522-6229/_design/views/_view/by_not_done_and_completed HTTP/1.1" 200 53 "-" "-" 269 0.024 [default-go-couchdb-server-svc-couchdb-5984] [] 10.244.1.49:5984 58 0.025 200 9591c4ccb6c15c7f7ae60f3f2c457207
10.244.1.1 - admin [31/Jul/2023:10:11:49 +0000] "GET /tenant-522-6230/_design/views/_view/by_not_done_and_completed HTTP/1.1" 200 53 "-" "-" 269 0.017 [default-go-couchdb-server-svc-couchdb-5984] [] 10.244.1.51:5984 58 0.018 200 09f78a7cbdba8b49f1da8e7042828769
10.244.1.1 - admin [31/Jul/2023:10:11:49 +0000] "GET /tenant-522-6231/_design/views/_view/by_not_done_and_completed HTTP/1.1" 200 53 "-" "-" 269 0.020 [default-go-couchdb-server-svc-couchdb-5984] [] 10.244.1.51:5984 58 0.020 200 76ba1fcbb0a86e60a20b2791fc3f95d6
10.244.1.1 - admin [31/Jul/2023:10:11:50 +0000] "GET /tenant-522-6232/_design/views/_view/by_not_done_and_completed HTTP/1.1" 200 53 "-" "-" 269 0.019 [default-go-couchdb-server-svc-couchdb-5984] [] 10.244.1.51:5984 58 0.019 200 21509be8d781fda8a5af12b56ea21e28
10.244.1.1 - admin [31/Jul/2023:10:11:50 +0000] "GET /tenant-522-6233/_design/views/_view/by_not_done_and_completed HTTP/1.1" 200 53 "-" "-" 269 0.022 [default-go-couchdb-server-svc-couchdb-5984] [] 10.244.1.49:5984 58 0.021 200 637a6b43a5bc0cb6d295fdcba8f6ce05
10.244.1.1 - admin [31/Jul/2023:10:11:50 +0000] "GET /tenant-522-6234/_design/views/_view/by_not_done_and_completed HTTP/1.1" 200 53 "-" "-" 269 0.018 [default-go-couchdb-server-svc-couchdb-5984] [] 10.244.1.50:5984 58 0.017 200 9d6e918046e82f7d422ecf22f6356aed
10.244.1.1 - admin [31/Jul/2023:10:11:50 +0000] "GET /tenant-522-6235/_design/views/_view/by_not_done_and_completed HTTP/1.1" 200 58 "-" "-" 269 0.018 [default-go-couchdb-server-svc-couchdb-5984] [] 10.244.1.51:5984 58 0.017 200 441a8544bf4ea1ce886dd1e8806e0419
10.244.1.1 - admin [31/Jul/2023:10:11:50 +0000] "GET /tenant-522-6236/_design/views/_view/by_not_done_and_completed HTTP/1.1" 200 53 "-" "-" 269 0.016 [default-go-couchdb-server-svc-couchdb-5984] [] 10.244.1.49:5984 58 0.015 200 8411552426c9558941f66d525f11dd55
10.244.1.1 - admin [31/Jul/2023:10:11:50 +0000] "GET /tenant-522-6289/_design/views/_view/by_not_done_and_completed HTTP/1.1" 200 53 "-" "-" 269 0.017 [default-go-couchdb-server-svc-couchdb-5984] [] 10.244.1.50:5984 58 0.017 200 b0b4ac459c989aa5d97c30c27a4ded85
10.244.1.1 - admin [31/Jul/2023:10:11:50 +0000] "GET /tenant-522-6290/_design/views/_view/by_not_done_and_completed HTTP/1.1" 200 53 "-" "-" 269 0.012 [default-go-couchdb-server-svc-couchdb-5984] [] 10.244.1.50:5984 58 0.011 200 9593703722886f167835a1ec89090498
10.244.1.1 - admin [31/Jul/2023:10:11:50 +0000] "GET /tenant-522-6291/_design/views/_view/by_not_done_and_completed HTTP/1.1" 200 53 "-" "-" 269 0.013 [default-go-couchdb-server-svc-couchdb-5984] [] 10.244.1.49:5984 58 0.013 200 76f333a1f9719494b828430ea5a63e13
10.244.1.1 - admin [31/Jul/2023:10:11:50 +0000] "GET /tenant-522-6292/_design/views/_view/by_not_done_and_completed HTTP/1.1" 200 58 "-" "-" 269 0.016 [default-go-couchdb-server-svc-couchdb-5984] [] 10.244.1.51:5984 58 0.017 200 f9375e19603ba562372c7b46d508756f
10.244.1.1 - admin [31/Jul/2023:10:11:50 +0000] "GET /tenant-522-6293/_design/views/_view/by_not_done_and_completed HTTP/1.1" 200 58 "-" "-" 269 0.013 [default-go-couchdb-server-svc-couchdb-5984] [] 10.244.1.51:5984 58 0.014 200 ecfff669e71e7549fe476184659e4dc7
10.244.1.1 - admin [31/Jul/2023:10:11:50 +0000] "GET /tenant-522-6533/_design/views/_view/by_not_done_and_completed HTTP/1.1" 200 53 "-" "-" 269 0.018 [default-go-couchdb-server-svc-couchdb-5984] [] 10.244.1.49:5984 58 0.018 200 6b44a74cc1b71e2bcffdbc42a255a01b
10.244.1.1 - admin [31/Jul/2023:10:11:50 +0000] "GET /tenant-522-8715/_design/views/_view/by_not_done_and_completed HTTP/1.1" 200 58 "-" "-" 269 0.020 [default-go-couchdb-server-svc-couchdb-5984] [] 10.244.1.50:5984 58 0.019 200 8f18eeb202c0327c72701296057a911c
10.244.1.1 - admin [31/Jul/2023:10:11:50 +0000] "GET /tenant-522-10758/_design/views/_view/by_not_done_and_completed HTTP/1.1" 200 53 "-" "-" 270 0.022 [default-go-couchdb-server-svc-couchdb-5984] [] 10.244.1.50:5984 58 0.022 200 2dda6139ecaf0ea9fd32ddf24e70f46e
10.244.1.1 - development [31/Jul/2023:10:11:50 +0000] "POST /realms/clarity-go/protocol/openid-connect/token HTTP/2.0" 200 2289 "-" "okhttp/4.9.0" 819 0.028 [default-keycloak-8080] [] 10.244.1.38:8080 2289 0.028 200 6d0bfe40b2eb00fd851fc182484eeaab
10.244.1.1 - - [31/Jul/2023:10:11:50 +0000] "GET /realms/clarity-go/.well-known/openid-configuration HTTP/1.1" 200 6180 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 308 0.006 [default-keycloak-8080] [] 10.244.1.38:8080 6180 0.006 200 54653626d560043c913f0549aee13b7e
10.244.1.1 - - [31/Jul/2023:10:11:50 +0000] "GET /realms/clarity-go/protocol/openid-connect/certs HTTP/1.1" 200 2941 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 279 0.006 [default-keycloak-8080] [] 10.244.1.38:8080 2941 0.006 200 d5518427000b8811f9e772c39049d376
10.244.1.1 - - [31/Jul/2023:10:11:50 +0000] "GET /featuretoggle-service/v1/development/accounts/583/features/PACKAGE_ALLOCATION HTTP/2.0" 204 0 "-" "okhttp/4.9.0" 1234 0.512 [default-gateway-service-3011] [] 10.244.1.16:3011 0 0.512 204 3b3cadaab2f635f180b6cb75a423ac9b
10.244.1.1 - development [31/Jul/2023:10:11:50 +0000] "POST /realms/clarity-go/protocol/openid-connect/token HTTP/2.0" 200 2289 "-" "okhttp/4.9.0" 815 0.010 [default-keycloak-8080] [] 10.244.1.38:8080 2289 0.011 200 7881613a0eb37800be1b9877a6cccbfc
10.244.1.1 - - [31/Jul/2023:10:11:51 +0000] "GET /featuretoggle-service/v1/development/accounts/8279/features/PACKAGE_ALLOCATION HTTP/2.0" 204 0 "-" "okhttp/4.9.0" 1239 0.056 [default-gateway-service-3011] [] 10.244.1.35:3011 0 0.056 204 2a8953682a97881a19c80b1e31e1cb8f
10.244.1.1 - user [31/Jul/2023:10:11:51 +0000] "PUT /eureka/apps/LICENCE-SERVICE/licence-service-7fc5b78d49-czvsd:licence-service:3003?status=UP&lastDirtyTimestamp=1690789665496 HTTP/1.1" 200 0 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 554 0.003 [default-discovery-service-3010] [] 10.244.1.21:3010 0 0.004 200 eaa0758aa5b8c301c6f9a647fe0243e0
10.244.1.1 - development [31/Jul/2023:10:11:51 +0000] "POST /realms/clarity-go/protocol/openid-connect/token HTTP/2.0" 200 2289 "-" "okhttp/4.9.0" 814 0.016 [default-keycloak-8080] [] 10.244.1.38:8080 2289 0.015 200 a4f70f47bc292e10e46eef7db8196248
10.244.1.1 - - [31/Jul/2023:10:11:51 +0000] "GET /featuretoggle-service/v1/development/accounts/14949/features/PACKAGE_ALLOCATION HTTP/2.0" 204 0 "-" "okhttp/4.9.0" 1283 0.081 [default-gateway-service-3011] [] 10.244.1.16:3011 0 0.081 204 8863b5fc8b63cdc800ce5effbe7ec5d3
10.244.1.1 - admin [31/Jul/2023:10:11:51 +0000] "GET /tenant-14949/_design/views/_view/by_not_done_and_completed HTTP/1.1" 404 58 "-" "-" 266 0.005 [default-go-couchdb-server-svc-couchdb-5984] [] 10.244.1.49:5984 58 0.005 404 8210c7821ec614db09cf7b7df6c3884a
10.244.1.1 - development [31/Jul/2023:10:11:51 +0000] "POST /realms/clarity-go/protocol/openid-connect/token HTTP/2.0" 200 2289 "-" "okhttp/4.9.0" 814 0.017 [default-keycloak-8080] [] 10.244.1.38:8080 2289 0.017 200 c96a20a3daad39f8d98851f47287798c
10.244.1.1 - - [31/Jul/2023:10:11:51 +0000] "GET /featuretoggle-service/v1/development/accounts/15618/features/PACKAGE_ALLOCATION HTTP/2.0" 204 0 "-" "okhttp/4.9.0" 1235 0.219 [default-gateway-service-3011] [] 10.244.1.35:3011 0 0.218 204 8a542806aeb06a1b175c7cf2d2060c9e
10.244.1.1 - admin [31/Jul/2023:10:11:51 +0000] "GET /tenant-15618/_design/views/_view/by_not_done_and_completed HTTP/1.1" 404 58 "-" "-" 266 0.019 [default-go-couchdb-server-svc-couchdb-5984] [] 10.244.1.51:5984 58 0.018 404 bfbce866b6ddfed9121514c4755a9f8d
10.244.1.1 - development [31/Jul/2023:10:11:51 +0000] "POST /realms/clarity-go/protocol/openid-connect/token HTTP/2.0" 200 2289 "-" "okhttp/4.9.0" 813 0.010 [default-keycloak-8080] [] 10.244.1.38:8080 2289 0.009 200 509236904ec509fc763629c707cfdf5d
10.244.1.1 - - [31/Jul/2023:10:11:51 +0000] "GET /featuretoggle-service/v1/development/accounts/24425/features/PACKAGE_ALLOCATION HTTP/2.0" 204 0 "-" "okhttp/4.9.0" 1238 0.074 [default-gateway-service-3011] [] 10.244.1.16:3011 0 0.075 204 b705bd01fc17a32a37371ff15800d820
10.244.1.1 - admin [31/Jul/2023:10:11:51 +0000] "GET /tenant-24425/_design/views/_view/by_not_done_and_completed HTTP/1.1" 404 58 "-" "-" 266 0.010 [default-go-couchdb-server-svc-couchdb-5984] [] 10.244.1.50:5984 58 0.009 404 279fc22e5fda82c3c79247d7b28b1a77
10.244.1.1 - admin [31/Jul/2023:10:11:51 +0000] "GET /tenant-24425-1/_design/views/_view/by_not_done_and_completed HTTP/1.1" 404 52 "-" "-" 268 0.016 [default-go-couchdb-server-svc-couchdb-5984] [] 10.244.1.50:5984 52 0.016 404 99a4e07e09fa6863a46940021630071f
10.244.1.1 - admin [31/Jul/2023:10:11:51 +0000] "GET /tenant-24425-2/_design/views/_view/by_not_done_and_completed HTTP/1.1" 404 52 "-" "-" 268 0.018 [default-go-couchdb-server-svc-couchdb-5984] [] 10.244.1.51:5984 52 0.017 404 68c39750861c2ff32b08df5dcfcb84e7
10.244.1.1 - admin [31/Jul/2023:10:11:51 +0000] "GET /tenant-24425-3/_design/views/_view/by_not_done_and_completed HTTP/1.1" 404 52 "-" "-" 268 0.012 [default-go-couchdb-server-svc-couchdb-5984] [] 10.244.1.49:5984 52 0.012 404 f48378a17fbb2b82059197a97a578a44
10.244.1.1 - development [31/Jul/2023:10:11:51 +0000] "POST /realms/clarity-go/protocol/openid-connect/token HTTP/2.0" 200 2289 "-" "okhttp/4.9.0" 814 0.016 [default-keycloak-8080] [] 10.244.1.38:8080 2289 0.015 200 4f4a53abea0b7539ce5c560370f661d1
10.244.1.1 - - [31/Jul/2023:10:11:51 +0000] "GET /featuretoggle-service/v1/development/accounts/25665/features/PACKAGE_ALLOCATION HTTP/2.0" 204 0 "-" "okhttp/4.9.0" 1284 0.047 [default-gateway-service-3011] [] 10.244.1.35:3011 0 0.047 204 8d173b92ed385667f19b606d5cfb6867
10.244.1.1 - development [31/Jul/2023:10:11:51 +0000] "POST /realms/clarity-go/protocol/openid-connect/token HTTP/2.0" 200 2289 "-" "okhttp/4.9.0" 814 0.018 [default-keycloak-8080] [] 10.244.1.38:8080 2289 0.018 200 5dc0026322e099a0b5c72d82f4d41ba6
10.244.1.1 - - [31/Jul/2023:10:11:51 +0000] "GET /featuretoggle-service/v1/development/accounts/25846/features/PACKAGE_ALLOCATION HTTP/2.0" 204 0 "-" "okhttp/4.9.0" 1244 0.072 [default-gateway-service-3011] [] 10.244.1.16:3011 0 0.072 204 e056c8fb69d9ed6cf68bf3bec6b4b550
10.244.1.1 - development [31/Jul/2023:10:11:51 +0000] "POST /realms/clarity-go/protocol/openid-connect/token HTTP/2.0" 200 2289 "-" "okhttp/4.9.0" 814 0.024 [default-keycloak-8080] [] 10.244.1.38:8080 2289 0.024 200 ff2741f7f29df78b510065d2425f9961
10.244.1.1 - - [31/Jul/2023:10:11:51 +0000] "GET /featuretoggle-service/v1/development/accounts/60825/features/PACKAGE_ALLOCATION HTTP/2.0" 204 0 "-" "okhttp/4.9.0" 1242 0.043 [default-gateway-service-3011] [] 10.244.1.35:3011 0 0.042 204 6d5c5df4820df464e6ac6874ca268436
10.244.1.1 - user [31/Jul/2023:10:11:52 +0000] "GET /eureka/apps/delta HTTP/1.1" 200 86 "-" "Apache-HttpClient/4.5.13 (Java/1.8.0_212)" 428 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 86 0.001 200 22277026060df14f5b32623827c66bca
10.244.1.1 - user [31/Jul/2023:10:11:53 +0000] "PUT /eureka/apps/GATEWAY-SERVICE/gateway-service-68b7fbf8d-8hnnt:gateway-service:3011?status=UP&lastDirtyTimestamp=1690790328891 HTTP/1.1" 200 0 "-" "Apache-HttpClient/4.5.13 (Java/1.8.0_212)" 553 0.001 [default-discovery-service-3010] [] 10.244.1.21:3010 0 0.002 200 adafd4a2d9c72bb0dac221fbc05b1cd0
10.244.1.1 - user [31/Jul/2023:10:11:53 +0000] "GET /eureka/apps/delta HTTP/1.1" 200 86 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 428 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 86 0.001 200 35a44834920a5b18dcb19159fab19b85
10.244.1.1 - user [31/Jul/2023:10:11:56 +0000] "PUT /eureka/apps/LICENCE-SERVICE/licence-service-7fc5b78d49-jkmsl:licence-service:3003?status=UP&lastDirtyTimestamp=1690794894857 HTTP/1.1" 200 0 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 554 0.003 [default-discovery-service-3010] [] 10.244.1.21:3010 0 0.003 200 f67743b6b9ffc9d467535c157b2219eb
10.244.1.1 - user [31/Jul/2023:10:11:56 +0000] "GET /eureka/apps/delta HTTP/1.1" 200 86 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 428 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 86 0.002 200 5785764e144fdca208fe12edef21ab1e
10.244.1.1 - user [31/Jul/2023:10:11:56 +0000] "PUT /eureka/apps/PAYMENT-SERVICE/payment-service-6f7689d57d-7t8r9:payment-service:3008?status=UP&lastDirtyTimestamp=1690787254935 HTTP/1.1" 200 0 "-" "Apache-HttpClient/4.5.13 (Java/1.8.0_212)" 554 0.001 [default-discovery-service-3010] [] 10.244.1.21:3010 0 0.002 200 dc73fa9ede76eb22169987781279f5eb
10.244.1.1 - user [31/Jul/2023:10:11:56 +0000] "GET /eureka/apps/delta HTTP/1.1" 200 86 "-" "Apache-HttpClient/4.5.13 (Java/1.8.0_212)" 428 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 86 0.002 200 bf0194f33cf5de87cd0c668340f5ea27
10.244.1.1 - user [31/Jul/2023:10:12:03 +0000] "GET /eureka/apps/delta HTTP/1.1" 200 86 "-" "Apache-HttpClient/4.5.13 (Java/1.8.0_212)" 428 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 86 0.002 200 d4c3464cf2f09cabdf1d1fc11f084982
10.244.1.1 - user [31/Jul/2023:10:12:06 +0000] "GET /eureka/apps/delta HTTP/1.1" 200 86 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 428 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 86 0.003 200 97b15ee3340dfbc1ecd86e298873ef17
10.244.1.1 - user [31/Jul/2023:10:12:06 +0000] "PUT /eureka/apps/CALLBACK-SERVICE/callback-service-7b876f9798-fgrgl:callback-service:3014?status=UP&lastDirtyTimestamp=1690794666923 HTTP/1.1" 200 0 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 557 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 0 0.003 200 7432c276afe3f257d43405c87e46e346
10.244.1.1 - user [31/Jul/2023:10:12:06 +0000] "PUT /eureka/apps/GATEWAY-SERVICE/gateway-service-68b7fbf8d-6kqm8:gateway-service:3011?status=UP&lastDirtyTimestamp=1690789651390 HTTP/1.1" 200 0 "-" "Apache-HttpClient/4.5.13 (Java/1.8.0_212)" 553 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 0 0.002 200 5f81d34f7b21b75b0dcfafb492b3ca70
10.244.1.1 - user [31/Jul/2023:10:12:12 +0000] "GET /eureka/apps/delta HTTP/1.1" 200 86 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 428 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 86 0.002 200 3ed71670e327293478532abb6979513d
10.244.1.1 - user [31/Jul/2023:10:12:12 +0000] "PUT /eureka/apps/FEATURETOGGLE-SERVICE/featuretoggle-service-97cd989f8-ml56b:featuretoggle-service:3002?status=UP&lastDirtyTimestamp=1690795362373 HTTP/1.1" 200 0 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 571 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 0 0.002 200 96a2898b2742197eed9ff6d5d173effc
10.244.1.1 - user [31/Jul/2023:10:12:14 +0000] "PUT /eureka/apps/PAYMENT-SERVICE/payment-service-6f7689d57d-qwtk8:payment-service:3008?status=UP&lastDirtyTimestamp=1690789658452 HTTP/1.1" 200 0 "-" "Apache-HttpClient/4.5.13 (Java/1.8.0_212)" 554 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 0 0.001 200 8f916b11c74beceb3ca724f5db094870
10.244.1.1 - user [31/Jul/2023:10:12:15 +0000] "PUT /eureka/apps/FEATURETOGGLE-SERVICE/featuretoggle-service-97cd989f8-bpvz9:featuretoggle-service:3002?status=UP&lastDirtyTimestamp=1690795364734 HTTP/1.1" 200 0 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 571 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 0 0.002 200 00ab7fbcfa35533b876db66ead8edf36
10.244.1.1 - user [31/Jul/2023:10:12:15 +0000] "GET /eureka/apps/delta HTTP/1.1" 200 86 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 428 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 86 0.002 200 ce367121e6f35044724b13c50c78081a
10.244.1.1 - user [31/Jul/2023:10:12:16 +0000] "GET /eureka/apps/delta HTTP/1.1" 200 86 "-" "Apache-HttpClient/4.5.13 (Java/1.8.0_212)" 428 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 86 0.001 200 642d60e2dc28b71b14e47c5a0ecce83b
10.244.1.1 - user [31/Jul/2023:10:12:21 +0000] "PUT /eureka/apps/LICENCE-SERVICE/licence-service-7fc5b78d49-czvsd:licence-service:3003?status=UP&lastDirtyTimestamp=1690789665496 HTTP/1.1" 200 0 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 554 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 0 0.002 200 4b337812287c3241236b69c616134646
10.244.1.1 - user [31/Jul/2023:10:12:22 +0000] "GET /eureka/apps/delta HTTP/1.1" 200 86 "-" "Apache-HttpClient/4.5.13 (Java/1.8.0_212)" 428 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 86 0.001 200 d8ed8180c6b5fdb678e0735a0d1b2cb7
10.244.1.1 - user [31/Jul/2023:10:12:23 +0000] "PUT /eureka/apps/GATEWAY-SERVICE/gateway-service-68b7fbf8d-8hnnt:gateway-service:3011?status=UP&lastDirtyTimestamp=1690790328891 HTTP/1.1" 200 0 "-" "Apache-HttpClient/4.5.13 (Java/1.8.0_212)" 553 0.004 [default-discovery-service-3010] [] 10.244.1.21:3010 0 0.005 200 c0eb2286e285c90588ee036528c3dbbf
10.244.1.1 - user [31/Jul/2023:10:12:23 +0000] "GET /eureka/apps/delta HTTP/1.1" 200 86 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 428 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 86 0.002 200 58e3cb4f313f7f0ef526c863f7c51af0
10.244.1.1 - user [31/Jul/2023:10:12:26 +0000] "PUT /eureka/apps/LICENCE-SERVICE/licence-service-7fc5b78d49-jkmsl:licence-service:3003?status=UP&lastDirtyTimestamp=1690794894857 HTTP/1.1" 200 0 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 554 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 0 0.003 200 d6e72927752202d0261a67db60f34e7a
10.244.1.1 - user [31/Jul/2023:10:12:26 +0000] "GET /eureka/apps/delta HTTP/1.1" 200 86 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 428 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 86 0.002 200 4e0edce7bda6607e087928549e0ea94a
10.244.1.1 - user [31/Jul/2023:10:12:26 +0000] "PUT /eureka/apps/PAYMENT-SERVICE/payment-service-6f7689d57d-7t8r9:payment-service:3008?status=UP&lastDirtyTimestamp=1690787254935 HTTP/1.1" 200 0 "-" "Apache-HttpClient/4.5.13 (Java/1.8.0_212)" 554 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 0 0.002 200 f0dfd51b71202b05b3378309ca03b9a9
10.244.1.1 - user [31/Jul/2023:10:12:26 +0000] "GET /eureka/apps/delta HTTP/1.1" 200 86 "-" "Apache-HttpClient/4.5.13 (Java/1.8.0_212)" 428 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 86 0.002 200 2a6e250d3ede16406564f60d0bfe2b09
10.244.1.1 - user [31/Jul/2023:10:12:33 +0000] "GET /eureka/apps/delta HTTP/1.1" 200 86 "-" "Apache-HttpClient/4.5.13 (Java/1.8.0_212)" 428 0.003 [default-discovery-service-3010] [] 10.244.1.21:3010 86 0.002 200 3ccfa34738152f42702267f99ea8688a
10.244.1.1 - user [31/Jul/2023:10:12:36 +0000] "GET /eureka/apps/delta HTTP/1.1" 200 86 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 428 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 86 0.002 200 4160ecceba00295c5e2133a02e0451fc
10.244.1.1 - user [31/Jul/2023:10:12:36 +0000] "PUT /eureka/apps/CALLBACK-SERVICE/callback-service-7b876f9798-fgrgl:callback-service:3014?status=UP&lastDirtyTimestamp=1690794666923 HTTP/1.1" 200 0 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 557 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 0 0.002 200 090b9dcc55bc5bf9e51499d9c986c11b
10.244.1.1 - user [31/Jul/2023:10:12:36 +0000] "PUT /eureka/apps/GATEWAY-SERVICE/gateway-service-68b7fbf8d-6kqm8:gateway-service:3011?status=UP&lastDirtyTimestamp=1690789651390 HTTP/1.1" 200 0 "-" "Apache-HttpClient/4.5.13 (Java/1.8.0_212)" 553 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 0 0.002 200 5a392742b3dbe97151072bd251db22ee
10.244.1.1 - user [31/Jul/2023:10:12:42 +0000] "GET /eureka/apps/delta HTTP/1.1" 200 86 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 428 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 86 0.003 200 98d85bf31f7e0fd4d78f690dd5f0af2b
10.244.1.1 - user [31/Jul/2023:10:12:42 +0000] "PUT /eureka/apps/FEATURETOGGLE-SERVICE/featuretoggle-service-97cd989f8-ml56b:featuretoggle-service:3002?status=UP&lastDirtyTimestamp=1690795362373 HTTP/1.1" 200 0 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 571 0.001 [default-discovery-service-3010] [] 10.244.1.21:3010 0 0.002 200 f1780057ff32aa0a22563fc2b7c34543
10.244.1.1 - user [31/Jul/2023:10:12:44 +0000] "PUT /eureka/apps/PAYMENT-SERVICE/payment-service-6f7689d57d-qwtk8:payment-service:3008?status=UP&lastDirtyTimestamp=1690789658452 HTTP/1.1" 200 0 "-" "Apache-HttpClient/4.5.13 (Java/1.8.0_212)" 554 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 0 0.001 200 97856716374568bd961a9dc4eba3baa0
10.244.1.1 - user [31/Jul/2023:10:12:45 +0000] "PUT /eureka/apps/FEATURETOGGLE-SERVICE/featuretoggle-service-97cd989f8-bpvz9:featuretoggle-service:3002?status=UP&lastDirtyTimestamp=1690795364734 HTTP/1.1" 200 0 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 571 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 0 0.002 200 53bb9e7c7558ae760f357027d12aff75
10.244.1.1 - user [31/Jul/2023:10:12:45 +0000] "GET /eureka/apps/delta HTTP/1.1" 200 86 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 428 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 86 0.003 200 aa1821eacafc4680bcb08812d2a0fd76
10.244.1.1 - user [31/Jul/2023:10:12:46 +0000] "GET /eureka/apps/delta HTTP/1.1" 200 86 "-" "Apache-HttpClient/4.5.13 (Java/1.8.0_212)" 428 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 86 0.002 200 49e021c4188fbf28ec81a9698cf1fad1
10.244.1.1 - user [31/Jul/2023:10:12:51 +0000] "PUT /eureka/apps/LICENCE-SERVICE/licence-service-7fc5b78d49-czvsd:licence-service:3003?status=UP&lastDirtyTimestamp=1690789665496 HTTP/1.1" 200 0 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 554 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 0 0.002 200 e02f4fc6dbcd1a6dd87c80a467451cd6
10.244.1.1 - user [31/Jul/2023:10:12:52 +0000] "GET /eureka/apps/delta HTTP/1.1" 200 86 "-" "Apache-HttpClient/4.5.13 (Java/1.8.0_212)" 428 0.003 [default-discovery-service-3010] [] 10.244.1.21:3010 86 0.003 200 e67ac6978f6abdf5288e027fca1b23f0
10.244.1.1 - user [31/Jul/2023:10:12:53 +0000] "PUT /eureka/apps/GATEWAY-SERVICE/gateway-service-68b7fbf8d-8hnnt:gateway-service:3011?status=UP&lastDirtyTimestamp=1690790328891 HTTP/1.1" 200 0 "-" "Apache-HttpClient/4.5.13 (Java/1.8.0_212)" 553 0.003 [default-discovery-service-3010] [] 10.244.1.21:3010 0 0.003 200 632d5b397733a5e764d11ccc175cb436
10.244.1.1 - user [31/Jul/2023:10:12:53 +0000] "GET /eureka/apps/delta HTTP/1.1" 200 86 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 428 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 86 0.002 200 8190576766265b7a21006683b3124468
10.244.1.1 - user [31/Jul/2023:10:12:56 +0000] "PUT /eureka/apps/LICENCE-SERVICE/licence-service-7fc5b78d49-jkmsl:licence-service:3003?status=UP&lastDirtyTimestamp=1690794894857 HTTP/1.1" 200 0 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 554 0.003 [default-discovery-service-3010] [] 10.244.1.21:3010 0 0.003 200 9f8eaa56b62385f1683739c61fd2c565
10.244.1.1 - user [31/Jul/2023:10:12:56 +0000] "GET /eureka/apps/delta HTTP/1.1" 200 86 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 428 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 86 0.002 200 77d71b57a9526c4e72322674c2ea8f8d
10.244.1.1 - user [31/Jul/2023:10:12:56 +0000] "PUT /eureka/apps/PAYMENT-SERVICE/payment-service-6f7689d57d-7t8r9:payment-service:3008?status=UP&lastDirtyTimestamp=1690787254935 HTTP/1.1" 200 0 "-" "Apache-HttpClient/4.5.13 (Java/1.8.0_212)" 554 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 0 0.001 200 a4ade43dc7a87e7b29b97ec934aca554
10.244.1.1 - user [31/Jul/2023:10:12:56 +0000] "GET /eureka/apps/delta HTTP/1.1" 200 86 "-" "Apache-HttpClient/4.5.13 (Java/1.8.0_212)" 428 0.003 [default-discovery-service-3010] [] 10.244.1.21:3010 86 0.002 200 14e366069223d26da89158d19bb6fb83
10.244.1.1 - user [31/Jul/2023:10:13:03 +0000] "GET /eureka/apps/delta HTTP/1.1" 200 86 "-" "Apache-HttpClient/4.5.13 (Java/1.8.0_212)" 428 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 86 0.002 200 33a620e83f41352bdc4f42b50e71b2f3
10.244.1.1 - user [31/Jul/2023:10:13:06 +0000] "GET /eureka/apps/delta HTTP/1.1" 200 86 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 428 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 86 0.002 200 25d599647965e209586091007606a7fa
10.244.1.1 - user [31/Jul/2023:10:13:06 +0000] "PUT /eureka/apps/CALLBACK-SERVICE/callback-service-7b876f9798-fgrgl:callback-service:3014?status=UP&lastDirtyTimestamp=1690794666923 HTTP/1.1" 200 0 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 557 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 0 0.002 200 76d59df1a057f2573b122e00986b10f8
10.244.1.1 - user [31/Jul/2023:10:13:06 +0000] "PUT /eureka/apps/GATEWAY-SERVICE/gateway-service-68b7fbf8d-6kqm8:gateway-service:3011?status=UP&lastDirtyTimestamp=1690789651390 HTTP/1.1" 200 0 "-" "Apache-HttpClient/4.5.13 (Java/1.8.0_212)" 553 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 0 0.002 200 e454226b5a0fd799849cf1c065599711
10.244.1.1 - user [31/Jul/2023:10:13:12 +0000] "GET /eureka/apps/delta HTTP/1.1" 200 86 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 428 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 86 0.001 200 1748c994d63a84ebe50a14ef00176b32
10.244.1.1 - user [31/Jul/2023:10:13:12 +0000] "PUT /eureka/apps/FEATURETOGGLE-SERVICE/featuretoggle-service-97cd989f8-ml56b:featuretoggle-service:3002?status=UP&lastDirtyTimestamp=1690795362373 HTTP/1.1" 200 0 "-" "Apache-HttpClient/4.5.14 (Java/1.8.0_212)" 571 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 0 0.002 200 f114695e9f3667841a8bafb70ebba223
10.244.1.1 - user [31/Jul/2023:10:13:14 +0000] "PUT /eureka/apps/PAYMENT-SERVICE/payment-service-6f7689d57d-qwtk8:payment-service:3008?status=UP&lastDirtyTimestamp=1690789658452 HTTP/1.1" 200 0 "-" "Apache-HttpClient/4.5.13 (Java/1.8.0_212)" 554 0.002 [default-discovery-service-3010] [] 10.244.1.21:3010 0 0.002 200 f9b1db6566d1f84236bc5a6b0e206e9f
更新:我们创建了一个全新的 Kubernetes 1.24 集群，并按照 https://kubernetes.github.io/ingress-nginx 上的基本安装步骤进行操作。这似乎工作正常。
最佳答案
通过将 ingress-nginx-controller 服务上的 externalTrafficPolicy 从 Cluster 更改为 Local 解决了此问题。
示例:
apiVersion: v1
kind: Service
metadata:
name: example-service
spec:
selector:
app: example
ports:
- port: 8765
targetPort: 9376
externalTrafficPolicy: Local
type: LoadBalancer
现阶段我不确定为什么这能解决问题。
Ingress-Nginx Controller 文档: https://kubernetes.github.io/ingress-nginx/user-guide/retaining-client-ipaddress/
关于此的 Kubernetes 文档: https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/#preserving-the-client-source-ip
关于Azure/Kubernetes (AKS) - Nginx 入口从互联网超时，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/76802105/