我正在努力获取外部 DNS 绑定(bind)的 azureIdentity 并将 DNS 条目获取到我们的区域中。
Key error: I0423 19:27:52.830107 1 mic.go:610] No AzureIdentityBinding found for pod default/external-dns-84dcc5f68c-cl5h5 that matches selector: external-dns. it will be ignored Also, no azureAssignedIdentity is created since there is no match for the pod and selector/aadpodidbinding.
我正在使用 Terraform、Helm、Azure、Azure AKS、VSCODE 构建 IaaC，到目前为止，还使用了三个 Kubernetes 插件 - aad pod Identity、application-gateway-kubernetes-ingress 和 Bitnami external-dns。
由于身份未绑定(bind)，因此不会创建 azureAssignedIdentity，并且ExternalDNS 无法将记录放入我们的 DNS 区域。
名称和 aadpodidbindings 似乎是正确的。我尝试在 Terraform kubectl_manifest 提供程序中传递 fullnameOverride 来安装 BitnamiExternalDNS 的 Helm。我尝试抑制外部 DNS 名称和标签上的后缀。我尝试在集群本身上编辑 Helm 和 Kubernetes YAML 以尝试强制绑定(bind)。我尝试使用 AKS 用户管理的身份，该身份用于 AAD Pod 身份，位于集群的节点池资源组中。我尝试让 BitnamiExternalDNS 配置并添加 azure.json 文件，并且在添加和安装ExternalDNS 之前我也手动完成了此操作。我已尝试将托管标识分配给 AKS 群集的 VMSS。
谢谢!
JBP
PS C:\Workspace\tf\HelmOne> kubectl logs pod/external-dns-84dcc5f68c-542mv
: Refresh request failed. Status Code = '404'. Response body: getting assigned identities for pod default/external-dns-84dcc5f68c-542mv in CREATED state failed after 16 attempts, retry duration [5]s, error: <nil>. Check MIC pod logs for identity assignment errors\n"
time="2021-04-24T19:57:30Z" level=debug msg="Retrieving Azure DNS zones for resource group: one-hi-sso-dnsrg-tf."
time="2021-04-24T20:06:02Z" level=error msg="azure.BearerAuthorizer#WithAuthorization: Failed to refresh the Token for request to https://management.azure.com/subscriptions/8fb55161-REDACTED-3400b5271a8c/resourceGroups/one-hi-sso-dnsrg-tf/providers/Microsoft.Network/dnsZones?api-version=2018-05-01: StatusCode=404 -- Original Error: adal: Refresh request failed. Status Code = '404'. Response body: getting assigned identities for pod default/external-dns-84dcc5f68c-542mv in CREATED state failed after 16 attempts, retry duration [5]s, error: <nil>. Check MIC pod logs for identity assignment errors\n"
time="2021-04-24T20:06:02Z" level=debug msg="Retrieving Azure DNS zones for resource group: one-hi-sso-dnsrg-tf."
PS C:\Workspace\tf\HelmOne> kubectl logs pod/aad-pod-identity-nmi-vtmwm
I0424 20:07:22.400942 1 server.go:196] status (404) took 80007557875 ns for req.method=GET reg.path=/metadata/identity/oauth2/token req.remote=10.0.8.7
E0424 20:08:44.427353 1 server.go:375] failed to get matching identities for pod: default/external-dns-84dcc5f68c-542mv, error: getting assigned identities for pod default/external-dns-84dcc5f68c-542mv in CREATED state failed after 16 attempts, retry duration [5]s, error: <nil>. Check MIC pod logs for identity assignment errors
I0424 20:08:44.427400 1 server.go:196] status (404) took 80025612263 ns for req.method=GET reg.path=/metadata/identity/oauth2/token req.remote=10.0.8.7
PS C:\Workspace\TF\HelmOne> kubectl logs pod/aad-pod-identity-mic-86944f67b8-k4hds
I0422 21:05:11.298958 1 main.go:114] starting mic process. Version: v1.7.5. Build date: 2021-04-02-21:14
W0422 21:05:11.299031 1 main.go:119] --kubeconfig not passed will use InClusterConfig
I0422 21:05:11.299038 1 main.go:136] kubeconfig () cloudconfig (/etc/kubernetes/azure.json)
I0422 21:05:11.299205 1 main.go:144] running MIC in namespaced mode: false
I0422 21:05:11.299223 1 main.go:148] client QPS set to: 5. Burst to: 5
I0422 21:05:11.299243 1 mic.go:139] starting to create the pod identity client. Version: v1.7.5. Build date: 2021-04-02-21:14
I0422 21:05:11.318835 1 mic.go:145] Kubernetes server version: v1.18.14
I0422 21:05:11.319465 1 cloudprovider.go:122] MIC using user assigned identity: c380##### REDACTED #####814b for authentication.
I0422 21:05:11.392322 1 probes.go:41] initialized health probe on port 8080
I0422 21:05:11.392351 1 probes.go:44] started health probe
I0422 21:05:11.392458 1 metrics.go:341] registered views for metric
I0422 21:05:11.392544 1 prometheus_exporter.go:21] starting Prometheus exporter
I0422 21:05:11.392561 1 metrics.go:347] registered and exported metrics on port 8888
I0422 21:05:11.392568 1 mic.go:244] initiating MIC Leader election
I0422 21:05:11.393053 1 leaderelection.go:243] attempting to acquire leader lease default/aad-pod-identity-mic...
E0423 01:47:52.730839 1 leaderelection.go:325] error retrieving resource lock default/aad-pod-identity-mic: etcdserver: request timed out
resource "helm_release" "external-dns" {
name = "external-dns"
repository = "https://charts.bitnami.com/bitnami"
chart = "external-dns"
namespace = "default"
version = "4.0.0"
set {
name = "azure.cloud"
value = "AzurePublicCloud"
}
#MyDnsResourceGroup
set {
name = "azure.resourceGroup"
value = data.azurerm_resource_group.dnsrg.name
}
set {
name = "azure.tenantId"
value = data.azurerm_subscription.currenttenantid.tenant_id
}
set {
name = "azure.subscriptionId"
value = data.azurerm_subscription.currentSubscription.subscription_id
}
set {
name = "azure.userAssignedIdentityID"
value = azurerm_user_assigned_identity.external-dns-mi-tf.client_id
}
#Verbosity of the logs (options: panic, debug, info, warning, error, fatal, trace)
set {
name = "logLevel"
value = "trace"
}
set {
name = "sources"
value = "{service,ingress}"
}
set {
name = "domainFilters"
value = "{${var.child_domain_prefix}.${lower(var.parent_domain)}}"
}
#DNS provider where the DNS records will be created (mandatory) (options: aws, azure, google, ...)
set {
name = "provider"
value = "azure"
}
#podLabels: {aadpodidbinding: <selector>} # selector you defined above in AzureIdentityBinding
set {
name = "podLabels.aadpodidbinding"
value = "external-dns"
}
set {
name = "azure.useManagedIdentityExtension"
value = true
}
}
resource "helm_release" "aad-pod-identity" {
name = "aad-pod-identity"
repository = "https://raw.githubusercontent.com/Azure/aad-pod-identity/master/charts"
chart = "aad-pod-identity"
}
resource "helm_release" "ingress-azure" {
name = "ingress-azure"
repository = "https://appgwingress.blob.core.windows.net/ingress-azure-helm-package/"
chart = "ingress-azure"
namespace = "default"
version = "1.4.0"
set {
name = "debug"
value = "true"
}
set {
name = "appgw.name"
value = data.azurerm_application_gateway.appgwpub.name
}
set {
name = "appgw.resourceGroup"
value = data.azurerm_resource_group.appgwpubrg.name
}
set {
name = "appgw.subscriptionId"
value = data.azurerm_subscription.currentSubscription.subscription_id
}
set {
name = "appgw.usePrivateIP"
value = "false"
}
set {
name = "armAuth.identityClientID"
value = azurerm_user_assigned_identity.agic-mi-tf.client_id
}
set {
name = "armAuth.identityResourceID"
value = azurerm_user_assigned_identity.agic-mi-tf.id
}
set {
name = "armAuth.type"
value = "aadPodIdentity"
}
set {
name = "rbac.enabled"
value = "true"
}
set {
name = "verbosityLevel"
value = "5"
}
set {
name = "appgw.environment"
value = "AZUREPUBLICCLOUD"
}
set {
name = "metadata.name"
value = "ingress-azure"
}
}
PS C:\Workspace\tf\HelmOne> kubectl get azureassignedidentities
NAME AGE
ingress-azure-68c97fd496-qbptf-default-ingress-azure 23h
PS C:\Workspace\tf\HelmOne> kubectl get azureidentity
NAME AGE
ingress-azure 23h
one-hi-sso-agic-mi-tf 23h
one-hi-sso-external-dns-mi-tf 23h
PS C:\Workspace\tf\HelmOne> kubectl edit azureidentity one-hi-sso-external-dns-mi-tf
apiVersion: aadpodidentity.k8s.io/v1
kind: AzureIdentity
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"aadpodidentity.k8s.io/v1","kind":"AzureIdentity","metadata":{"annotations":{},"name":"one-hi-sso-external-dns-mi-tf","namespace":"default"},"spec":{"clientID":"f58e7c55-REDACTED-a6e358e53912","resourceID":"/subscriptions/8fb55161-REDACTED-3400b5271a8c/resourceGroups/one-hi-sso-kuberg-tf/providers/Microsoft.ManagedIdentity/userAssignedIdentities/one-hi-sso-external-dns-mi-tf","type":0}}
creationTimestamp: "2021-04-22T20:44:42Z"
generation: 2
name: one-hi-sso-external-dns-mi-tf
namespace: default
resourceVersion: "432055"
selfLink: /apis/aadpodidentity.k8s.io/v1/namespaces/default/azureidentities/one-hi-sso-external-dns-mi-tf
uid: f8e22fd9-REDACTED-6cdead0d7e22
spec:
clientID: f58e7c55-REDACTED-a6e358e53912
resourceID: /subscriptions/8fb55161-REDACTED-3400b5271a8c/resourceGroups/one-hi-sso-kuberg-tf/providers/Microsoft.ManagedIdentity/userAssignedIdentities/one-hi-sso-external-dns-mi-tf
type: 0
PS C:\Workspace\tf\HelmOne> kubectl edit azureidentitybinding external-dns-mi-binding
apiVersion: aadpodidentity.k8s.io/v1
kind: AzureIdentityBinding
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"aadpodidentity.k8s.io/v1","kind":"AzureIdentityBinding","metadata":{"annotations":{},"name":"external-dns-mi-binding","namespace":"default"},"spec":{"AzureIdentity":"one-hi-sso-external-dns-mi-tf","Selector":"external-dns"}}
creationTimestamp: "2021-04-22T20:44:42Z"
generation: 1
name: external-dns-mi-binding
namespace: default
resourceVersion: "221101"
selfLink: /apis/aadpodidentity.k8s.io/v1/namespaces/default/azureidentitybindings/external-dns-mi-binding
uid: f39e7418-e896-4b8e-b596-035cf4b66252
spec:
AzureIdentity: one-hi-sso-external-dns-mi-tf
Selector: external-dns
resource "kubectl_manifest" "one-hi-sso-external-dns-mi-tf" {
yaml_body = <<YAML
apiVersion: "aadpodidentity.k8s.io/v1"
kind: AzureIdentity
metadata:
name: one-hi-sso-external-dns-mi-tf
namespace: default
spec:
type: 0
resourceID: /subscriptions/8fb55161-REDACTED-3400b5271a8c/resourceGroups/one-hi-sso-kuberg-tf/providers/Microsoft.ManagedIdentity/userAssignedIdentities/one-hi-sso-external-dns-mi-tf
clientID: f58e7c55-REDACTED-a6e358e53912
YAML
}
resource "kubectl_manifest" "external-dns-mi-binding" {
yaml_body = <<YAML
apiVersion: "aadpodidentity.k8s.io/v1"
kind: AzureIdentityBinding
metadata:
name: external-dns-mi-binding
spec:
AzureIdentity: one-hi-sso-external-dns-mi-tf
Selector: external-dns
YAML
}最佳答案
我正在使用的托管标识未添加到虚拟机规模集 VMSS。添加后，绑定(bind)将起作用并创建 azureAssignedIdentity。
此外 - 我将 AzureIdentity YAML 中的 AzureIdentity 和选择器行从大写首字母转换为小写首字母。
正确: azure 身份: 选择器:
关于azure - 通过 Terraform 和 Helm 添加并安装 Bitnami 外部 DNS 后，NMI pod 中出现错误 - 找不到 pod 的 AzureIdentityBinding，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/67247657/