sql-server - Debezium:无法在 Kafka-Connect Docker 容器上设置观察者。获取关联 ID 为 2 的元数据时出错... LEADER_NOT_AVAILABLE

Question

问题:

如何设置观察者以在对数据库进行更改时实时观察 Kafka-Connect 流？我收到 Error while fetching metadata with correlation id 2 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} .是什么导致了问题，我该如何解决？

上下文:

这个问题与:Debezium How do I correctly register the SqlServer connector with Kafka Connect - connection refused有关和
https://github.com/debezium/debezium-examples/blob/master/tutorial/README.md#using-sql-server

在 Windows 10 中，我使用以下命令在各自独立的 Docker 容器中设置了 Zookeeper、Kafka 和 Kafka-Connector:

docker run -it --rm --name zookeeper -p 2181:2181 -p 2888:2888 -p 3888:3888 debezium/zookeeper:1.1
docker run -it --rm --name kafka -p 9092:9092 -e KAFKA_ADVERTISED_HOST_NAME: kafka --link zookeeper:zookeeper debezium/kafka:1.1
docker run -it --rm --name connect -p 8083:8083 -e GROUP_ID=1 -e CONFIG_STORAGE_TOPIC=my_connect_configs -e OFFSET_STORAGE_TOPIC=my_connect_offsets -e STATUS_STORAGE_TOPIC=my_connect_statuses --link zookeeper:zookeeper --link kafka:kafka debezium/connect:1.1

然后，我将 Kafka 连接器设置为在 Docker 容器外部的本地主机上运行的 SQL Server 实例。日志显示连接器可以连接到数据库并获取 CDC 数据。到目前为止，我还好。

但是，当我尝试创建观察者时，使用以下命令:

docker run -it --rm --name watcher --link zookeeper:zookeeper --link kafka:kafka debezium/kafka:1.1 

我收到以下错误。

watch-topic -a -k dbhistory.Test.Posts
WARNING: Using default BROKER_ID=1, which is valid only for non-clustered installations.
Using ZOOKEEPER_CONNECT=172.17.0.2:2181
Using KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://172.17.0.5:9092
Using KAFKA_BROKER=172.17.0.3:9092
Contents of topic dbhistory.Test.Posts:
[2020-04-08 15:18:39,373] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 2 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:20,501] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:20,605] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:20,807] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:21,262] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:22,120] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:23,330] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:24,189] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:59,826] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:04,051] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:07,121] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:08,024] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:09,354] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11570 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:09,458] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11572 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:09,571] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11574 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:09,677] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11576 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:09,786] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11578 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:09,895] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11580 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,002] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11582 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,112] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11584 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,220] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11586 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,326] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11588 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,435] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11590 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,537] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11592 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,648] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11594 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,751] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11596 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,854] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11598 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 17:08:48,268] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 17:09:55,602] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)

Answer 1

该问题很可能是由您定义环境变量的方式引起的。
代替:

-e KAFKA_ADVERTISED_HOST_NAME: kafka

和:

-e KAFKA_ADVERTISED_HOST_NAME=kafka