go - kafka_exporter 不适用于计算实例

Question

我尝试按照他们的 github 上的步骤设置 kafka_exporter在运行 Kafka 的计算实例 (CentOS 7) 上。

当我运行以下命令时，我得到“错误初始化 Kafka 客户端”:

/opt/kafkaexporter/kafka_exporter --kafka.server=kafka1:9092 --kafka.version="2.11.0" --tls.enabled  --log.enable-sarama

非常感谢在运行 Kafka 的计算实例上运行 kafka_exporter 的任何建议。

先感谢您，

控制台输出:

/opt/kafkaexporter/kafka_exporter --kafka.server=kafka1:9092 --kafka.version="2.11.0" --tls.enabled  --log.enable-sarama

INFO[0000] Starting kafka_exporter (version=1.2.0, branch=HEAD, revision=830660212e6c109e69dcb1cb58f5159fe3b38903)  source="kafka_exporter.go:474"
INFO[0000] Build context (go=go1.10.3, user=root@981cde178ac4, date=20180707-14:34:48)  source="kafka_exporter.go:475"
[sarama] 2020/03/27 05:47:00 Initializing new client
[sarama] 2020/03/27 05:47:00 client/metadata fetching metadata for all topics from broker kafka1:9092
[sarama] 2020/03/27 05:47:00 Failed to connect to broker kafka1:9092: x509: certificate signed by unknown authority
[sarama] 2020/03/27 05:47:00 client/metadata got error from broker while fetching metadata: x509: certificate signed by unknown authority
[sarama] 2020/03/27 05:47:00 client/metadata no available broker to send metadata request to
[sarama] 2020/03/27 05:47:00 client/brokers resurrecting 1 dead seed brokers
[sarama] 2020/03/27 05:47:00 client/metadata retrying after 250ms... (3 attempts remaining)
[sarama] 2020/03/27 05:47:00 client/metadata fetching metadata for all topics from broker kafka1:9092
[sarama] 2020/03/27 05:47:00 Failed to connect to broker kafka1:9092: x509: certificate signed by unknown authority
[sarama] 2020/03/27 05:47:00 client/metadata got error from broker while fetching metadata: x509: certificate signed by unknown authority
[sarama] 2020/03/27 05:47:00 client/metadata no available broker to send metadata request to
[sarama] 2020/03/27 05:47:00 client/brokers resurrecting 1 dead seed brokers
[sarama] 2020/03/27 05:47:00 client/metadata retrying after 250ms... (2 attempts remaining)
[sarama] 2020/03/27 05:47:01 client/metadata fetching metadata for all topics from broker kafka1:9092

[sarama] 2020/03/27 05:47:01 Failed to connect to broker kafka1:9092: x509: certificate signed by unknown authority
[sarama] 2020/03/27 05:47:01 client/metadata got error from broker while fetching metadata: x509: certificate signed by unknown authority
[sarama] 2020/03/27 05:47:01 client/metadata no available broker to send metadata request to
[sarama] 2020/03/27 05:47:01 client/brokers resurrecting 1 dead seed brokers
[sarama] 2020/03/27 05:47:01 client/metadata retrying after 250ms... (1 attempts remaining)

[sarama] 2020/03/27 05:47:01 client/metadata fetching metadata for all topics from broker kafka1:9092
[sarama] 2020/03/27 05:47:01 Failed to connect to broker kkafka1:9092: x509: certificate signed by unknown authority
[sarama] 2020/03/27 05:47:01 client/metadata got error from broker while fetching metadata: x509: certificate signed by unknown authority
[sarama] 2020/03/27 05:47:01 client/metadata no available broker to send metadata request to
[sarama] 2020/03/27 05:47:01 client/brokers resurrecting 1 dead seed brokers
[sarama] 2020/03/27 05:47:01 Closing Client
ERRO[0000] Error Init Kafka Client                       source="kafka_exporter.go:210"
panic: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)

goroutine 1 [running]:
main.NewExporter(0xc42017a6f0, 0x1, 0x1, 0x100, 0x8f22f8, 0x0, 0x8f22f8, 0x0, 0x1, 0x8f22f8, ...)
  /go/src/github.com/danielqsj/kafka_exporter/kafka_exporter.go:211 +0x847
main.main()
  /go/src/github.com/danielqsj/kafka_exporter/kafka_exporter.go:481 +0x215b

我能够 ping 到 kafka1 实例

Answer 1

在将 Java keystore 和信任库 (JKS) 证书转换为 PEM 并将它们用作 kafka_exporter 的输入(标志)后解决了该问题。 (如 https://medium.com/processone/using-tls-authentication-for-your-go-kafka-client-3c5841f2a625 中所述)

/opt/kafkaexporter/kafka_exporter --kafka.server=kafka1:9092 --tls.enabled --tls.ca-file="/home/kafka_user/serverpub_truststore_go.cer.pem"  --tls.key-file="/home/kafka_user/serverpub_keystore_go.cer.key.pem"  --tls.cert-file="/home/kafka_user/serverpub_keystore_go.cer.pem" --kafka.version="2.11.0" --log.enable-sarama

输出:

INFO[0000] Starting kafka_exporter (version=1.2.0, branch=HEAD, revision=8306602b58f5159fe3b38903)  source="kafka_exporter.go:474"
INFO[0000] Build context (go=go1.10.3, user=root@981ac4, date=20180707-14:34:48)  source="kafka_exporter.go:475"
[sarama] 2020/03/28 01:20:38 Initializing new client
[sarama] 2020/03/28 01:20:38 client/metadata fetching metadata for all topics from broker kafka2:9092
[sarama] 2020/03/28 01:20:38 Connected to broker at kafka2:9092 (unregistered)
[sarama] 2020/03/28 01:20:38 client/brokers registered new broker #2 at kafka2:9092
[sarama] 2020/03/28 01:20:38 client/brokers registered new broker #1 at kafka1:9092
[sarama] 2020/03/28 01:20:38 client/brokers registered new broker #3 at kafka3:9092
[sarama] 2020/03/28 01:20:38 Successfully initialized new client
INFO[0000] Done Init Clients                             source="kafka_exporter.go:213"
INFO[0000] Listening on :9308                            source="kafka_exporter.go:499"