apache-spark - 不受控制的代理关闭后，Kafka 分区领导者选举失败

Question

我们有 3 个 kafka broker 和主题，有 40 个分区，复制因子设置为 1。在某些分区不受控制的 kafka broker 关闭后，我们发现无法选出新的领导者(见下面的日志)。最终我们无法阅读主题。 请告知，是否有可能在不将复制因子更改为大于 1 的情况下幸免于此类崩溃。

我们希望目标数据库具有一致的状态(基于来自 kafka 主题的事件创建)，因此我们还将参数 unclean.leader.election.enable 设置为 false。

崩溃后的分区信息:

extenr-topic:1:882091242
extenr-topic:19:882091615
extenr-topic:28:882092273
Error: partition 18 does not have a leader. Skip getting offsets
Error: partition 27 does not have a leader. Skip getting offsets
Error: partition 36 does not have a leader. Skip getting offsets

kafka 代理异常:

2017-10-09 05:56:50,302 ERROR state.change.logger: Controller 236 epoch 267 initiated state change for partition [extenr-topic,15] from OfflinePartition to OnlinePartition failed
kafka.common.NoReplicaOnlineException: No broker in ISR for partition [extenr-topic,15] is alive. Live brokers are: [Set(236, 237)], ISR brokers are: [235]
at kafka.controller.OfflinePartitionLeaderSelector.selectLeader(PartitionLeaderSelector.scala:66)
at kafka.controller.PartitionStateMachine.electLeaderForPartition(PartitionStateMachine.scala:342)
at kafka.controller.PartitionStateMachine.kafka$controller$PartitionStateMachine$$handleStateChange(PartitionStateMachine.scala:203)
at kafka.controller.PartitionStateMachine$$anonfun$triggerOnlinePartitionStateChange$3.apply(PartitionStateMachine.scala:118)
at kafka.controller.PartitionStateMachine$$anonfun$triggerOnlinePartitionStateChange$3.apply(PartitionStateMachine.scala:115)
at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:733)
at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)

日志中也有如下错误

2017-10-09 04:11:25,509 ERROR state.change.logger: Broker 235 received LeaderAndIsrRequest with correlation id 1 from controller 236 epoch 267 for partition [extenr-topic,36] but cannot become follower since the new leader -1 is unavailable.

Answer 1

replication.factor 为 1 的分区将在其领导者崩溃/关闭时变为离线，因为没有其他可用的副本可以接管。

如果可用性对您很重要，我建议增加复制因子。高可用性的推荐配置 [1] 是 replication.factor 设置为 3，min.insync.replicas 设置为 2。

1:http://kafka.apache.org/documentation/#brokerconfigs