apache-kafka - NotEnoughReplicasException : The size of the current ISR Set(2) is insufficient to satisfy the min. isr 要求 3

标签 apache-kafka confluent-platform

我有以下设置 经纪人:3 - 全部启动并运行 min.insync.replicas=3。

我创建了一个topic,配置如下

bin\windows\kafka-topics --zookeeper 127.0.0.1:2181 --topic topic-ack-all --create --partitions 4 --replication-factor 3

我用“ack = all”触发了生产者,生产者能够发送消息。 但是,当我启动消费者时,问题就开始了

bin\windows\kafka-console-consumer --bootstrap-server localhost:9094,localhost:9092 --topic topic-ack-all --from-beginning

错误是

NotEnoughReplicasException: The size of the current ISR Set(2) is insufficient to satisfy the min.isr requirement of 3 NotEnoughReplicasException:The size of the current ISR Set(3) is insufficient to satisfy the min.isr requirement of 3 for partition __con

我在这里看到两种错误。我浏览了文档并且也了解了“min.isr”,但是,这些错误消息并不清楚。

  1. 当前 ISR 集是什么意思?每个主题是否有所不同及其含义?
  2. 我猜 min.isr 和 min.insync.replicas 一样。我希望它至少应该与“复制因子”具有相同的值(value)?

更新 #1

Topic: topic-ack-all    PartitionCount: 4       ReplicationFactor: 3    Configs:            
        Topic: topic-ack-all    Partition: 0    Leader: 1       Replicas: 1,2,3 Isr: 1,2,3  
        Topic: topic-ack-all    Partition: 1    Leader: 1       Replicas: 2,3,1 Isr: 1,2,3  
        Topic: topic-ack-all    Partition: 2    Leader: 1       Replicas: 3,1,2 Isr: 1,2,3  
        Topic: topic-ack-all    Partition: 3    Leader: 1       Replicas: 1,3,2 Isr: 1,2,3 

更新 2

Topic: __consumer_offsets       PartitionCount: 50      ReplicationFactor: 1    Configs: compression.type=producer,cleanup.policy=compact,segment.bytes=104857600
    Topic: __consumer_offsets       Partition: 0    Leader: 2       Replicas: 2     Isr: 2
    Topic: __consumer_offsets       Partition: 1    Leader: 3       Replicas: 3     Isr: 3
    Topic: __consumer_offsets       Partition: 2    Leader: 1       Replicas: 1     Isr: 1
    Topic: __consumer_offsets       Partition: 3    Leader: 2       Replicas: 2     Isr: 2
    Topic: __consumer_offsets       Partition: 4    Leader: 3       Replicas: 3     Isr: 3
    Topic: __consumer_offsets       Partition: 5    Leader: 1       Replicas: 1     Isr: 1
    Topic: __consumer_offsets       Partition: 6    Leader: 2       Replicas: 2     Isr: 2
    Topic: __consumer_offsets       Partition: 7    Leader: 3       Replicas: 3     Isr: 3
    Topic: __consumer_offsets       Partition: 8    Leader: 1       Replicas: 1     Isr: 1
    Topic: __consumer_offsets       Partition: 9    Leader: 2       Replicas: 2     Isr: 2
    Topic: __consumer_offsets       Partition: 10   Leader: 3       Replicas: 3     Isr: 3
    Topic: __consumer_offsets       Partition: 11   Leader: 1       Replicas: 1     Isr: 1
    Topic: __consumer_offsets       Partition: 12   Leader: 2       Replicas: 2     Isr: 2
    Topic: __consumer_offsets       Partition: 13   Leader: 3       Replicas: 3     Isr: 3
    Topic: __consumer_offsets       Partition: 14   Leader: 1       Replicas: 1     Isr: 1
    Topic: __consumer_offsets       Partition: 15   Leader: 2       Replicas: 2     Isr: 2
    Topic: __consumer_offsets       Partition: 16   Leader: 3       Replicas: 3     Isr: 3
    Topic: __consumer_offsets       Partition: 17   Leader: 1       Replicas: 1     Isr: 1
    Topic: __consumer_offsets       Partition: 18   Leader: 2       Replicas: 2     Isr: 2
    Topic: __consumer_offsets       Partition: 19   Leader: 3       Replicas: 3     Isr: 3
    Topic: __consumer_offsets       Partition: 20   Leader: 1       Replicas: 1     Isr: 1
    Topic: __consumer_offsets       Partition: 21   Leader: 2       Replicas: 2     Isr: 2
    Topic: __consumer_offsets       Partition: 22   Leader: 3       Replicas: 3     Isr: 3
    Topic: __consumer_offsets       Partition: 23   Leader: 1       Replicas: 1     Isr: 1
    Topic: __consumer_offsets       Partition: 24   Leader: 2       Replicas: 2     Isr: 2
    Topic: __consumer_offsets       Partition: 25   Leader: 3       Replicas: 3     Isr: 3
    Topic: __consumer_offsets       Partition: 26   Leader: 1       Replicas: 1     Isr: 1
    Topic: __consumer_offsets       Partition: 27   Leader: 2       Replicas: 2     Isr: 2
    Topic: __consumer_offsets       Partition: 28   Leader: 3       Replicas: 3     Isr: 3
    Topic: __consumer_offsets       Partition: 29   Leader: 1       Replicas: 1     Isr: 1
    Topic: __consumer_offsets       Partition: 30   Leader: 2       Replicas: 2     Isr: 2
    Topic: __consumer_offsets       Partition: 31   Leader: 3       Replicas: 3     Isr: 3
    Topic: __consumer_offsets       Partition: 32   Leader: 1       Replicas: 1     Isr: 1
    Topic: __consumer_offsets       Partition: 33   Leader: 2       Replicas: 2     Isr: 2
    Topic: __consumer_offsets       Partition: 34   Leader: 3       Replicas: 3     Isr: 3
    Topic: __consumer_offsets       Partition: 35   Leader: 1       Replicas: 1     Isr: 1
    Topic: __consumer_offsets       Partition: 36   Leader: 2       Replicas: 2     Isr: 2
    Topic: __consumer_offsets       Partition: 37   Leader: 3       Replicas: 3     Isr: 3
    Topic: __consumer_offsets       Partition: 38   Leader: 1       Replicas: 1     Isr: 1
    Topic: __consumer_offsets       Partition: 39   Leader: 2       Replicas: 2     Isr: 2
    Topic: __consumer_offsets       Partition: 40   Leader: 3       Replicas: 3     Isr: 3
    Topic: __consumer_offsets       Partition: 41   Leader: 1       Replicas: 1     Isr: 1
    Topic: __consumer_offsets       Partition: 42   Leader: 2       Replicas: 2     Isr: 2
    Topic: __consumer_offsets       Partition: 43   Leader: 3       Replicas: 3     Isr: 3
    Topic: __consumer_offsets       Partition: 44   Leader: 1       Replicas: 1     Isr: 1
    Topic: __consumer_offsets       Partition: 45   Leader: 2       Replicas: 2     Isr: 2
    Topic: __consumer_offsets       Partition: 46   Leader: 3       Replicas: 3     Isr: 3
    Topic: __consumer_offsets       Partition: 47   Leader: 1       Replicas: 1     Isr: 1
    Topic: __consumer_offsets       Partition: 48   Leader: 2       Replicas: 2     Isr: 2
    Topic: __consumer_offsets       Partition: 49   Leader: 3       Replicas: 3     Isr: 3

最佳答案

从您粘贴的输出来看,__consumer_offsets 主题似乎是使用单个副本创建的。

您的代理似乎也设置了 min.insync.replicas = 3。

使用此配置,如果消费者提交偏移量,预计会得到 NotEnoughReplicasException

__consumer_offsets 主题在第一个消费者连接到集群时自动创建。结束你的情况的一种常见方法是,如果你在只有一个代理运行的情况下运行一个消费者。在这种情况下,__consumer_offsets 将使用单个副本创建。

假设这是一个开发环境,回到有效状态的最简单方法是删除 __consumer_offsets 并在所有 3 个代理都启动时重新运行您的消费者。

关于apache-kafka - NotEnoughReplicasException : The size of the current ISR Set(2) is insufficient to satisfy the min. isr 要求 3,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/62770272/

相关文章:

json - Kafka JDBC Sink Connector 对于具有可选字段的模式的消息给出空指针异常

jdbc - kafka 连接 JDBC 接收器。展平 JSON 记录时出错

apache-kafka - 从主题的两个日期时间之间获取 Kafka 消息?

apache-kafka - 要终止哪个进程才能停止 Kafka Connect Worker?

docker - 融合Kafka和docker-compose-错误运行示例

apache-kafka - kafka中的消费群体有什么需求?

amazon-web-services - 动物园管理员-Kafka : ConnectException - Connection refused

docker - kafka-avro-console-consumer 命令在 `confluentinc/cp-enterprise-kafka` 中不可用

load-balancing - apache kafka 中的负载均衡

java - 如何在kafka中创建自定义序列化器?