我正在使用kafka_2.11-2.2.1。我在 server.properties 中有以下设置。
message.max.bytes=20971520
replica.fetch.max.bytes=23068672
代码级别的生产者配置。
@Bean("serviceKafkaTemplate")
public KafkaTemplate kafkaTemplate(@Autowired final ProducerFactory producerFactory) {
return new KafkaTemplate(producerFactory);
}
@Bean
public ProducerFactory<String, Object> producerFactory() {
return new DefaultKafkaProducerFactory(props());
}
private Properties props() {
Properties properties = new Properties();
properties.put(ProducerConfig.COMPRESSION_TYPE_CONFIG, "gzip");
properties.put(ProducerConfig.REQUEST_TIMEOUT_MS_CONFIG, 10000);
properties.put(ProducerConfig.BATCH_SIZE_CONFIG, 1);
properties.put(ProducerConfig.LINGER_MS_CONFIG, 1);
properties.put(ProducerConfig.MAX_REQUEST_SIZE_CONFIG, 15728640);
properties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
properties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
return properties;
}
@Autowired
private KafkaTemplate serviceKafkaTemplate;
serviceKafkaTemplate.send("topic_name", data);
我只谈论生产者而不是消费者。当我通过 cmd 检查偏移量时,它没有增加,这意味着生产者没有发送数据。如果数据大小小于 1 MB,则它可以正常工作,当前大小为 1.75 MB。 kafka server.log 文件中也没有与此相关的内容。如果我遗漏任何配置,请告诉我。
最佳答案
请看一下房子
max.request.size=1048576
当前文档中的描述:
The maximum size of a request in bytes. This setting will limit the number of record batches the producer will send in a single request to avoid sending huge requests. This is also effectively a cap on the maximum record batch size. Note that the server has its own cap on record batch size which may be different from this.
认为这应该对您有帮助。另请查看文档。
关于java - KafkaProducer 未生成> 1 MB 的消息到主题,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/58659442/