Kafka producer producing values in alternative even partitions like 0,2,4...etc in org.apache.kafka.clients.producer.RoundRobinPartitioner
I have 6 partitions & their are 6 concurrent consumer to those & following config org.apache.kafka.clients.producer.RoundRobinPartitioner. While producing the messages kafka only puts the messages to 3 partitions on alternative way. which is leaving the other partitions not being utilized & messages in queue for 6 concurrent calls as only 3 are being used.
How can i achieve equal distribution? Is there is any config i'm missing?
props.put(partitioner.class, "org.apache.kafka.clients.producer.RoundRobinPartitioner");
I'm using org.springframework.kafka:spring-kafka jar. And instead of providing key passing null key as in producer record. ie. Not passing any key just passing dto. Below is the producer config :-
@Bean
public KafkaTemplate<String, Dto> kafkaTemplate() {
return new KafkaTemplate<>(producerFactory());
}
@Bean
public ProducerFactory<String, Dto> dtoProducerFactory() {
Map<String, Object> props = getProducerFactoryConfigMap();
return new DefaultKafkaProducerFactory<>(props);
}
private Map<String, Object> getProducerFactoryConfigMap() {
Map<String, Object> props = new HashMap<>();
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, kafkaConfig.getBootStrapServers());
props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, JsonSerializer.class);
props.put(ProducerConfig.PARTITIONER_CLASS_CONFIG, "org.apache.kafka.clients.producer.RoundRobinPartitioner");;
kafkaConfig.populateSaslProperties(props);
props.put(ProducerConfig.ACKS_CONFIG, PRODUCER_ACK_CONFIG);
return props;
}