Is it possible to control how often a Spring Kafka Message Listener switches between its assigned partitions?

Question

When a Spring Kafka MessageListener is consuming messages from multiple partitions, it keeps processing messages from one partition until there are no more messages and only after that it continues with the next partition. (based on my observations)

Is it possible to set a max number of messages/batches and tell the Listener to switch faster to the next partition rather than later?

This would improve fairness and consume evenly from all assigned partitions.

Answer 1

switch faster to the next partition, consume evenly from all assigned partitions

I don't think Kafka has any properties for this. kafka consumer config

It's weird. You could see a partition replica in Kafka as a log file. Your consumer poll runs in one thread, for better performance, it should consume from one file, and the next poll will consume from another file rather than separate it and consume evenly from many partitions for each poll, right? Eventually, you still need to consume all of the messages on the topic.