I want to be able to test kafka-consumer flows after a canary deployments. Assume we have service which consumes a kafka topic. I have 3 instances in production and 3 partitions on kafka.
At T0 they have the V1 of the application.
- Server #1 (v1)
- Server #2 (v1)
- Server #3 (v1)
I start a canary deploy, and now I have the new version on Server #1
- Server #1 (v2)
- Server #2 (v1)
- Server #3 (v1)
With some routing trick, I can ensure that my API tests are hitting the new instances. But, I couldn't find a feasible way to test kafka-consumer flows for Server #1. There is no guarantee that the test message I emit, will be consumed by Server #1.
I am looking for testing strategies, but I couldn't find a good way without changing the kafka topology, or application logic. I wonder what is your thoughts on this problem.
Kafka doesn't really have such an approach since v2 will likely maintain the same consumer group id, and there's no guarantee it'll consume the same partition as before after a rebalance
You can use embedded Kafka or test containers to do integration tests with Kafka clients, or use builtin MockConsumer. Don't depend on external infrastructure to run your tests