I am working on a Integration layer, where I want to use Kafka Connect between Kafka Topics to achieve dynamic routing.
I want to know if it is possible to use Kafka Connect to connect between 2 Topics (where Kafka Connect act consumer and then producer) and achieve dynamic routing between topics using Message header without Kafka Stream.
As Kafka Stream also act as a consumer and needs to run on the client application (outside Kafka Broker and Kafka Connect). While I want Integration Layer have all the routing internally (Kafka Connect + Kafka Routing).
This is exactly what MirrorMaker2 is, however it is not recommended to use it within the same cluster without disabling its internal state management of heartbeats and offset syncs
Apache Camel connector with Kafka source and sink tasks may work better.
If you just don't want to write Kafka Steams code, ksqlDB is another option that abstracts that away