Is the Confluent platform required for IIDR CDC for Kafka?

Question

I have installed IIDR CDC for Kafka 11.4.0. Is it mandatory to use the Confluent package or can I connect directly to my Kafka instance using CDC?

Answer 1

I assume you are looking at Kafka Connect or Debezium-based solutions.

In either case, Confluent Platform packages Apache Kafka, which includes Kafka Connect. There is nothing specfic needed from Confluent Platform unless you wanted to use KSQL / Schema Registry / Control Center, for example, but none are required for CDC.

Answer 2

Hi: Confluent Avro Binary is the default message format, but you can use one of the Kafka Custom Operation Processor (KCOP) provided with IIDR CDC to write to Kafka without using Confluent Avro Binary Format. If none of the provided KCOP satisfies your requirements, their java source code are provided and you can modify them to your needs or create your own.

The list of provided KCOP are in: https://ibm.biz/BdqBvW

Answer 3

Confluent is a good option, Some clients they have clouderra, Event streams etc., But you may have to configure the schema registry. As Daniel mentioned you can also use one of the Kafka Custom Operation Processor (KCOP) provided with IIDR CDC to write to Kafka without using Confluent Avro Binary Format.