Is there functionality in the akka-Kafka (scala framework) to list all available topics

Question

I am trying to check if a topic passed to my "start stream" method is a valid/already existing topic in the Kafka that my program is connected to.

I know Java has the KafkaConsumer.ListTopics, but I'm working with the akka-kafka library and it seems the ConsumerSetting doesn't have the same method. I could have code that executes a script to list the kafka topics using the Kafka-topics --list command, but I would prefer a less janky way than that.

Answer 1

You would have to create the KafkaConsumer from that Settings object, then you can use the API methods you mentioned.

You should not expose Zookeeper directly to unauthenticated clients.

Answer 2

The best way for that is by getting the topics list from zookeeper, like this:

 import org.apache.zookeeper.ZooKeeper;

 val zk = new ZooKeeper("localhost:2181", 10000, null)
 val topics = zk.getChildren("/brokers/topics", false)