Spring cloud azure kafka binding with multiple eventhub namespace

Question

Got to know, how to configure spring cloud with eventhub kafka to send and receive message in a same namespace as below https://github.com/MicrosoftDocs/azure-dev-docs/blob/master/articles/java/spring-framework/configure-spring-cloud-stream-binder-java-app-kafka-azure-event-hub.md

However, in my case I need to receive from one eventhub and send it to different namespace, How can I configure ?

Answer 1

You can refer to the multi-binder sample of Spring Cloud Stream Kafka binder, https://github.com/spring-cloud/spring-cloud-stream-samples/tree/main/multi-binder-samples/multi-binder-two-kafka-clusters, it should be the same for working with Event Hubs

Answer 2

When you configure multiple binders, one for each eventhub namespace, kafka detects that the sasl configurations of all the binders are equal. Thus, kafka will only create a LoginManager for the first binder and reuse it for the other binders.

Relevant Code in Kafka Client: https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/security/authenticator/LoginManager.java#L103

As a result, everything works as expected for the first binder. An oauth token can still be acquired for the other binders, but the token belongs to the eventHub of the first namespace. This then leads to a strange error.

Workaround:

You can force the sasl configurations to be unique be adding a configuration that is not needed, but can be used to get a unique config e.g.:

spring:
  cloud:
    stream:
      binders:
        event-hub-1:
          type: kafka
          environment:
            spring.cloud.stream.kafka.binder:
              brokers: "eventhub-1.servicebus.windows.net:9093"
              configuration:
                # only present to make the sasl config unique
                sasl.kerberos.service.name: event-hub-1
        event-hub-2:
          type: kafka
          environment:
            spring.cloud.stream.kafka.binder:
              brokers: "eventhub-2.servicebus.windows.net:9093"
              autoCreateTopics: false
              configuration:
                # only present to make the sasl config unique
                sasl.kerberos.service.name: event-hub-2