Kafka Producer Connection Issue

1.6k Views Asked by At

I am trying send messages through kafka producer but getting following error. I am using following command to establish connection.

export KAFKA_OPTS="-Djava.security.auth.login.config=$OSI_HOME/jaas.conf -Djava.security.krb5.conf=$OSI_HOME/krb5.conf"$OSI_HOME/Custom/confluent/bin/kafka-console-producer --topic testTopic --bootstrap-server <server list> --producer.config producer.properties

Here's the JAAS config I am using

KafkaClient { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true storeKey=true keyTab="[Location of keytabfile]" serviceName="kafka" useTicketCache=true principal="svc_account@REALM"; };

I can see producer was able to connect. But when I try to send messages I get following error.

[2022-02-23 12:45:56,562] WARN [Producer clientId=console-producer] Connection to node -7 terminated during authentication. This may happen due to any of the following reasons: (1) Authentication failed due to invalid credentials with brokers older than 1.0.0, (2) Firewall blocking Kafka TLS traffic (eg it may only allow HTTPS traffic), (3) Transient network issue. (org.apache.kafka.clients.NetworkClient)
[2022-02-23 12:45:56,563] WARN [Producer clientId=console-producer] Bootstrap broker  (id: -7 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2022-02-23 12:45:56,707] WARN [Producer clientId=console-producer] Connection to node -13 terminated during authentication. This may happen due to any of the following reasons: (1) Authentication failed due to invalid credentials with brokers older than 1.0.0, (2) Firewall blocking Kafka TLS traffic (eg it may only allow HTTPS traffic), (3) Transient network issue. (org.apache.kafka.clients.NetworkClient)
[2022-02-23 12:45:56,707] WARN [Producer clientId=console-producer] Bootstrap broker  (id: -13 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2022-02-23 12:45:56,863] WARN [Producer clientId=console-producer] Connection to node -12 terminated during authentication. This may happen due to any of the following reasons: (1) Authentication failed due to invalid credentials with brokers older than 1.0.0, (2) Firewall blocking Kafka TLS traffic (eg it may only allow HTTPS traffic), (3) Transient network issue. (org.apache.kafka.clients.NetworkClient)
[2022-02-23 12:45:56,863] WARN [Producer clientId=console-producer] Bootstrap broker (id: -12 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)

Does anyone know how to resolve this? I tried telnet command for broker list and it's connected so it doesn't seem to be a firewall issue. I can also do kinit from the server so doesn't seem to be an authentication issue.

1

There are 1 best solutions below

1
user18349017 On

git clone https://gerrit.googlesource.com/gerrit-release-tools && (cd gerrit-release-tools && f=git rev-parse --git-dir/hooks/commit-msg ; mkdir -p $(dirname $f) ; curl -Lo $f https://gerrit-review.googlesource.com/tools/hooks/commit-msg ; chmod +x $f) enter code here