Mirrormaker: Not authorized to access topics

33 Views Asked by At

I've installed the redpanda on 2 EKS clusters thosr are in us-east-1 and us-east-2 regions. I've enabled the replication from us-east-2 cluster to us-east-1 cluster. I've configured kafka mirrormaker on us-east-2 region's EKS cluster using helm chart. While checking the logs of mirrormaker pod, I am getting the below error.

[2024-02-05 06:35:40,838] ERROR [Worker clientId=connect-2, groupId=redpanda-us-east-1-mm2] Uncaught exception in herder work thread, exiting:  (org.apache.kafka.connect.runtime.distributed.DistributedHerder:334)
org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [mm2-offsets.redpanda-us-east-1.internal]

My mm2.properties file.

#Primary and Secondary Redpanda Cluster Names
clusters=redpanda-us-east-2, redpanda-us-east-1

source.cluster.alias=redpanda-us-east-2
target.cluster.alias=redpanda-us-east-1

#Comma Saperated list of bootstrap servers for source Redpanda cluster
redpanda-us-east-2.bootstrap.servers = <DNS/IP>

#Comma Saperated list of bootstrap servers for target Redpanda cluster
redpanda-us-east-1.bootstrap.servers = <DNS/IP>

#Enable replication from Source Redpanda Cluster to Target Redpanda Cluster.
redpanda-us-east-2->redpanda-us-east-1.enabled = true

#Replicate all the topics from Source Redpanda Cluster to Target Redpanda Cluster
topics = .*

#Setting replication factor of newly created remote topics
replication.factor=3

sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
    username="<username>" \
    password="<password>";

sasl.mechanism=SCRAM-SHA-512

security.protocol=SASL_PLAINTEXT

security.inter.broker.protocol=SASL_PLAINTEXT

ssl.enabled.protocols = ""

ssl.protocol = ""

ssl.endpoint.identification.algorithm = ""

auto.create.topics.enable = true

I've tried to create the topic manualy using the rpk create topic command with the same username & password and it got created but the mirror-maker is not able to create the topic.

After the above error, I am seeing the below error as well the in the logs.

org.apache.kafka.connect.errors.ConnectException: Worker is shutting down
    at org.apache.kafka.connect.runtime.distributed.DistributedHerder.halt(DistributedHerder.java:712)
    at org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:329)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
0

There are 0 best solutions below