I am running kafka server using docker-compose with below configuration in node2.
---
version: '3'
services:
zoo1:
image: confluentinc/cp-zookeeper:7.3.2
hostname: zoo1
container_name: zoo1
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
ZOOKEEPER_SERVER_ID: 1
ZOOKEEPER_SERVERS: zoo1:2888:3888
ports:
- "2181:2181"
kafka1:
image: confluentinc/cp-kafka:7.3.2
hostname: kafka1
container_name: kafka1
ports:
# To learn about configuring Kafka for access across networks see
# https://www.confluent.io/blog/kafka-client-cannot-connect-to-broker-on-aws-on-docker-etc/
- "9092:9092"
- "29092:29092"
- "29093:29093"
depends_on:
- zoo1
environment:
KAFKA_BROKER_ID: 1
KAFKA_LISTENERS: INTERNAL://:9092,EXTERNAL_SAME_HOST://:29092,EXTERNAL_DIFFERENT_HOST://:29093
KAFKA_ADVERTISED_LISTENERS: INTERNAL://:9092,EXTERNAL_SAME_HOST://localhost:29092,EXTERNAL_DIFFERENT_HOST://:29093
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,EXTERNAL_SAME_HOST:PLAINTEXT,EXTERNAL_DIFFERENT_HOST:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
KAFKA_ZOOKEEPER_CONNECT: "zoo1:2181"
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
From node1, I am running a kafka client from a docker container, to create topic in kafka broker as shown below and it fails
~ $ ./goflow2 -transport=kafka -transport.kafka.brokers=10.10.26.207:9092 -transport.kafka.topic=flows -format=text
INFO[0000] starting GoFlow2
INFO[0000] starting collection blocking=false count=1 hostname= port=6343 queue_size=1000000 scheme=sflow workers=2
INFO[0000] starting collection blocking=false count=1 hostname= port=2055 queue_size=1000000 scheme=netflow workers=2
ERRO[0089] transport error error="kafka transport kafka: Failed to produce message to topic flows: dial tcp: lookup kafka1 on 10.10.13.22:53: server misbehaving"
ERRO[0104] transport error error="kafka transport kafka: Failed to produce message to topic flows: dial tcp: lookup kafka1 on 10.10.13.22:53: server misbehaving"
^CINFO[0108] closed receiver blocking=false count=1 hostname= port=6343 queue_size=1000000 scheme=sflow workers=2
INFO[0108] closed receiver blocking=false count=1 hostname= port=2055 queue_size=1000000 scheme=netflow workers=2
ERRO[0113] transport error error="kafka transport kafka: Failed to produce message to topic flows: dial tcp: lookup kafka1 on 10.10.13.22:53: server misbehaving"
ERRO[0133] transport error error="kafka transport kafka: Failed to produce message to topic flows: dial tcp: lookup kafka1 on 10.10.13.22:53: server misbehaving"
INFO[0153] closed transporter
INFO[0153] closed HTTP server http=":8080"
ERRO[0153] transport error error="kafka transport kafka: Failed to produce message to topic flows: dial tcp: lookup kafka1 on 10.10.13.22:53: server misbehaving"
From the same node i.e. node1 when I use kafkacat it seems to work fine as shown below
root@mm312:/home/ip# kafkacat -b 10.10.26.207:9092 -L -v
Metadata for all topics (from broker -1: 10.10.26.207:9092/bootstrap):
1 brokers:
broker 1 at kafka1:9092 (controller)
52 topics:
topic "_confluent-controlcenter-7-3-2-1-MonitoringTriggerStore-repartition" with 12 partitions:
partition 0, leader 1, replicas: 1, isrs: 1
partition 1, leader 1, replicas: 1, isrs: 1
partition 2, leader 1, replicas: 1, isrs: 1
partition 3, leader 1, replicas: 1, isrs: 1
partition 4, leader 1, replicas: 1, isrs: 1
partition 5, leader 1, replicas: 1, isrs: 1
partition 6, leader 1, replicas: 1, isrs: 1
partition 7, leader 1, replicas: 1, isrs: 1
partition 8, leader 1, replicas: 1, isrs: 1
partition 9, leader 1, replicas: 1, isrs: 1
partition 10, leader 1, replicas: 1, isrs: 1
partition 11, leader 1, replicas: 1, isrs: 1
topic "_confluent-controlcenter-7-3-2-1-metrics-trigger-measurement-rekey" with 12 partitions:
partition 0, leader 1, replicas: 1, isrs: 1
partition 1, leader 1, replicas: 1, isrs: 1
partition 2, leader 1, replicas: 1, isrs: 1
partition 3, leader 1, replicas: 1, isrs: 1
partition 4, leader 1, replicas: 1, isrs: 1
partition 5, leader 1, replicas: 1, isrs: 1
partition 6, leader 1, replicas: 1, isrs: 1
partition 7, leader 1, replicas: 1, isrs: 1
partition 8, leader 1, replicas: 1, isrs: 1
partition 9, leader 1, replicas: 1, isrs: 1
partition 10, leader 1, replicas: 1, isrs: 1
partition 11, leader 1, replicas: 1, isrs: 1
topic "flows" with 1 partitions:
partition 0, leader 1, replicas: 1, isrs: 1
Any suggestions to troubleshoot this issue would be of great help.