Creating a topic on startup of Kafka container - connection to node could not be established

Question

I'm using the Docker compose file below to deploy a simple Kafka server in a container. I also deploy a Nodejs application that bootstraps a consumer on the topic 'my-topic'. So I need the topic to be available when I run the docker compose up command.

I keep getting the "Connection to node" errors in the Kafka container logs indicating that my simple kafka-topics.sh script is failing. Could anyone provide some insight on the issue, or suggest some workarounds?

Compose file:

services:
  nodeapp:
    build: .
    container_name: nodeapp
    depends_on:
      - kafka
  zookeeper:
    image: 'bitnami/zookeeper:latest'
    container_name: zookeeper
    ports:
      - '2181:2181'
    networks:
      - kafka
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
  kafka:
    image: 'bitnami/kafka:3.4.0-debian-11-r21'
    container_name: kafka
    command:
      ["kafka-topics.sh", "--bootstrap-server", "localhost:9092", "--topic", "my-topic", "--create"]
    ports:
      - '9093:9093'
    networks:
      - kafka
    environment:
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CONTAINER:PLAINTEXT
      - KAFKA_CFG_LISTENERS=CONTAINER://:9092
      - KAFKA_CFG_ADVERTISED_LISTENERS=CONTAINER://localhost:9092
      - KAFKA_CFG_INTER_BROKER_LISTENER_NAME=CONTAINER
    depends_on:
      - zookeeper
networks:
  kafka:
    name: kafka-network

Logs:

kafka      | WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.
kafka      | [2023-05-03 22:25:30,465] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
kafka      | [2023-05-03 22:25:30,577] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
kafka      | [2023-05-03 22:25:30,780] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
kafka      | [2023-05-03 22:25:30,986] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
kafka      | [2023-05-03 22:25:31,389] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
kafka      | [2023-05-03 22:25:32,195] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
kafka      | [2023-05-03 22:25:33,404] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
kafka      | [2023-05-03 22:25:34,614] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
kafka      | [2023-05-03 22:25:35,823] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)

I've tried changing the advertised listeners environment variables but could not get a connection established.

I've confirmed that I can create the topic by using bash on the running Kafka container IF I remove the command that attempts to create the topic from the compose file. This command works for that use case:

docker exec -it /bin/bash kafka
kafka-topics.sh --bootstrap-server localhost:9092 --topic my-topic --create

Please note that the issue above is not the same as this one posted by Sasha Shpota. Sasha's issue is the host machine not being able to resolve the domain. My issue is that the command in the compose file, which should be running in the container, is throwing an error and may be the root cause of the issue.

Note that if I remove the command in the compose file that creates the topic, I'm able to use the docker exec command I mentioned above to create the topic. However, when that line is present in the compose file I get the connection error when executing the same docker exec command.