Can not connect to the Kafka broker from a Kafka consumer pod in the same namespace

Question

Problem

I am trying to connect to a Kafka pod from a different Kafka consumer pod in the same namespace but the consumer can't connect to the broker.

I am seeing this error in the consumer pod logs

[2024-02-28 16:51:02,193] WARN [Consumer clientId=consumer-console-consumer-81790-1, groupId=console-consumer-81790] Connection to node 0 (localhost/127.0.0.1:19092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2024-02-28 16:51:03,098] WARN [Consumer clientId=consumer-console-consumer-81790-1, groupId=console-consumer-81790] Connection to node 0 (localhost/127.0.0.1:19092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)

My Questions

What am I missing in the K8s config? Any pointers to troubleshoot? I have shared multiple things that I have tried below.

Context

My Kafka yaml file

---
apiVersion: v1
kind: Pod
metadata:
  name: kafka
  labels:
    app: kafka
    service: kafka
  namespace: datastores
spec:
  containers:
    - image: xxx.xxx.xx/images/kafka:latest
      name: kafka
      ports:
        - containerPort: 19092
      environment:
        # These are some failed trial attempts to get the kafka pod to listen on localhost
        - name: KAFKA_ADVERTISED_LISTENERS
          value: PLAINTEXT://127.0.0.1:19092,PLAINTEXT_HOST://localhost:19092,CONNECTIONS_FROM_HOST://localhost:19092
        - name: KAFKA_LISTENER_SECURITY_PROTOCOL_MAP
          value: PLAINTEXT:PLAINTEXT,CONNECTIONS_FROM_HOST:PLAINTEXT

      resources: {}
  restartPolicy: Always
status: {}
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: kafka
    service: kafka
  name: kafka
  namespace: datastores
spec:
  ports:
    - name: "19092"
      port: 19092
      targetPort: 19092
  selector:
    app: kafka
status:
  loadBalancer: {}

My kafka-consumer yaml file

---
apiVersion: v1
kind: Pod
metadata:
  name: kafka-consumer-test
  labels:
    app: kafka-consumer-test
  namespace: datastores
spec:
  containers:
  - name: kafka-consumer
    image: xxx.xxx.xx/images/kafka:latest
    command: ["/bin/bash", "-c"]
    args:
      - |
        kafka-console-consumer.sh \
          --bootstrap-server localhost:19092 \
          --topic my-test-topic \
          --from-beginning
  restartPolicy: Never

What I have tried so far

1. Setting listeners=PLAINTEXT://127.0.0.1:9092 in server.properties as suggested in this answer.
2. Tried using these values for --bootstrap-server kafka.datastores.svc.cluster.local:19092 or --bootstrap-server kafka:19092
3. set advertised.listeners as suggested in this answer. You can see the alternative option to set this via the config in the yaml file

Lemme know if any additional information is required. TIA!