How to use kafka broker filtering at envoy?

Question

I'm deploying a 3 replica Kafka using a Envoy proxy as a gateway in a VM environment (Ubuntu 22.04 LTS). The replicated Kafka works fine, but now I am trying to deploy the envoy using this documentation: https://www.envoyproxy.io/docs/envoy/latest/configuration/listeners/network_filters/kafka_broker_filter

This is the config.yaml result for my use case:

static_resources:
  listeners:
    - address:
        socket_address:
          address: xxx.xxx.xxx.xxx
          port_value: 10000
      filter_chains:
      - filters:
        - name: envoy.filters.network.kafka_broker
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.network.kafka_broker.v3.KafkaBroker
            stat_prefix: exampleprefix1
            id_based_broker_address_rewrite_spec:
              rules:
              - id: 1
                host: xxx.xxx.xxx.xxx
                port: 9092
              - id: 2
                host: yyy.yyy.yyy.yyy
                port: 9092
              - id: 3
                host: zzz.zzz.zzz.zzz
                port: 9092
        - name: envoy.filters.network.tcp_proxy
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
            stat_prefix: tcp
            cluster: broker_1

    - address:
        socket_address:
          address: yyy.yyy.yyy.yyy
          port_value: 9092
      filter_chains:
      - filters:
        - name: envoy.filters.network.kafka_broker
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.network.kafka_broker.v3.KafkaBroker
            stat_prefix: exampleprefix2
            id_based_broker_address_rewrite_spec:
              rules:
              - id: 1
                host: xxx.xxx.xxx.xxx
                port: 9092
              - id: 2
                host: yyy.yyy.yyy.yyy
                port: 9092
              - id: 3
                host: zzz.zzz.zzz.zzz
                port: 9092
        - name: envoy.filters.network.tcp_proxy
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
            stat_prefix: tcp
            cluster: broker_2

    - address:
        socket_address:
          address: zzz.zzz.zzz.zzz
          port_value: 9092
      filter_chains:
      - filters:
        - name: envoy.filters.network.kafka_broker
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.network.kafka_broker.v3.KafkaBroker
            stat_prefix: exampleprefix2
            id_based_broker_address_rewrite_spec:
              rules:
              - id: 1
                host: xxx.xxx.xxx.xxx
                port: 9092
              - id: 2
                host: yyy.yyy.yyy.yyy
                port: 9092
              - id: 3
                host: zzz.zzz.zzz.zzz
                port: 9092
        - name: envoy.filters.network.tcp_proxy
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
            stat_prefix: tcp
            cluster: broker_3

  clusters:
    - name: broker_1
      connect_timeout: 0.25s
      type: STATIC
      lb_policy: ROUND_ROBIN
      load_assignment:
        cluster_name: kafka_cluster
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: xxx.xxx.xxx.xxx
                      port_value: 9092

    - name: broker_2
      connect_timeout: 0.25s
      type: STATIC
      lb_policy: ROUND_ROBIN
      load_assignment:
        cluster_name: kafka_cluster
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: yyy.yyy.yyy.yyy
                      port_value: 9092

    - name: broker_3
      connect_timeout: 0.25s
      type: STATIC
      lb_policy: ROUND_ROBIN
      load_assignment:
        cluster_name: kafka_cluster
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: zzz.zzz.zzz.zzz
                      port_value: 9092

admin:
  access_log_path: /dev/null
  profile_path: /dev/null
  address:
    socket_address: { address: 127.0.0.1, port_value: 9901 }

This is the received error:

[2024-03-03 14:01:46.442][587067][info][main] [external/envoy/source/server/server.cc:334]   envoy.matching.http.input: request-headers, request-trailers, response-headers, response-trailers
[2024-03-03 14:01:46.442][587067][info][main] [external/envoy/source/server/server.cc:334]   envoy.http.cache: envoy.extensions.http.cache.simple
[2024-03-03 14:01:46.454][587067][critical][main] [external/envoy/source/server/server.cc:113] error initializing configuration 'envoy-config.yaml': Protobuf message (type envoy.config.bootstrap.v3.Bootstrap reason INVALID_ARGUMENT:id_based_broker_address_rewrite_spec: Cannot find field.) has unknown fields
[2024-03-03 14:01:46.455][587067][info][main] [external/envoy/source/server/server.cc:815] exiting
Protobuf message (type envoy.config.bootstrap.v3.Bootstrap reason INVALID_ARGUMENT:id_based_broker_address_rewrite_spec: Cannot find field.) has unknown fields

I have searched about it, but it seems pretty new functionality at Envoy, so I didn't find anything.

Answer 1

The official Envoy install tutorial gets an old version of Envoy (I'm my case installed a v1.18, the current one is v1.29).

So, I downloaded Docker and created a docker compose using envoyproxy/envoy:contrib-v1.29.1. Until now, it seems to work fine.

Obs: The contrib version is important according to https://github.com/envoyproxy/envoy/issues/21193#issuecomment-1120332571