Envoy proxy making more retries than configured

Question

I'm configuring envoy dynamic_forward_proxy. under HttpConnectionManager virtual_hosts i've setup two routes "/hello" and "/bye" respectively. For both of these i've configured retry_policy with num_retries:1 (retry only once), I've also include_request_attempt_count: true to ensure that envoy adds x-envoy-attempt-count header to the upstream when retrying. In my upstream services, i log the requests received, along with x-envoy-attempt-count header. And I notice that envoy consistently makes n x 2 retry requests where n = num_retries.

If i setup num_retries:1 envoy makes 3 requests, 1 with x-envoy-attempt-count=1, and two requests with x-envoy-attempt-coun=2 If i setup num_retries:2 envoy makes 5 requests, 1 with x-envoy-attempt-count=1, and two requests each with x-envoy-attempt-coun=2 and x-envoy-attempt-coun=3, and so on.

I've provided complete envoy.yaml config file along with this post. What am I doing wrong? can anyone please help here?

Config:

admin:
  address:
    socket_address:
      protocol: TCP
      address: 0.0.0.0
      port_value: 9901
static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address:
        protocol: TCP
        address: 0.0.0.0
        port_value: 9001
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: egress_http
          access_log:
          - name: envoy.access_loggers.stdout
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
          codec_type: AUTO
          route_config:
            name: local_route
            virtual_hosts:
            - name: local_service
              domains: ["*"]
              routes:
              - match:
                  prefix: "/hello"
                route:
                  cluster: dynamic_forward_proxy_cluster
                  host_rewrite_header: X-Fwd-HostWithPort
                  retry_policy:
                    retry_on: 5xx
                    num_retries: 1
              - match:
                  prefix: "/bye"
                route:
                  cluster: dynamic_forward_proxy_cluster
                  host_rewrite_header: X-Fwd-HostWithPort
                  retry_policy:
                    retry_on: 5xx
                    num_retries: 1
              include_request_attempt_count: true
          http_filters:
          - name: envoy.filters.http.dynamic_forward_proxy
            typed_config:
              '@type': type.googleapis.com/envoy.extensions.filters.http.dynamic_forward_proxy.v3.FilterConfig
              dns_cache_config:
                name: dynamic_forward_proxy_cache_config
                dns_lookup_family: V4_ONLY
          - name: envoy.filters.http.router
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
  clusters:
  - name: dynamic_forward_proxy_cluster
    lb_policy: CLUSTER_PROVIDED
    cluster_type:
      name: envoy.clusters.dynamic_forward_proxy
      typed_config:
        "@type": type.googleapis.com/envoy.extensions.clusters.dynamic_forward_proxy.v3.ClusterConfig
        dns_cache_config:
          name: dynamic_forward_proxy_cache_config
          dns_lookup_family: V4_ONLY