Got a 400 error when redirecting a WS connection to another backend

Question

I want to forward a Web Socket (WS) connection from a domain-1 to domain-2.

The goal is that domain-1 uses weak cypher algos, TLS 1.2 and shorter URL and forward to domain2 which has TLS 1.3 only strong cypher algos and a longer domain name.

Here is the flow:

[![Web Socket flow][1]][1] [1]: https://i.stack.imgur.com/WzFMO.png

I'm getting a 400 error in step (3)

Log of NGINX in domain-1:

2023/10/06 13:17:01 [notice] 1152#1152: *37717323 "(?i)/std(/|$)(.*)" matches "/std/base/key1/key2/name1", client: 10.0.179.122, server: domain-1, request: "GET /std/base/key1/key2/name1 HTTP/1.1", host: "domain-1"

2023/10/06 13:17:01 [notice] 1152#1152: *37717323 rewritten data: "/base/key1/key2/name1", args: "", client: 10.0.179.122, server: domain-1, request: "GET /std/base/key1/key2/name1 HTTP/1.1", host: "domain-1"

10.0.179.122 - - [06/Oct/2023:13:17:01 +0000] "GET /std/base/key1/key2/name1 HTTP/1.1" 400 248 "-" "-" 336 0.004 [standard-external-service-443] [] 3.79.135.121:443 248 0.005 400 3111ad9748961e237011dd0449dafe08

Log of NGINX in domain-2:

2023/10/10 13:17:01 [info] 140#140: *40730708 client sent plain HTTP request to HTTPS port while reading client request headers, client: 10.0.62.0, server: _, request: "GET /base/key1/key2/CS-SIEMENS HTTP/1.1", host: "domain-1"

We are in k8s env, with NGINX ingress + External Service:

External service:

apiVersion: v1
kind: Service
metadata:
  name: standard-external-service
  namespace: qa
spec:
  type: ExternalName
  externalName: domain-2

Ingress:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-standard-external
  namespace: qa
  annotations:
    cert-manager.io/issuer: letsencrypt-prod
    nginx.ingress.kubernetes.io/enable-access-log: "true"
    nginx.ingress.kubernetes.io/enable-rewrite-log: "true"
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
    nginx.ingress.kubernetes.io/proxy-body-size: 1m
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
    nginx.ingress.kubernetes.io/rewrite-target: /$2
    nginx.ingress.kubernetes.io/secure-backends: "true"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/use-regex: "true"
    nginx.ingress.kubernetes.io/websocket-services: standard-external-service
    nginx.org/websocket-services: standard-external-service
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - "domain-1"
        - "*.domain-1"
      secretName: tls-secret-qa
  rules:
    - host: "domain-1"
      http:
        paths:
          # e-Mobility Standard
          - path: /std(/|$)(.*)
            pathType: Prefix
            backend:
              service:
                name: standard-external-service
                port:
                  number: 443
    - host: "*.domain-1"
      http:
        paths:
          # e-Mobility Standard
          - path: /std(/|$)(.*)
            pathType: Prefix
            backend:
              service:
                name: standard-external-service
                port:
                  number: 443

NGINX controller:

{
  "apiVersion": "v1",
  "data": {
    "allow-snippet-annotations": "true",
    "keep-alive": "349",
    "large-client-header-buffers": "4 16k",
    "ssl-ciphers": "ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:AES256-SHA256",
    "ssl-protocols": "TLSv1.3 TLSv1.2"
  },
  "kind": "ConfigMap",
  "metadata": {
    "labels": {
      "app.kubernetes.io/component": "controller",
      "app.kubernetes.io/instance": "ingress-nginx",
      "app.kubernetes.io/managed-by": "Helm",
      "app.kubernetes.io/name": "ingress-nginx",
      "app.kubernetes.io/part-of": "ingress-nginx",
      "app.kubernetes.io/version": "1.8.1",
      "argocd.argoproj.io/instance": "ingress-nginx",
      "helm.sh/chart": "ingress-nginx-4.7.1",
    },
    "name": "ingress-nginx-controller",
    "namespace": "ingress-nginx",
  },
}

NGINX rewrites the URL and redirect the call and I got the error 400: Client sent plain HTTP request to HTTPS port

I think the SSL terminaison occurs at NGINX level (not NLB) and forwarded to the domain-2, but I don't get why I have this error as SSL should be used all along!