Exposing OpenVPN via TCP in k8s cluster with nginx-controller

129 Views Asked by At

I am quite stuck on the instructions to correctly receive TCP traffic and route it to an OpenVPN server withon a k8s cluster.

I use nginx-controller and AWS NLB successfully for http traffic using the classic Ingress resource with nginx annotations. However, when it comes to TCP traffic, documentation points to this

I deployed the nginx-controller using its helm-chart (through Terraform, so I did a simplified copy paste to make it easily readable) with the following values:

controller.service.annotations...->/aws-load-balancer-type: "external"
controller.service.annotations...->/aws-load-balancer-nlb-target-type: "ip"
controller.service.annotations...->/aws-load-balancer-scheme: "internet-facing"
controller.service.annotations...->/aws-load-balancer-ssl-cert: var.acm_cert_arn
controller.service.annotations...->/aws-load-balancer-ssl-ports: "443"
# Proxy protocol from AWS NLB to Ingress and source-IP preservation configuration
# Preserve source IP address
# Ref. https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.1/guide/service/annotations/#proxy-protocol-v2
controller.service.annotations...->/aws-load-balancer-target-group-attributes: "preserve_client_ip.enabled=true"
controller.service.annotations...->/aws-load-balancer-proxy-protocol: "*"
# Ref. https://github.com/kubernetes/ingress-nginx/issues/7675#issuecomment-968914295
controller.config.use-proxy-protocol: "true"
controller.config.proxy-body-size: "512M"
controller.config.large-client-header-buffers: "8 8k"
controller.config.generate-request-id: "true"
controller.config.enable-real-ip: "true"
controller.config.real-ip-recursive: "on"
controller.config.use-forwarded-headers: "true"
controller.config.compute-full-forwarded-for: "true"
controller.config.proxy-add-original-uri-header: "true"
controller.service.targetPorts.https: "80"
tcp.${var.openvpn_port}": "${var.systems_namespace}/openvpn-server:${var.openvpn_port}"

I have set DNS to point to the NLB (let's assume vpn.example.org) and I see within k8s the configmap required by nginx-controller for TCP traffic and that looks like:

17700: "namespace/openvpn-server:1770"

Now, whenever I try to connect with a client to the OpenVPN server I get:

WARNING: Bad encapsulated packet length from peer (3338), which must be > 0 and <= 1627 -- please ensure that --tun-mtu or --link-mtu is equal on both peers -- this condition could also indicate a possible active attack on the TCP link -- [Attempting restart...]
Connection reset, restarting [0]

While on the client side:

TCP/UDP: Preserving recently used remote address: [AF_INET]xx.xxx.xxx.xx:17700
Attempting to establish TCP connection with [AF_INET]xx.xxx.xxx.xx:17700 [nonblock]
TCP connection established with [AF_INET]xx.xxx.xxx.xx:17700
TCP_CLIENT link local: (not bound)
TCP_CLIENT link remote: [AF_INET]xx.xxx.xxx.xx:17700
Connection reset, restarting [0]
SIGUSR1[soft,connection-reset] received, process restarting

The problem to me seems in the PROXY configuration for nginx. I tried to follow the documentation and adding a PROXY to the configmap but I couldn't find a real example for that so I tried different ones with either the same result or no packet at all arriving

Main attempts where for the configmap:

17700: "namespace/openvpn-server:1770:vpn.example.org"
17700: "namespace/openvpn-server:1770:vpn.example.org:1770"
17700: "namespace/openvpn-server:1770:vpn.example.org:vpn.example.org"

The openvpn server is created from this one

While the service that routes the traffic is:

apiVersion: v1
kind: Service
metadata:
  name: openvpn-server
  namespace: namespace-name
  labels:
    app: openvpn-server
spec:
  ports:
    - port: 17700
      targetPort: 1194
      name: openvpn-server
    - port: 2222
      targetPort: 2222
      name: webssh2
  selector:
    app: openvpn-server

Does anyone have an idea on how to configure this PROXY configmap to make it work, or, if there is something else I have to change in the nginx-controller settings?

Thanks in advance

0

There are 0 best solutions below