I am quite stuck on the instructions to correctly receive TCP traffic and route it to an OpenVPN server withon a k8s cluster.
I use nginx-controller and AWS NLB successfully for http traffic using the classic Ingress resource with nginx annotations. However, when it comes to TCP traffic, documentation points to this
I deployed the nginx-controller using its helm-chart (through Terraform, so I did a simplified copy paste to make it easily readable) with the following values:
controller.service.annotations...->/aws-load-balancer-type: "external"
controller.service.annotations...->/aws-load-balancer-nlb-target-type: "ip"
controller.service.annotations...->/aws-load-balancer-scheme: "internet-facing"
controller.service.annotations...->/aws-load-balancer-ssl-cert: var.acm_cert_arn
controller.service.annotations...->/aws-load-balancer-ssl-ports: "443"
# Proxy protocol from AWS NLB to Ingress and source-IP preservation configuration
# Preserve source IP address
# Ref. https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.1/guide/service/annotations/#proxy-protocol-v2
controller.service.annotations...->/aws-load-balancer-target-group-attributes: "preserve_client_ip.enabled=true"
controller.service.annotations...->/aws-load-balancer-proxy-protocol: "*"
# Ref. https://github.com/kubernetes/ingress-nginx/issues/7675#issuecomment-968914295
controller.config.use-proxy-protocol: "true"
controller.config.proxy-body-size: "512M"
controller.config.large-client-header-buffers: "8 8k"
controller.config.generate-request-id: "true"
controller.config.enable-real-ip: "true"
controller.config.real-ip-recursive: "on"
controller.config.use-forwarded-headers: "true"
controller.config.compute-full-forwarded-for: "true"
controller.config.proxy-add-original-uri-header: "true"
controller.service.targetPorts.https: "80"
tcp.${var.openvpn_port}": "${var.systems_namespace}/openvpn-server:${var.openvpn_port}"
I have set DNS to point to the NLB (let's assume vpn.example.org) and I see within k8s the configmap required by nginx-controller for TCP traffic and that looks like:
17700: "namespace/openvpn-server:1770"
Now, whenever I try to connect with a client to the OpenVPN server I get:
WARNING: Bad encapsulated packet length from peer (3338), which must be > 0 and <= 1627 -- please ensure that --tun-mtu or --link-mtu is equal on both peers -- this condition could also indicate a possible active attack on the TCP link -- [Attempting restart...]
Connection reset, restarting [0]
While on the client side:
TCP/UDP: Preserving recently used remote address: [AF_INET]xx.xxx.xxx.xx:17700
Attempting to establish TCP connection with [AF_INET]xx.xxx.xxx.xx:17700 [nonblock]
TCP connection established with [AF_INET]xx.xxx.xxx.xx:17700
TCP_CLIENT link local: (not bound)
TCP_CLIENT link remote: [AF_INET]xx.xxx.xxx.xx:17700
Connection reset, restarting [0]
SIGUSR1[soft,connection-reset] received, process restarting
The problem to me seems in the PROXY configuration for nginx. I tried to follow the documentation and adding a PROXY to the configmap but I couldn't find a real example for that so I tried different ones with either the same result or no packet at all arriving
Main attempts where for the configmap:
17700: "namespace/openvpn-server:1770:vpn.example.org"
17700: "namespace/openvpn-server:1770:vpn.example.org:1770"
17700: "namespace/openvpn-server:1770:vpn.example.org:vpn.example.org"
The openvpn server is created from this one
While the service that routes the traffic is:
apiVersion: v1
kind: Service
metadata:
name: openvpn-server
namespace: namespace-name
labels:
app: openvpn-server
spec:
ports:
- port: 17700
targetPort: 1194
name: openvpn-server
- port: 2222
targetPort: 2222
name: webssh2
selector:
app: openvpn-server
Does anyone have an idea on how to configure this PROXY configmap to make it work, or, if there is something else I have to change in the nginx-controller settings?
Thanks in advance