How to setup Kubernetes ingress behind nginx reverse proxy

141 Views Asked by At

I have kubernetes master-node running on my linux machine and on the very same machine there is also standalone nginx running as a service. My goal was to have some monitoring and databases running on the machine as well as kubernetes node. (I know it's a bad idea, but this is just for development phase) and cover everything with nginx reverse proxy. I had no problem handling proxying to some services running on the machine, however whatever I try I cannot seem to find resolution how to pass requests to ingress controller that would further forward it to service running inside kubernetes.

My nginx config (that runs outside kubernetes and should be the reverse proxy)

server {
        listen 8080 default_server;
        root /var/www/html;
        server_name _;

        location / {
                try_files $uri $uri/ =404;
        }
}


server {
    listen 80;
    server_name nginx.k8s.domain.me;

    location / {
        proxy_pass http://127.0.0.1:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

server {
    listen 80;
    server_name dashboard.kubernetes.k8s.domain.me;
    location / {
        proxy_pass http://192.168.77.139:80;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_buffer_size   128k;
        proxy_buffers   4 256k;
        proxy_busy_buffers_size   256k;
    }
}

When I try to enter nginx.k8s.domain.me I am greeted with default nginx landing page, so we can consider it a success. But trying to enter dashboard.kubernetes.k8s.domain.me ends up with 404.

I applied the dashboard deployment from kubernetes wiki and used this ingress:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: dashboard-ingress
  namespace: kubernetes-dashboard
spec:
  rules:
  - host: dashboard.kubernetes.k8s.domain.me
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: kubernetes-dashboard
            port:
              number: 80

Some kubernetes info for reference:

kubectl get nodes
NAME          STATUS   ROLES           AGE    VERSION
master-node   Ready    control-plane   141m   v1.28.2
kubectl get ingresses --all-namespaces -o wide
NAMESPACE              NAME                  CLASS    HOSTS                                 ADDRESS   PORTS   AGE
kubernetes-dashboard   dashboard-ingress     <none>   dashboard.kubernetes.k8s.domain.me               80      131m
kubectl get pods --all-namespaces -o wide
NAMESPACE              NAME                                         READY   STATUS      RESTARTS   AGE    IP               NODE          NOMINATED NODE   READINESS GATES
calico-apiserver       calico-apiserver-5bd8dc87cf-4p5rc            1/1     Running     0          141m   192.168.77.133   master-node   <none>           <none>
calico-apiserver       calico-apiserver-5bd8dc87cf-9szzg            1/1     Running     0          141m   192.168.77.134   master-node   <none>           <none>
calico-system          calico-kube-controllers-685f7c9b88-6mls5     1/1     Running     0          141m   192.168.77.131   master-node   <none>           <none>
calico-system          calico-node-4zqpw                            1/1     Running     0          141m   <hidden>    master-node   <none>           <none>
calico-system          calico-typha-5d74745fdd-2cmnt                1/1     Running     0          141m   <hidden>    master-node   <none>           <none>
calico-system          csi-node-driver-mfmql                        2/2     Running     0          141m   192.168.77.130   master-node   <none>           <none>
default                hello-world-deployment-5cccb4989-8jdsw       1/1     Running     0          41m    192.168.77.140   master-node   <none>           <none>
default                hello-world-deployment-5cccb4989-cnxzz       1/1     Running     0          41m    192.168.77.142   master-node   <none>           <none>
default                hello-world-deployment-5cccb4989-l4ltg       1/1     Running     0          41m    192.168.77.141   master-node   <none>           <none>
ingress-nginx          ingress-nginx-admission-create-kswjk         0/1     Completed   0          135m   192.168.77.138   master-node   <none>           <none>
ingress-nginx          ingress-nginx-admission-patch-gsmjs          0/1     Completed   1          135m   192.168.77.137   master-node   <none>           <none>
ingress-nginx          ingress-nginx-controller-68fb8cf9cc-9bkft    1/1     Running     0          135m   192.168.77.139   master-node   <none>           <none>
kube-system            coredns-5dd5756b68-lvqsj                     1/1     Running     0          142m   192.168.77.132   master-node   <none>           <none>
kube-system            coredns-5dd5756b68-t4ssf                     1/1     Running     0          142m   192.168.77.129   master-node   <none>           <none>
kube-system            etcd-master-node                             1/1     Running     2          143m   <hidden>    master-node   <none>           <none>
kube-system            kube-apiserver-master-node                   1/1     Running     2          143m   <hidden>    master-node   <none>           <none>
kube-system            kube-controller-manager-master-node          1/1     Running     1          143m   <hidden>    master-node   <none>           <none>
kube-system            kube-proxy-dlm74                             1/1     Running     0          142m   <hidden>    master-node   <none>           <none>
kube-system            kube-scheduler-master-node                   1/1     Running     2          143m   <hidden>    master-node   <none>           <none>
kubernetes-dashboard   dashboard-metrics-scraper-5657497c4c-qx5nr   1/1     Running     0          137m   192.168.77.135   master-node   <none>           <none>
kubernetes-dashboard   kubernetes-dashboard-78f87ddfc-5vljg         1/1     Running     0          137m   192.168.77.136   master-node   <none>           <none>
tigera-operator        tigera-operator-8547bd6cc6-5k6s7             1/1     Running     0          142m   <hidden>    master-node   <none>           <none>
kubectl get services --all-namespaces -o wide
NAMESPACE              NAME                                 TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE    SELECTOR
calico-apiserver       calico-api                           ClusterIP      10.96.71.22      <none>        443/TCP                      142m   apiserver=true
calico-system          calico-kube-controllers-metrics      ClusterIP      None             <none>        9094/TCP                     142m   k8s-app=calico-kube-controllers
calico-system          calico-typha                         ClusterIP      10.101.144.111   <none>        5473/TCP                     142m   k8s-app=calico-typha
default                hello-world-service                  ClusterIP      10.106.93.244    <none>        80/TCP                       42m    app=hello-world
default                kubernetes                           ClusterIP      10.96.0.1        <none>        443/TCP                      144m   <none>
ingress-nginx          ingress-nginx-controller             LoadBalancer   10.102.254.176   <pending>     80:31488/TCP,443:32443/TCP   136m   app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
ingress-nginx          ingress-nginx-controller-admission   ClusterIP      10.97.17.56      <none>        443/TCP                      136m   app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
kube-system            kube-dns                             ClusterIP      10.96.0.10       <none>        53/UDP,53/TCP,9153/TCP       144m   k8s-app=kube-dns
kubernetes-dashboard   dashboard-metrics-scraper            ClusterIP      10.107.180.47    <none>        8000/TCP                     138m   k8s-app=dashboard-metrics-scraper
kubernetes-dashboard   kubernetes-dashboard                 ClusterIP      10.108.179.123   <none>        443/TCP                      138m   k8s-app=kubernetes-dashboard

I tried messing with nginx config and ingresses but they did not seem to work at all.

1

There are 1 best solutions below

1
Oskar Migowski On BEST ANSWER

Alright guys, that was pretty simple to be honest. I had my ingress config pointing to port 80, while dashboard exposes port 443. Second thing that I had to change is adding annotation to the ingress, so final version looks like this

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: dashboard-ingress
  namespace: kubernetes-dashboard
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: dashboard.kubernetes.k8s.domain.me
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: kubernetes-dashboard
            port:
              number: 443