How to make an app accessable over the internet using Ingress or metalLB on bare metal

Question

I'm running a k8s cluter with one control and one worker node on bare metal ubuntu machines (IPs: 123.223.149.27 and 22.36.211.68).

I deployed a sample app:

kubectl create deployment nginx --image=nginx
kubectl expose deploy nginx --port 80 --target-port 80 --type NodePort

Running kubectl get services shows me:

NAME           TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
kubernetes     ClusterIP      10.96.0.1        <none>        443/TCP        5d23h
nginx          NodePort       10.100.107.184   <none>        80:30799/TCP   5h48m

and I can access this appllication inside of the cluster by

kubectl run alpine --image=alpine --restart=Never --rm -it -- wget -O- 10.100.107.184:80

But now I want to access the sample app outside of the cluster in the internet via http://123.223.149.27 or later within the domain mywebsite.com as the DNS of the domain is pointing to 123.223.149.27.

I applied:

kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.12.1/manifests/namespace.yaml
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.12.1/manifests/metallb.yaml

with this config map:

apiVersion: v1
kind: ConfigMap
metadata:
  namespace: metallb-system
  name: config
data:
  config: |
    address-pools:
    - name: production-public-ips
      protocol: layer2
      addresses:
      - 123.223.149.27/32
      - 22.36.211.68/32

and this ingress:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.3.1/deploy/static/provider/cloud/deploy.yaml

For me it is not clear, if I have to use ingress (then I would use ingress-nginx) and metalLB and how to configure both. I read a lot of service types like loadBalancer and NodePorts, but I think I didn't understand the concept correctly. I even didn't understand if I have to use ingress-nginx OR metalLB OR both of them. I only understand that if I'm using type LoadBalancer I have to use a loadbalancer as I am on bare metal, so in that case I have to use metalLB.

It would be very helpful for my understanding, if someone could explain on this example app how to make this accessable over the internet.

Answer 1

Since, you have a running service inside your Kubernetes cluster, you can expose via an ingress-controller which is a reverse-proxy that routes traffics from outside to your dedicated service(s) inside the cluster,

We'll use for example ingress-nginx,See https://github.com/kubernetes/ingress-nginx

These are the requirements you'll need for reaching your service at mywebsite.com:

1. Have access to DNS records of your domain mywebsite.com

2. Install ingress-nginx in your Cluster, See https://github.com/kubernetes/ingress-nginx/tree/main/charts/ingress-nginx

Install it using helm

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm install ingress-nginx ingress-nginx/ingress-nginx \
    --namespace $NAMESPACE \
    --set controller.replicaCount=2 \
    --set controller.nodeSelector."kubernetes\.io/os"=linux \
    --set controller.admissionWebhooks.patch.nodeSelector."kubernetes\.io/os"=linux \
    --set defaultBackend.nodeSelector."kubernetes\.io/os"=linux \
    --version $VERSIONS

You can look for versions compatibles with your Kubernetes cluster version using:

helm search repo ingress-nginx/ingress-nginx --versions

When installation is well finished, you should see ingress-controller service that holds an $EXTERNAL-IP

NAME                                 TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)                                  AGE

ingress-nginx-controller             LoadBalancer   10.0.XXX.XXX   XX.XXX.XXX.XX   80:30578/TCP,443:31874/TCP               548d

1. Now that you have a running ingress controller, you need to create a ingress object that manages external access to your $Service,

See https://kubernetes.io/docs/concepts/services-networking/ingress/

For example:

#ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    #cert-manager.io/cluster-issuer: letsencrypt-prod
    #nginx.ingress.kubernetes.io/ssl-redirect: 'true'
spec:
  tls:
    - hosts:
        - mywebsite.com
      #secretName: cert-wildcard # get it from certificate.yaml
  rules:
    - host: mywebsite.com
      http:
        paths:
          - path: / #/?(.*) #/(.*)
            pathType: Prefix
            backend:
              service:
                name: $Service
                port:
                  number: 80
  ingressClassName: nginx
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  name: nginx
spec:
  controller: k8s.io/ingress-nginx

4. As you saw in the ingress file, commented lines refere to the usage of an SSL certificate generated by cert-manager from LetsEncrypt, this can be achieved by another process which is described here https://cert-manager.io/docs/configuration/acme/, it depends mainly on your cloud provider (Cloudflare, Azure, ...)

5. Finally, In your DNS zone, Add a DNS record which maps mywebsite.com to $EXTERNAL-IP, wait a few minutes and you should be able to access your service under mywebsite.com