AWS route53 external dns point only one A records to the (E)LB of the (nginx) ingress controller

Question

I have created two ingresses, one for Grafana and one for my App. When the external dns write them into the route53 hosted zone as a A record, only one of them (the Myapp dns) get the (E)LB alias (dns), though the second A record get the internal IP as an ip address in the route53 A record.

the big question: is there a way to set them all to the same alias/or at list to the same elb? why it doesn't success doing so by default?

using:

terraform:
   helm_release:
      nginx-controller-bitnami
      external-dns-bitnami
      prometheus community

grafana-ingress:

apiVersion: v1
items:
- apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    annotations:
      kubernetes.io/ingress.class: nginx
      meta.helm.sh/release-name: kube-prometheus-stack
      meta.helm.sh/release-namespace: prometheus
    generation: 1
    labels:
      app.kubernetes.io/instance: kube-prometheus-stack
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/name: grafana
      app.kubernetes.io/version: 8.3.3
      helm.sh/chart: grafana-6.20.5
    name: kube-prometheus-stack-grafana
    namespace: prometheus
    resourceVersion: "2419"
  spec:
    rules:
    - host: grafana.dns.io
      http:
        paths:
        - backend:
            service:
              name: kube-prometheus-stack-grafana
              port:
                number: 80
          path: /
          pathType: Prefix
  status:
    loadBalancer:
      ingress:
      - ip: 10.0.1.19
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

my-app-ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: app-ingress
  annotations:
    ingress.kubernetes.io/ssl-redirect: /
    nginx.ingress.kubernetes.io/rewrite-target: /$1
    kubernetes.io/ingress.class: nginx
spec:
  rules:
  - host: "some.website.io"
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: "{{ .Release.Name }}-app"
            port:
              number: 80

status:
  loadBalancer:
    ingress:
    - ip: 10.0.1.19

nginx-controller-service

kind: Service
metadata:
  annotations:
    external-dns.alpha.kubernetes.io/hostname: website.my-app.io, some.grafana.io
    meta.helm.sh/release-name: nginx-ingress-controller
    meta.helm.sh/release-namespace: default
  finalizers:
  - service.kubernetes.io/load-balancer-cleanup
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: nginx-ingress-controller
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: nginx-ingress-controller
    helm.sh/chart: nginx-ingress-controller-9.1.4
  name: nginx-ingress-controller
  namespace: default
  resourceVersion: "997"
spec:
  clusterIP: 172.30.0.140
  clusterIPs:
  - 172.30.0.140
  externalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - name: http
    nodePort: 30337
    port: 80
    protocol: TCP
    targetPort: http
  - name: https
    nodePort: 31512
    port: 443
    protocol: TCP
    targetPort: https
  selector:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: nginx-ingress-controller
    app.kubernetes.io/name: nginx-ingress-controller
  sessionAffinity: None
  type: LoadBalancer
status:
  loadBalancer:
    ingress:
    - hostname: someElbDns.eu-west-1.elb.amazonaws.com

Answer 1

Since your ingress controller service is of type LoadBalancer, creating this service will provision an NLB for you which will have your nginx controller pods as target groups.

A request will pass from the NLB to the ingress pods and then to grafana/your app subsequently.

I would try to use the same host (website.io) and the same nginx ingress class so both ingresses get assigned the same NLB address:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  namespace: default
  name: grafana-ingress
  annotations:
    nginx.ingress.kubernetes.io/force-ssl-redirect: "false"
    kubernetes.io/ingress.class: "nginx"
spec:
  tls:
    - hosts:
        - grafana.website.io
      secretName: tls-secret-website-io
  rules:
    - host: website.io
    - http:
        paths:
          - path: /
            backend:
              serviceName: grafana-service
              servicePort: 80

---

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  namespace: default
  name: awebsite-ingress
  annotations:
    nginx.ingress.kubernetes.io/force-ssl-redirect: "false"
    kubernetes.io/ingress.class: "nginx"
spec:
  tls:
    - hosts:
        - subdomain.website.io
      secretName: tls-secret-website-io
  rules:
    - host: website.io
    - http:
        paths:
          - path: /(/|$)(.*)
            backend:
              serviceName: website-service
              servicePort: 80


kind: Service
apiVersion: v1
metadata:
  name: nginx-controller
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: nginx-controller
    app.kubernetes.io/part-of: nginx-controller
  annotations:
    # by default the type is elb (classic load balancer).
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
spec:
  # this setting is to make sure the source IP address is preserved.
  externalTrafficPolicy: Local
  type: LoadBalancer
  selector:
    app.kubernetes.io/name: nginx-controller
    app.kubernetes.io/part-of: nginx-controller
  ports:
    - name: http
      port: 80
      targetPort: http
    - name: https
      port: 443
      targetPort: https

Last step is to create a CNAME entry in Rote53 from your domain to the loadbalancer DNS.