Auto reload configuration changes without restarting pod/container using Kubernetes ConfigMap for application with large number of configuration files

Question

Our team is planning the migration of legacy enterprise application developed in ASP.net web-Forms, to .Net Core 6 as use the containerized approach. For this, mostly we will target the Kubernetes container orchestration platform.

The application is highly configurable and can be integrated with related apps up to certain extent. It has large number of XML based configuration files (more than 100). Current mode of deployment is IIS (on-premise).

The major technical challenge that we are facing is to manage our application configuration.

So ConfigMap is one the option available in Kubernetes can be used for configuration management. ConfigMap APIs allows to generate ConfigMap from environment, yaml file, existing configuration file or directory. Directory based approach seems more suitable. However, considering the maximum size limit of ConfigMap we may end up creating multiple ConfigMap.

We need to make sure:

1. Migrated app should be able to use configuration however application image should be separate and configuration can be injected from outside.
2. The configuration changes should be reflected in application without POD.

Since the ConfigMap is kind of read-only resource when container starts, I am currently looking for mechanism to use with configuration reload without the need of restarting POD/container.

Initial focus is to achieve this. (The impact of changed configuration on active users who might be referring to application feature based on previous configuration is a different topic altogether).

Answer 1

You can do it without restarting the POD using configmap only, however still it more depends on your application end.

You can inject your configmap and mount it to POD Kubernetes auto-reload the config map if mounted to the directory. To note it does not work if you are using the subpath.

Auto reload config map into Kubernetes without restarting the POD, you can read more here: https://medium.com/@harsh.manvar111/update-configmap-without-restarting-pod-56801dce3388

YAML example

apiVersion: v1
kind: ConfigMap
metadata:
  name: test-config
data:
  hello: world
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: test
spec:
  selector:
    matchLabels:
      app: test
  replicas: 1
  template:
    metadata:
      labels:
        app: test
    spec:
      containers:
      - name: configmaptestapp
        image: <Image>
        volumeMounts:
        - mountPath: /config
          name: data-volume
        ports:
        - containerPort: 80
      volumes:
        - name: data-volume
          configMap:
            name: test-config

Official documentation : https://kubernetes.io/docs/concepts/configuration/configmap/#mounted-configmaps-are-updated-automatically

Mounted ConfigMaps are updated automatically When a ConfigMap currently consumed in a volume is updated, projected keys are eventually updated as well. The kubelet checks whether the mounted ConfigMap is fresh on every periodic sync. However, the kubelet uses its local cache for getting the current value of the ConfigMap. The type of the cache is configurable using the ConfigMapAndSecretChangeDetectionStrategy field in the KubeletConfiguration struct. A ConfigMap can be either propagated by watch (default), ttl-based, or by redirecting all requests directly to the API server. As a result, the total delay from the moment when the ConfigMap is updated to the moment when new keys are projected to the Pod can be as long as the kubelet sync period + cache propagation delay, where the cache propagation delay depends on the chosen cache type (it equals to watch propagation delay, ttl of cache, or zero correspondingly).

ConfigMaps consumed as environment variables are not updated automatically and require a pod restart.

Note:

A container using a ConfigMap as a subPath volume mount will not receive ConfigMap updates.

In this case your application need to be handling the content properly with change detection etc.