I was trying to resize persistent volumes associated with a stateful set today. I am using Azure Kubernetes service v1.26.6. The persistent voluem is created from a storage class of type "default".
As per the official Kubernetes documentation at: https://kubernetes.io/blog/2022/05/05/volume-expansion-ga/, it is now possible to expand the size of a persistent volume without any downtime, just by updating the spec field of the PVC of the stateful set and re-deploying the stateful set. However, I attempted the same today(i.e tried to redeploy an already deployed stateful set but with increased size for the PVC) and i ran into the usual error:
Error: UPGRADE FAILED: cannot patch "grafana-loki-querier" with kind StatefulSet: StatefulSet.apps "grafana-loki-querier" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden
To overcome this error, I then followed the instructions at: https://www.giffgaff.io/tech/resizing-statefulset-persistent-volumes-with-zero-downtime and was then finally able to successfully deploy the helm chart containing updated sizes for the resized PVC.
Questions
Am I misunderstanding the official Kubernetes documentation that I have linked in the question?
I am curious. Technically, if the size of the PVC can be increased by editing the spec field in the PVC object directly(kubectl edit pvc), why is Kubernetes imposing a restriction to do the same by directly updating the size in the PVC manifest files ie. why cannot I increase the size of the PVC using a helm deployment?
Why is the workflow not an integral part of Kubernetes deployments using Helm?
Why does it involve deleting the stateful set followed by recreation of the stateful set? What is the reason for the same?
Not really, but you are only partially true.
Even if StorageClass supports resizing, Kubernetes doesn't support (yet) volume expansion through StatefulSets. Please refer to the official enhancement issue at Support Volume Expansion Through StatefulSets and Kubernetes StatefulSet: support resize pvc storage.
More Insights
In this case, this has nothing to do with helm charts but actually with the Kubernetes controller involved in the application stack, as the controller is continuously monitoring the app and making the required changes as per its defined manifest and hence as mentioned in your workaround document you had to scale it to
0Second, we had to delete the stateful state with the orphan strategy so that the actual pod ( working application ) does not get deleted and only the stateful set controller(kind) is deleted. Because of that, you are allowed to Update the PVC and then it gets resized only if resizing is supported and enabled on your storageClass->Storage Driver which seems the case.