I am developing a Helm chart for application that consists of several services that require PVC to be provisioned (one umbrella chart with multiple sub-charts). As an example, the application use database, ELK stack, and messaging like RabbitMQ, and all of them need to have a PVC to persist data.
Based on my research, I can use PVC in ReadWriteMany access mode, which should allow to use the same PVC multiple pods. Then, in each deployment, I can specify the subPath of volumeMount, that will create and use specific subfolder in the share PVC.
Let's say, I have the following structure in the PVC based on the subPath for my services that need to persist data:
.
..
postgresql/
elastic/
rabbitmq/
PostgreSQL will use only postgresql subfolder to store data, etc.
And it is based on distributed file-system to support the sharing.
I came across opinions that this is not good way how to manage persistent data for application managed by the Helm chart, although it can be dynamically provisioned, if the storage class supports it.
Alternative is that each service will provision its own PVC in ReadWriteOnce access mode. But in that case, the service will be allowed to work with the data, but other pods will not be even read anything from such PVC, when needed.
The first approach seems to me more consistent, with data in one place (in one PVC).
I would like to know, what is the best practice in such case you have an application managed by the Helm chart, and the application needs to have PVC for multiple sub-charts that can be in some cases shared?
Using a disk (e.g, PVC) as a shared resource in Kubernetes is not your best option. It's very fragile and limiting; you should aim using the API between components (containers), rather than using the disk. Using shared disk brings limitations, such as difficulties moving a Pod between nodes (because the disk / mount needs to be handled as well). It makes it harder later to separate the chart, in case you need to.
Why a different container other than postgresql would need access to it's disk? Same goes for elastic, and rabbit.
Each one of these containers need to get its own PVC, as its the only consumer of this data. It will ease the maintenance of them and reduce the blast radius if something goes wrong.
Keeping each app (or chart) defining its own dependencies (such as PVC) is a better choice. Kubernetes also takes care for it, so why would you want to create a dependency between the apps (psql, elastic, rabbit, etc.)
Keep it simple. Each app defines its own dependencies (when possible) and isolate it from other containers.