RabbitMQ Kubernetes RabbitmqCluster Operator not adding members to Quorum Queues

Question

I am using the rabbitmq cluster operator to deploy a RabbitMQ HA cluster on kubernetes. I am importing the defintions by referring to [this example](https://github.com/rabbitmq/cluster-operator/blob/main/docs/examples/import-definitions/rabbitmq.yaml Import Definitions Example). I have provided configuration as below (3 replicas)

After the cluster is up and i access the management console using port-forward on the service, i see that the queue is declared with type set to "quorum" but it only shows the 1st node of the cluster under leader, online and members. The cluster is set up with 3 replicas and the default value for quorum initial group size is 3 if i dont specify any.(although i am specifying it explicitly in the defintions file). It should show other members of cluster under online and members section but it shows only the first node (rabbitmq-ha-0) Am i missing any configuration ?

apiVersion: rabbitmq.com/v1beta1
kind: RabbitmqCluster
metadata:
  name: import-definitions
spec:
  replicas: 3
  override:
    statefulSet:
      spec:
        template:
          spec:
            containers:
            - name: rabbitmq
              volumeMounts:
              - mountPath: /path/to/exported/ # filename left out intentionally
                name: definitions
            volumes:
            - name: definitions
              configMap:
                name: definitions # Name of the ConfigMap which contains definitions you wish to import
  rabbitmq:
    additionalConfig: |
      load_definitions = /path/to/exported/definitions.json # Path to the mounted definitions file

and my definitions file is something like this:

{
   "users": [
    {
      "name": "my-vhost",
      "password": "my-vhost",
      "tags": "",
      "limits": {}
    }
  ],
  "vhosts": [
    {
      "name": "/"
    },
    {
      "name": "my-vhost"
    }
  ],
  "permissions": [
    {
      "user": "my-vhost",
      "vhost": "my-vhost",
      "configure": ".*",
      "write": ".*",
      "read": ".*"
    }
  ],
  "topic_permissions": [
    {
      "user": "my-vhost",
      "vhost": "my-vhost",
      "exchange": "",
      "write": ".*",
      "read": ".*"
    }
  ],
   "parameters":[
      {
         "value":{
            "max-connections":100,
            "max-queues":15
         },
         "vhost":"my-vhost",
         "component":"vhost-limits",
         "name":"limits"
      }
   ],
   "policies":[
      {
         "vhost":"my-vhost",
         "name":"Queue-Policy",
         "pattern":"my-*",
         "apply-to":"queues",
         "definition":{
            "delivery-limit":3
         },
         "priority":0
      }
   ],
   "queues":[
      {
         "name":"my-record-q",
         "vhost": "my-vhost",
         "durable":true,
         "auto_delete":false,
         "arguments":{
            "x-queue-type":"quorum",
            "x-quorum-initial-group-size":3
         }
      }
   ],
   "exchanges":[
      {
         "name":"my.records.topic",
         "vhost": "my-vhost",
         "type":"topic",
         "durable":true,
         "auto_delete":false,
         "internal":false,
         "arguments":{
            
         }
      }
   ],
   "bindings":[
      {
         "source":"my.records-changed.topic",
         "vhost": "my-vhost",
         "destination":"my-record-q",
         "destination_type":"queue",
         "routing_key":"#.record.#",
         "arguments":{
            
         }
      }
   ]
}

Answer 1

The problem is that the queue definitions along with other configuration are mounted as volume on the pod.In a stateful set the pods are created only after the first pod is successfully created and up and so on. Because of the Pod 1 will create the queue and since the cluster is not yet up (as other pods are not yet up) there are not other members to be added to the queue. The correct way to set the queues is to create them after the cluster (all the pods) is up and operational through some webhook or create the queues from the application when the application connects to the RMQ cluster.