How to specify non default compute class in KerbenetesPodOperator for cloud composer 2

Question

I am using KurbenetesPodOperator in Cloud composer 2 to create pods to execute the spark jobs. As default, the compute_class general-purpose class is assumed when you create Pods using KubernetesPodOperator. I would like the KurbenetesPodOperator to request pods to be created in Balanced Class nodes.

I went through these articles 1, 2.

I tried to add nodeaffinity.

"nodeAffinity": {
            "requiredDuringSchedulingIgnoredDuringExecution": {
                "nodeSelectorTerms": [
                    {
                       "matchExpressions": [
                            {
                                "key": "cloud.google.com/compute-class",
                                "operator": "In",
                                "values": ["Balanced"],
                            }
                        ]
                   }
                ]
            }

Receiving the below error.

Pod 'r-migrations-8u8idq49' specifies non-default compute class. Cloud Composer 2 does not support creating workloads with non-default compute classes.

Answer 1

Cloud Composer 2 supports only general-purpose compute class at the moment.

https://cloud.google.com/composer/docs/composer-2/use-kubernetes-pod-operator#begin https://issuetracker.google.com/issues/273587652