Question

How to Resolve Workers Not Scaling with 100s of Queued Tasks in Google Cloud Composer?

score 13 · Answer 1 · 2024-03-30T10:23:26.340000

13

Views

How to Resolve Workers Not Scaling with 100s of Queued Tasks in Google Cloud Composer?

Published on 30 March 2024 at 10:23

score 27 · Answer 2 · 2024-03-27T06:49:08.360000

27

Views

Using Composer version 2.6.2 and Airflow version 2.5.3, getting AttributeError: 'bytes' object has no attribute 'encode' error in pubsub message

Published on 27 March 2024 at 06:49

score 21 · Answer 3 · 2024-03-26T11:25:36.130000

21

Views

getting ValueError: Cannot determine path without bucket name

Published on 26 March 2024 at 11:25

score 38 · Answer 4 · 2024-03-22T04:39:24.917000

38

Views

Using Airflow version 2.5.3 and Cloud Composer version 2.6.2, DAGs frequently throw this error of log file not found

Published on 22 March 2024 at 04:39

score 30 · Answer 5 · 2024-03-21T00:43:42.590000

30

Views

Can't install private Artifactory Cloud PyPi package within Cloud Composer environment

Published on 21 March 2024 at 00:43

score 32 · Answer 6 · 2024-03-19T05:40:43.040000

32

Views

How can we use GCSObjectExistenceSensor to sense one or more files with filemask

Published on 19 March 2024 at 05:40

score 51 · Answer 7 · 2024-03-18T10:20:49.720000

51

Views

not able to connect via cloud sql proxy

Published on 18 March 2024 at 10:20

score 46 · Answer 8 · 2024-03-13T19:06:40.620000

46

Views

Automatically Triggering Airflow DAG in Cloud Composer on GCP without Cloud Functions

Published on 13 March 2024 at 19:06

score 43 · Answer 9 · 2024-03-11T17:03:37.997000

43

Views

Error in Cloud Composer with DBT where path /dbt_packages/dbt_utils/dbt_project.yml is not found

Published on 11 March 2024 at 17:03

score 20 · Answer 10 · 2024-03-08T07:53:48.433000

20

Views

Is it possible to amend environment config for GCP Cloud Composer?

Published on 08 March 2024 at 07:53

score 29 · Answer 11 · 2024-03-06T15:03:38.220000

29

Views

Airflow - Distribute flow execution on multiple workers

Published on 06 March 2024 at 15:03

score 34 · Answer 12 · 2024-03-06T04:18:07.333000

34

Views

Provision Google Cloud Composer 2 with Terraform

Published on 06 March 2024 at 04:18

score 53 · Answer 13 · 2024-03-05T16:22:02.170000

53

Views

Stopping the next execution of an Airflow DAG while the current execution is still running

Published on 05 March 2024 at 16:22

score 136 · Answer 14 · 2024-02-28T13:53:01.917000

136

Views

KubernetesPodOperator shows completed in Airflow UI but keep on running in backend

Published on 28 February 2024 at 13:53

score 106 · Answer 15 · 2024-02-23T20:33:18.730000

106

Views

Updating a Variable doesn't seem to have any effect

Published on 23 February 2024 at 20:33

score 33 · Answer 16 · 2024-02-20T13:42:06.473000

33

Views

How to install composer through cloud function in GCP?

Published on 20 February 2024 at 13:42

score 23 · Answer 17 · 2024-02-20T00:29:50.980000

23

Views

Using Composer to Migrate Data from GCS to BigQuery

Published on 20 February 2024 at 00:29

score 54 · Answer 18 · 2024-02-19T16:50:07.253000

54

Views

Error rendering Jinja template for Airflow task using BigQueryOperator with SQL file containing template fields

Published on 19 February 2024 at 16:50

score 44 · Answer 19 · 2024-02-14T21:38:23.743000

44

Views

Cloud Composer DAG Scheduling based on Big Query Table

Published on 14 February 2024 at 21:38

score 38 · Answer 20 · 2024-02-14T20:24:47.230000

38

Views

Airflow GCSToBigQueryOperator request too large - Error 413 Request Entity Too Large

Published on 14 February 2024 at 20:24

List Question

How to Resolve Workers Not Scaling with 100s of Queued Tasks in Google Cloud Composer?

Using Composer version 2.6.2 and Airflow version 2.5.3, getting AttributeError: 'bytes' object has no attribute 'encode' error in pubsub message

getting ValueError: Cannot determine path without bucket name

Using Airflow version 2.5.3 and Cloud Composer version 2.6.2, DAGs frequently throw this error of log file not found

Can't install private Artifactory Cloud PyPi package within Cloud Composer environment

How can we use GCSObjectExistenceSensor to sense one or more files with filemask

not able to connect via cloud sql proxy

Automatically Triggering Airflow DAG in Cloud Composer on GCP without Cloud Functions

Error in Cloud Composer with DBT where path /dbt_packages/dbt_utils/dbt_project.yml is not found

Is it possible to amend environment config for GCP Cloud Composer?

Airflow - Distribute flow execution on multiple workers

Provision Google Cloud Composer 2 with Terraform

Stopping the next execution of an Airflow DAG while the current execution is still running

KubernetesPodOperator shows completed in Airflow UI but keep on running in backend

Updating a Variable doesn't seem to have any effect

How to install composer through cloud function in GCP?

Using Composer to Migrate Data from GCS to BigQuery

Error rendering Jinja template for Airflow task using BigQueryOperator with SQL file containing template fields

Cloud Composer DAG Scheduling based on Big Query Table

Airflow GCSToBigQueryOperator request too large - Error 413 Request Entity Too Large

Trending Questions

Popular # Hahtags

Popular Questions