Question

Apache Airflow sparksubmit

score 33 · Answer 1 · 2024-03-29T06:06:23.763000

33

Views

Apache Airflow sparksubmit

Published on 29 March 2024 at 06:06

score 33 · Answer 2 · 2024-03-27T19:08:17.937000

33

Views

PySpark Script fails intermittently with "Failed to download resource"

Published on 27 March 2024 at 19:08

score 51 · Answer 3 · 2024-03-26T01:53:23.580000

51

Views

Spark Driver running on Kubernetes hung with too old resource version message

Published on 26 March 2024 at 01:53

score 19 · Answer 4 · 2024-03-21T13:22:22.857000

19

Views

Insert Overwrite partition data using Spark SQL on MINIO table

Published on 21 March 2024 at 13:22

score 61 · Answer 5 · 2024-03-14T11:18:25.450000

61

Views

spark-submit command: --driver-class-path is working and --jars is not working

Published on 14 March 2024 at 11:18

score 35 · Answer 6 · 2024-03-11T10:42:03.053000

35

Views

Can I use spark-submit to run a non-pyspark python code on spark, that is, without or with minimal changes?

Published on 11 March 2024 at 10:42

score 77 · Answer 7 · 2024-03-11T06:07:53.320000

77

Views

Run non-spark python code on spark to use its distributive compute to optimize the performance

Published on 11 March 2024 at 06:07

score 51 · Answer 8 · 2024-03-07T16:54:28.027000

51

Views

Why do spark workers write all tmp files, including shuffle and cache files, to different directories even though we define spark.local.dir?

Published on 07 March 2024 at 16:54

score 33 · Answer 9 · 2024-03-07T09:43:07.727000

33

Views

Spark-submit java.nio.file.NoSuchFileException on kubernetes

Published on 07 March 2024 at 09:43

score 116 · Answer 10 · 2024-02-21T08:29:39.360000

116

Views

Airflow - NameError: name 'ti' is not defined

Published on 21 February 2024 at 08:29

score 25 · Answer 11 · 2024-02-16T17:04:03.393000

25

Views

spark submit job fails in airflow but works in container

Published on 16 February 2024 at 17:04

score 48 · Answer 12 · 2024-02-11T04:07:10.160000

48

Views

Inconsistent results in Spark-shell and Spark-submit

Published on 11 February 2024 at 04:07

score 48 · Answer 13 · 2024-01-23T18:59:19.650000

48

Views

How to use external Spark with the Cloudera cluster?

Published on 23 January 2024 at 18:59

score 38 · Answer 14 · 2024-01-16T19:21:57.633000

38

Views

Error while deploying spark-submit container

Published on 16 January 2024 at 19:21

score 39 · Answer 15 · 2024-01-12T15:05:49.810000

39

Views

Spark submit repo via a remote repository

Published on 12 January 2024 at 15:05

score 29 · Answer 16 · 2024-01-10T07:58:09.467000

WARN YarnScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered

score 43 · Answer 17 · 2023-12-27T10:26:38.677000

43

Views

when running spark-submit on yarn then error?

Published on 27 December 2023 at 10:26

score 120 · Answer 18 · 2023-12-01T08:24:08.533000

120

Views

Caused by: java.lang.ClassNotFoundException: org.apache.kafka.common.serialization.ByteArraySerializer

Published on 01 December 2023 at 08:24

score 36 · Answer 19 · 2023-11-10T16:49:27.640000

36

Views

Can spark-submit upload dependencies using WebHDFS?

Published on 10 November 2023 at 16:49

score 56 · Answer 20 · 2023-11-08T15:24:47.310000

56

Views

How to submit spark jobs without direct access to the datanode dfs port (dfs.datanode.address) (9866) in our case

Published on 08 November 2023 at 15:24

List Question

Apache Airflow sparksubmit

PySpark Script fails intermittently with "Failed to download resource"

Spark Driver running on Kubernetes hung with too old resource version message

Insert Overwrite partition data using Spark SQL on MINIO table

spark-submit command: --driver-class-path is working and --jars is not working

Can I use spark-submit to run a non-pyspark python code on spark, that is, without or with minimal changes?

Run non-spark python code on spark to use its distributive compute to optimize the performance

Why do spark workers write all tmp files, including shuffle and cache files, to different directories even though we define spark.local.dir?

Spark-submit java.nio.file.NoSuchFileException on kubernetes

Airflow - NameError: name 'ti' is not defined

spark submit job fails in airflow but works in container

Inconsistent results in Spark-shell and Spark-submit

How to use external Spark with the Cloudera cluster?

Error while deploying spark-submit container

Spark submit repo via a remote repository

WARN YarnScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered

when running spark-submit on yarn then error?

Caused by: java.lang.ClassNotFoundException: org.apache.kafka.common.serialization.ByteArraySerializer

Can spark-submit upload dependencies using WebHDFS?

How to submit spark jobs without direct access to the datanode dfs port (dfs.datanode.address) (9866) in our case

Trending Questions

Popular # Hahtags

Popular Questions