List questions from spark-kafka-integration | Devhide

Failed to find data source: kafka. Please deploy the application as per the deployment section of Structured Streaming + Kafka Integration Guide Spark

Published on 17 December 2023 at 00:20

Structured streaming with spark-kafka 3.3.2 failing

Published on 16 December 2023 at 20:07

why pyspark task not stop？

Published on 24 November 2023 at 08:13

Spark-Kafka Integration not working: Kafka broker with producer and consumer script are getting stuck as soon as we run spark script(consumer.py)

Published on 04 October 2023 at 07:26

How to add Kafka dependencies for PySpark on a Jupyter notebook

Published on 16 August 2023 at 18:05

Spark Kafka: understanding offset management with enable.auto.commit

Published on 25 July 2023 at 09:07

Spark Streaming: How to handle failure in Spark when connecting to multiple kafka cluster via Union of Dstream?

Published on 10 July 2023 at 22:53

Running Kafka and Spark with docker-compose

Published on 12 May 2023 at 16:02

How to map a message to a object with `schema` and `payload` in Spark structured streaming correctly?

Published on 27 April 2023 at 22:06

Difference between spark-streaming-kafka-0-10 vs spark-sql-kafka-0-10

Published on 25 April 2023 at 02:38

Getting Error for org.apache.spark.sql.Encoder and missing or invalid dependency find while loading class file SQLImplicits, LowPrioritySQLImplicits

Published on 19 April 2023 at 09:36

What is the best way to set up a kafka connection with apache spark

Published on 25 March 2023 at 02:28

I have more data in a kafka topic but when i extract data using my pyspark application, I am getting only 1 row extracted, how to fix?

Published on 01 March 2023 at 07:37

How to read DF Column of struct type and add they key value pairs to kafka headers?

Published on 09 February 2023 at 21:20

spark streaming from kafka on spark operator(Kubernetes)

Published on 09 February 2023 at 15:19

Pyspark : Kafka consumer for multiple topics

Published on 21 January 2023 at 09:08

I am facing "java.lang.NoSuchMethodError: org.apache.spark.kafka010.KafkaTokenUtil$.needTokenUpdate" error while working with pyspark

Published on 16 January 2023 at 08:27

java.lang.ClassNotFoundException: org.apache.spark.sql.kafka010.KafkaSourceRDDPartition

Published on 05 January 2023 at 13:06

Error while pulling kafka jks certificates from hdfs (trying with s3 as well) in spark

Published on 04 January 2023 at 15:13

How to process the dataframe which was read from Kafka Topic using Spark Streaming

Published on 15 December 2022 at 11:15