Why does a show() on a pyspark.sql.dataframe.DataFrame cause Zeppelin to hang?

33 Views Asked by At

I have a simple paragraph of Python code in Apache Zeppelin 0.10.0:

%livy.pyspark3
handMadeFrame = spark.createDataFrame(
    [
        (1, "foo"),  # create your data here, be consistent in the types.
        (2, "bar"),
    ],
    ["id", "label"]  # add your column names here
)

print(type(handMadeFrame))
handMadeFrame.printSchema()

It prints:

<class 'pyspark.sql.dataframe.DataFrame'>
root
 |-- id: long (nullable = true)
 |-- label: string (nullable = true)

All good so far. If I add either:

handMadeFrame.take(2)

or

handMadeFrame.show()

Zeppelin hangs. Why? How can I debug this?

0

There are 0 best solutions below