When I save a PySpark DataFrame with saveAsTable in AWS EMR Studio, where does it get saved?

Question

I can save a dataframe using df.write.saveAsTable('tableName') and read the subsequent table with spark.table('tableName') but I'm not sure where the table is actually getting saved?

Answer 1

It is stored under the default location of your database.

You can get the location by running the following spark sql query:

spark.sql("DESCRIBE TABLE EXTENDED tableName")

You can find the Location under the # Detailed Table Information section. Please find a sample output below: