I have a folder containing delta files. I try to create an external table as follows:
spark.sql("""
CREATE TABLE IF NOT EXISTS my_table
(
id INT,
name STRING
)
USING DELTA
LOCATION 'abfss://[email protected]/aaa/cc' """)
However, I get the following error:
The specified schema does not match the existing schema at abfss://[email protected]/aaa/cc
The problem is that the delta files foresee an additional column (age) which I don't want to have in my table. How can I achieve that?