My data is coming from a source X. However, everyday the data is getting updated. So when that data is coming to lakehouse in data fabric, it is getting appended everytime. How do I overwrite the old with the new ones and also add new data.
Eg. This is my data:
When the value column is updated in source, the entire data is getting appended. Like below:
However, I want the values to be overwritten like below.
I assume you're writing data to the lakehouse using a notebook in Fabric ? If yes, you should use the
mode("overwrite")when writing the table :df.write.format("delta").mode("overwrite").saveAsTable("myTable")