How to merge (upsert) in Python?

[hongbo-miao]

I have dataframe with a a few columns timestamp, current, voltage, temperature.
I want timestamp to be unique, like an index.

Currently below code will keep appending if I run same code on same dataframe.

write_deltalake("s3a://my-bucket/my-delta-tables/motor", df, mode="append")

Is there a way to merge (upsert)?
Something like this? https://docs.databricks.com/delta/merge.html

Thanks! 😃

[wjones127]

That's not something we've implemented yet, but we are working our way towards that. We are actively looking at Delete, and then later update. I suspect after we will start on merge.

[hongbo-miao]

Thanks @wjones127 ! Is there any workaround way such as creating a empty table with primary key first, like https://docs.databricks.com/tables/constraints.html

[wjones127]

We also don't support constraints right now, so unfortunately no.

[hongbo-miao]

Thanks for the info! It would be great to support as they are quite useful. 😃

UPDATE: the following up feature request is at #1357