[Spark] Support RESTORE for clustered table #3194
Conversation
zedtang
force-pushed
the
restore-clustered-table
branch
from
f2c8b3f to
f22acc0
Compare
| verifyClusteringColumns(tableIdentifier, "a") | ||
| } | ||
|
|
||
| // Scenario 3: restore unclustered table to clustered version. |
There was a problem hiding this comment.
This comment doesn't look right? All versions below are all clustered table. Are you meaning restore from table with clustering columns to non-empty clustering columns?
There was a problem hiding this comment.
Yeah, this is more accurate, updated
| @@ -825,6 +826,45 @@ trait ClusteredTableDDLSuiteBase | |||
| } | |||
| } | |||
|
|
|||
| test("validate RESTORE on clustered table") { | |||
There was a problem hiding this comment.
Is there a column mapping enabled variant for these tests? If not let's add one since column names are maninulated
There was a problem hiding this comment.
Sure, added them to column mapping's selected tests.
| verifyClusteringColumns(tableIdentifier, "a") | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
Can we add the following test :
- Restore to latest snapshot: basically it should work for fromSnapshot = toSnapshot
There was a problem hiding this comment.
Actually we do a shortcut when fromSnapshot = toSnapshot (src)
So I modified the test to restore to the start version.
zedtang
force-pushed
the
restore-clustered-table
branch
from
bb18a1c to
4dd0c34
Compare
Support RESTORE for clustered tables by adding a new domain metadata to overwrite the existing one so that clustering columns are correctly restored. New unit tests.
Which Delta project/connector is this regarding?
Description
Support RESTORE for clustered tables by adding a new domain metadata to overwrite the existing one so that clustering columns are correctly restored.
How was this patch tested?
New unit tests.
Does this PR introduce any user-facing changes?
No