Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[590] Add interfaces for CatalogSyncClient and CatalogSync #603

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

vinishjail97
Copy link
Contributor

@vinishjail97 vinishjail97 commented Dec 18, 2024

Important Read

  • Please ensure the GitHub issue is mentioned at the beginning of the PR

What is the purpose of the pull request

Add interface for CatalogSyncClient which allows syncing table format metadata to a catalog.
Add interface for CatalogTableIdentifier and an implementation HierarchicalTableIdentifier for a three level naming convention. This allows future implementations to be plugged in following a different convention.
Add a new class CatalogSync syncs the canonical object (InternalTable) to multiple catalogs.
Introduce a new method getCurrentTable in the interface ConversionSource which extracts the latest table state and will be used when using CatalogSync

See RFC PR for more details about design and flow chart -> #605

Brief change log

(for example:)

  • Add interface for CatalogSyncClient and CatalogSync
  • Introduce getCurrentTable method in ConversionSource interface for catalog sync

Verify this pull request

(Please pick either of the following options)

This change added tests and can be verified as follows:

(example:)

  • xtable-api/src/test/java/org/apache/xtable/spi/sync/TestCatalogSync.java
  • xtable-api/src/test/java/org/apache/xtable/spi/sync/TestCatalogUtils.java
  • xtable-core/src/test/java/org/apache/xtable/delta/ITDeltaConversionTargetSource.java#getCurrentTableTest
  • xtable-core/src/test/java/org/apache/xtable/hudi/ITHudiConversionSourceSource.java#getCurrentTableTest
  • xtable-core/src/test/java/org/apache/xtable/iceberg/ITIcebergConversionTargetSource.java#getCurrentTableTest

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant