Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Reconcile data file / sst files between deltacat and rivulet #469

Open
mcember opened this issue Jan 24, 2025 · 0 comments
Open

Reconcile data file / sst files between deltacat and rivulet #469

mcember opened this issue Jan 24, 2025 · 0 comments
Assignees
Labels
enhancement New feature or request P2 Resolve if not working on P0/P1 (< 4 weeks) V2 Related to DeltaCAT V2 native catalog support

Comments

@mcember
Copy link
Collaborator

mcember commented Jan 24, 2025

Acceptance Criteria

  • Are we writing data files how deltacat expects? If not, make a storage interface for writing data files and move rivulet memtable_dataset_writer to use that
  • In PR 464, we re-use the data_files field in a manifest to point to SST files. Should we instead make an SST_files field or otherwise extend Delta/Manifest to support SST files?
  • Should we officially add "level" to delta spec or just keep it as a key that only rivulet understands?
@pdames pdames added enhancement New feature or request P2 Resolve if not working on P0/P1 (< 4 weeks) V2 Related to DeltaCAT V2 native catalog support labels Mar 24, 2025
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
enhancement New feature or request P2 Resolve if not working on P0/P1 (< 4 weeks) V2 Related to DeltaCAT V2 native catalog support
Projects
None yet
Development

No branches or pull requests

5 participants