Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

feat: Implement load table api. #89

Merged
merged 14 commits into from
Nov 8, 2023
Merged

Conversation

liurenjie1024
Copy link
Collaborator

@liurenjie1024 liurenjie1024 commented Oct 31, 2023

This pr implements load table in rest catalog, also add accesses to table metadata.

related: #60

crates/catalog/rest/src/catalog.rs Show resolved Hide resolved
crates/iceberg/src/spec/table_metadata.rs Show resolved Hide resolved
crates/iceberg/src/spec/table_metadata.rs Outdated Show resolved Hide resolved
crates/iceberg/src/spec/table_metadata.rs Show resolved Hide resolved
crates/iceberg/src/spec/table_metadata.rs Show resolved Hide resolved
crates/iceberg/src/spec/table_metadata.rs Show resolved Hide resolved
crates/iceberg/src/spec/table_metadata.rs Show resolved Hide resolved
crates/iceberg/src/spec/table_metadata.rs Show resolved Hide resolved
crates/iceberg/src/spec/table_metadata.rs Show resolved Hide resolved
"Creating table not supported yet!",
))
async fn load_table(&self, table: &TableIdent) -> Result<Table> {
let request = self
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the typical pattern here in Rust? I would split this out into a function

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean the construction of http request? I think they are simple enough, and moving them to another method doesn't help much in avoid duplication, how do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For Python we split it out, because we're also doing some additional things: https://github.com/apache/iceberg-python/blob/main/pyiceberg/catalog/rest.py#L209-L240 But we can refactor this later as well 👍


let resp = self
.client
.query::<LoadTableResponse, ErrorResponse, OK>(request)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took a look at the error response and they all same schema: ErrorModel.

They are all converted to iceberg, and left to user. All necessary information are kept, such as type, code, message, event stacktrace. You can refer to :

impl From<ErrorModel> for Error {

crates/catalog/rest/src/catalog.rs Show resolved Hide resolved
crates/catalog/rest/src/catalog.rs Show resolved Hide resolved
crates/catalog/rest/src/catalog.rs Show resolved Hide resolved
}

impl Table {
/// Returns table identifier.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for context, in PyIceberg we also keep a reference to the config, in case you want to refresh the table.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's already kept in TableMetadata?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, you mean reference to Catalog? If so, this may not be desirable since it add life time of Catalog instance to Table.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Python this makes a few things easier, but we can also defer that. Easier to add later on, than removing it :D

@liurenjie1024
Copy link
Collaborator Author

CC @Xuanwo @Fokko Any other comments?

Copy link
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, let's go!

Copy link
Contributor

@Fokko Fokko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @liurenjie1024 and @Xuanwo for the review 👍

@Fokko Fokko merged commit e26bda3 into apache:main Nov 8, 2023
6 checks passed
@liurenjie1024 liurenjie1024 deleted the renjie/issue-60-2 branch November 8, 2023 09:31
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants