Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

SQL Catalog #503

Closed
wants to merge 52 commits into from
Closed
Show file tree
Hide file tree
Changes from 45 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
1409590
initialize sql catalog crate
JanKaul Mar 1, 2024
d372098
implement basic sql catalog functionality
JanKaul Mar 4, 2024
a12bd20
fix clippy warnings
JanKaul Mar 4, 2024
1790e1e
fix format
JanKaul Mar 4, 2024
08ad46f
fix cargo sort
JanKaul Mar 4, 2024
761f03f
fix ordering
JanKaul Mar 4, 2024
8685c5a
fix ordering
JanKaul Mar 4, 2024
83829e8
use prepared statements
JanKaul Mar 12, 2024
2b54962
run test
JanKaul Mar 13, 2024
ca6fe42
rebase
JanKaul Apr 8, 2024
ac8623b
fix connection pool issue for sql catalog
Apr 10, 2024
3ebb8ef
move the sqllite database creation part inside test case.
Apr 10, 2024
5e82b78
style fix and few additional logic removal and add todo check.
Apr 10, 2024
c51cd96
create sqlconfig, fix rest of the tests and remove todo
Apr 17, 2024
07b531e
use varchar
JanKaul Apr 24, 2024
f4c3196
use static strings for sql identifiers
JanKaul Apr 25, 2024
0d7fb09
fix namespace encoding
JanKaul Apr 25, 2024
1e0258c
fix typo
JanKaul Apr 25, 2024
cd34f87
Fix heading
JanKaul Apr 25, 2024
0651db0
Update crates/catalog/sql/src/catalog.rs
JanKaul Apr 26, 2024
fca5338
rename fileio
JanKaul Apr 26, 2024
88e7a32
simplify conversion from string to NamespaceIdent
JanKaul Apr 26, 2024
de91409
use uri for database connection
JanKaul Apr 26, 2024
bc69606
use features for sqlx database support
JanKaul Apr 26, 2024
5c1e7f0
use statics for pool default values
JanKaul Apr 26, 2024
9746a29
move derive Default
JanKaul Apr 26, 2024
da85c88
create new tempdir for every test run
JanKaul Apr 26, 2024
131555e
use table record type
JanKaul Apr 26, 2024
7115401
use parent for list namespaces
JanKaul Apr 29, 2024
5e3749c
fix typo
JanKaul Apr 29, 2024
077427f
fix clippy warning
JanKaul Apr 29, 2024
ec28753
use statics for namespace sql
JanKaul Apr 29, 2024
5ea41af
create namespace
JanKaul Jun 20, 2024
b356c95
remove cache
JanKaul Jun 20, 2024
ce58c76
fix list_namespaces
JanKaul Jun 20, 2024
2c27294
feat: some major mess to support pg
callum-ryan Jul 19, 2024
0e53c31
fix: remove un-needed .to_strings()
callum-ryan Jul 19, 2024
2262f64
feat: add some more bits
callum-ryan Jul 21, 2024
50295cc
feat: add some more namespace ops
callum-ryan Jul 21, 2024
17c56ad
feat: add drop table
callum-ryan Jul 23, 2024
1c55256
feat: add update namespace
callum-ryan Jul 23, 2024
0ddf09a
fix: rebase + to_url_string
callum-ryan Jul 29, 2024
dc36b83
fix: clippy + machete
callum-ryan Jul 29, 2024
6f33fae
fix: clean up matches and remove panic
callum-ryan Jul 31, 2024
327517e
fix: clean up matches once more
callum-ryan Aug 2, 2024
4bb2fde
Update crates/catalog/sql/src/catalog.rs
callum-ryan Aug 3, 2024
2c4ca25
Update crates/catalog/sql/Cargo.toml
callum-ryan Aug 3, 2024
605184f
Update crates/catalog/sql/src/catalog.rs
callum-ryan Aug 3, 2024
a16cd56
Update crates/catalog/sql/src/catalog.rs
callum-ryan Aug 3, 2024
b2ce80b
Update crates/catalog/sql/src/catalog.rs
callum-ryan Aug 3, 2024
b099ee9
Update crates/catalog/sql/src/catalog.rs
callum-ryan Aug 3, 2024
b91b818
Merge branch 'apache:main' into sql-catalog
callum-ryan Aug 5, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions crates/catalog/sql/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

[package]
name = "iceberg-catalog-sql"
version = { workspace = true }
edition = { workspace = true }
homepage = { workspace = true }
rust-version = { workspace = true }

categories = ["database"]
description = "Apache Iceberg Rust Sql Catalog"
repository = { workspace = true }
license = { workspace = true }
keywords = ["iceberg", "sql", "catalog"]

[dependencies]
async-trait = { workspace = true }
iceberg = { workspace = true }
serde_json = { workspace = true }
sqlx = { version = "0.7.4", features = ["tls-rustls", "any" ], default-features = false }
typed-builder = { workspace = true }
uuid = { workspace = true, features = ["v4"] }

[dev-dependencies]
iceberg_test_utils = { path = "../../test_utils", features = ["tests"] }
sqlx = { version = "0.7.4", features = ["tls-rustls", "runtime-tokio", "any", "sqlite", "migrate"], default-features = false }
tempfile = { workspace = true }
tokio = { workspace = true }

[features]
sqlite = ["sqlx/sqlite"]
postgres = ["sqlx/postgres"]
mysql = ["sqlx/mysql"]
Comment on lines +48 to +50
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, I'm considering whether it's possible to avoid providing features ourselves. Instead, users can active whether they want by adding sqlx in their dependencs try:

iceberg_catalog_sql = "0.x"
sqlx = { version = "0.7.4", features = ["sqlite", "postgres"] }

So we only depends on the interface of sqlx. Users can pick up whatever driver they want (even those implemented by their own).

21 changes: 21 additions & 0 deletions crates/catalog/sql/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
<!--
~ Licensed to the Apache Software Foundation (ASF) under one
~ or more contributor license agreements. See the NOTICE file
~ distributed with this work for additional information
~ regarding copyright ownership. The ASF licenses this file
~ to you under the Apache License, Version 2.0 (the
~ "License"); you may not use this file except in compliance
~ with the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing,
~ software distributed under the License is distributed on an
~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
~ KIND, either express or implied. See the License for the
~ specific language governing permissions and limitations
~ under the License.
-->

# Apache Iceberg Sql Catalog Official Native Rust Implementation

Loading
Loading