Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Add support for OpenDAL for store #258

Open
prabirshrestha opened this issue Aug 26, 2024 · 4 comments
Open

Add support for OpenDAL for store #258

prabirshrestha opened this issue Aug 26, 2024 · 4 comments

Comments

@prabirshrestha
Copy link

Would be great if Apache OpenDAL is supported by for Store. It supports tons of file systems include s3 and would make it easy to use other filesystems besides s3. you can find the list of supported filesystems at https://docs.rs/opendal/latest/opendal/services/index.html.

@paulgb
Copy link
Member

paulgb commented Aug 26, 2024

Neat, I'll look into this. I imagine that any non-S3 backend would have to be something that Y-Sweet explicitly exposed through the binary (unless it were embedded into another server as a library); is there a backend that OpenDAL would enable that you are interested in in particular?

I'm also planning to evaluate slatedb.io as a storage layer, and it happens that slatedb uses the object_store crate which abstracts over storage backends. I see that OpenDAL has an object_store integration, so maybe that's a path here.

@prabirshrestha
Copy link
Author

I'm primarily looking for the SurrealDB integration. Internally it supports in memory, rocksdb, tikv and foundation db.

@okatrppr
Copy link

SurrealDB has wasm also, https://github.com/surrealdb/surrealdb.wasm
So y-sweet can solve the crdt for both server side surrealdb and wasm side surrealdb, great for offline first apps

@paulgb
Copy link
Member

paulgb commented Dec 13, 2024

I'm working on a rewrite (for other reasons) that happens to use object_store, which would make it possible to create a light wrapper around the y-sweet package to use other backends supported by object_store, including OpenDAL.

I'm generally wary of this, because the further you get from S3-like object storage, the less some of the design decisions made by Y-Sweet (especially in the rewrite) make sense, even if you can shoehorn any KV store into "looking like" an object store.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants