Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Browser storage support in WebAssembly bindings #531

Open
penberg opened this issue Dec 21, 2024 · 1 comment
Open

Browser storage support in WebAssembly bindings #531

penberg opened this issue Dec 21, 2024 · 1 comment
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed JavaScript bindings

Comments

@penberg
Copy link
Collaborator

penberg commented Dec 21, 2024

Currently, the Wasm binding implement I/O for Node filesystem APIs. Let's work on supporting OPFS, for example, to run limbo-wasm in the browser. One complication is that the current API is synchronous like better-sqlite3 so we either need to use synchronous OPFS or implement libSQL async-style API.

@penberg penberg added enhancement New feature or request help wanted Extra attention is needed good first issue Good for newcomers JavaScript bindings labels Dec 21, 2024
vishwamartur added a commit to vishwamartur/limbo that referenced this issue Dec 21, 2024
Related to tursodatabase#531

Add support for OPFS in WebAssembly bindings for browser storage.

* **bindings/wasm/lib.rs**
  - Update `Database` constructor to handle OPFS option.
  - Modify `PlatformIO` to use OPFS for I/O in the browser.

* **bindings/wasm/vfs.js**
  - Replace Node `fs` module with OPFS API.
  - Implement OPFS methods for `open`, `close`, `pread`, `pwrite`, `size`, and `sync`.

* **bindings/wasm/docs/api.md**
  - Update documentation to mention OPFS support.
  - Add examples demonstrating usage with OPFS.

* **bindings/wasm/examples/drizzle.js**
  - Update example to use OPFS for database storage.

* **bindings/wasm/examples/example.js**
  - Update example to use OPFS for database storage.

* **bindings/wasm/integration-tests/tests/test.js**
  - Update tests to use OPFS for database storage.
@BLucky-gh
Copy link

I just looked into it and synchronous OPFS is not that simple. There is FileSystemSyncAccessHandle that has synchronous read/write/etc functions, but 1) it is only obtainable in a web worker (to prevent blocking the main thread with sync io) and 2) you can only get it from a FilesystemFileHandle which you can only get via async means. You can still make it work with an awkward wrapper for bindings/wasm that creates a web worker, creates the sync handle, and then passes that to the Database constructor, and interfaces with the main thread with message passing, which would also incur non-insignificant overhead. So I feel like an async API would be a better option here.

Now that I'm writing this I figured there's also the option to expose an in-memory only db client, and let the API consumer just dump the db (raw bytes or sql dump) to be persisted however the consumer wants, be it OPFS, base64'd into LocalStorage, sent to a backend, or any other less than perfect option, but that is very much a bandaid solution since it would be impossible to guarantee that it gets persisted correctly and safely, among a myriad of other issues

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed JavaScript bindings
Projects
None yet
Development

No branches or pull requests

2 participants