Optimizing the wasi web polyfill? #152
Comments
|
That's possible. Especially if making executables relocatable just means we add relocations, and don't necessarily change memory or table to imports like we do in .o files, so they remain executable just as they are now. Would Emscripten's runtime need to call malloc? If so, would it need the applications' malloc, or would there be two disjoint malloc heaps in the same memory? Another idea is use separate modules for application and runtime, as the current polyfill does, but do something like binaryen's metadce across them to reduce the size of the Emcsripten runtime module. That'd have the advantage that it wouldn't require relocations or modifying the WASI program at all -- you could ship the WASI program as-is to the browser and to other places, and in the browser it'd be accompanied by a specialized runtime build that just contains the things the program needs. It'd have the disadvantage of using two linear memory regions at runtime, so more memory usage. |
|
Good point about malloc, yeah - they would need to be not just relocatable, but also be able to share memory management (it won't work if both have an sbrk implementation that assumes they control all of memory except for static data and the stack). Does the wasi API provide something like sbrk, that a malloc would need to use? Or is that an internal wasi libc detail? Something like metadce across it all is possible, yeah - actually @dschuff and I had the idea yesterday that we could scan the list of wasi API calls in the executable, then populate emscripten's |
|
WASI doesn't have an API for sbrk; applications just do The second paragraph makes me a little unclear on the actual use case here. Someone using this would seem to (a) want to run their code on the Web, (b) use Emscripten, (c) be ok shipping a wasm program to which isn't strictly a WASI program, and (d) want really optimal code size. I don't yet see the value in having such a user compile to WASI first and then do a separate Emscripten compile step, instead of just compiling from source with Emscripten. |
|
The use case is someone that has a wasi executable, and wants to also run it on the web. Since on the web you must ship the wasi runtime with it, code size matters, unlike on the server where you install wasmtime or wasmer etc. once. The current polyfill is larger than it needs to be, and so it would be nice if users had a simple way to get an optimal small wasm+js from their wasi executable. I'm not opposed to people cross-compiling from source with both wasi and emscripten, that may well be the optimal thing to do, but I was hoping maybe we could make this simpler for users. |
* Use generated type bindings. Use the witx API descriptions to generate the bulk of the contents of host.rs, wasi.rs, and wasi32.rs. This also prunes out many of the miscellaneous libc definitions from those files which aren't currently in use by wasi-common. If there's anything removed that's still needed by someone, it's easy to add things back in. * Remove unneeded iovec conversion routines.
|
Is there any current planning for the things @kripken mentions above? Thanks. |
|
Over the past few days I wrote a pure js web polyfill for a subset of wasi: https://github.com/bjorn3/rust/blob/compile_rustc_wasm4/rustc.html It is just enough to get rustc compiled to wasi working. |
* Check type when resuming function * Remove pub(crate) * Update lib.rs
|
I think this can be closed now as the polyfill at https://wasi.dev/polyfill/index.html is gone. |
…ixes path_open_dirfd test. (bytecodealliance#152) * adapter: special case for NOTDIR instead of BADF fixes path_open_dirfd test. * get_dir now fails with NOTDIR for non-dir files * get rid of special case * complete test coverage for all path_ functions giving NOTDIR errors
…ixes path_open_dirfd test. (bytecodealliance#152) * adapter: special case for NOTDIR instead of BADF fixes path_open_dirfd test. * get_dir now fails with NOTDIR for non-dir files * get rid of special case * complete test coverage for all path_ functions giving NOTDIR errors
Introduce ZKASM WASM environment
…tecodealliance#152) This is just a minor cleanup, influencing where different parts of translating wasmfx instructions happens. For `resume` instructions, at some point we moved all of the translation logic out of `code_translator.rs` and into `func_environ.rs`. This means that the `Operator::Resume` case in `translate_operator` only manipulates the value stack, and then delegates the actual translation to `translate_resume` in `FuncEnvironment` (which then in turn delegates to the corresponding translation functions in the baseline or optimized implementation). This PR does the same for `cont.bind`, `cont.new`, and `suspend`, where some of the translation logic was scattered between `code_translator.rs` and `func_environ.rs`. The previous situation had the downside of requiring a bunch of `typed_continuations_*` functions (e.g., `typed_continuations_store_payloads`) to be part of the interface of `FuncEnvironment`, and then both the baseline and optimized implementation had to implement that same interface. This created some unnecessary coupling between the two implementations. After these changes, the case for any wasmfx instruction `foo` in `code_translator.rs` only manipulates the value stack, and then calls `translate_foo` in `FuncEnvironment`, which then calls `translate_foo` in the baseline or optimized implementation. In particular, no `typed_continuations_*` functions are required in the `FuncEnvironment` trait itself anymore.
The current polyfill, if I understand correctly, is an emscripten-compiled program that contains a wasi runtime implemented in POSIX. Emscripten's POSIX support is then used to run the wasi. And the polyfill program contains all the code necessary to run the entire wasi API, so that it can run an arbitrary wasi program.
@dschuff and I were thinking about the following use case: an existing wasi executable exists, and someone wants to package it for the web. Using the current polyfill would work, but it's larger than it needs to be for most programs. In theory it would be nice if we could do
emcc -Os wasi_program.wasm -o wasi_program.htmland get html+js+wasm that works and is as small as possible - that is, if emscripten could take wasi programs as input, link in necessary stuff, emit an optimized and minified JS runtime, etc. However it seems like that can't quite work, because wasi executables are not relocatable - and so we can't link in the wasm parts of the emscripten runtime.Alternatively, if a relocatable wasi was provided, that could work. However, if we're not using a plain wasi executable anymore, then we can maybe just recompile the program from source using emscripten anyhow - that is, the recommended path for people would be to make sure their programs compile with both the wasi and emscripten sdks, if they want to both run on the server and also run on the web with minimal size. But this has downsides. Maybe there's a better option?
Another possible thought here - what if wasi executables were relocatable?
The text was updated successfully, but these errors were encountered: