topiary.tweag.io Interactive Playground

[Xophmeister]

Is your feature request related to a problem? Please describe.
To prototype new queries and to play with Topiary as a formatting engine (e.g., to drive adoption), it would be really useful to have an online, interactive playground at, say, topiary.tweag.io (or similar). These are very common in this space: e.g., Tree Sitter, Shellcheck, Rust, Black (third-party playground), etc.

Describe the solution you'd like
Provide a simple playground website (frontend and backend), with the following widgets:

• Source code input
• Source code grammar selector
• Topiary query input (defaults to the bundled language queries, based on grammar selector)
• Output panel

The output panel would update interactively, on changes to the input widgets.

Describe alternatives you've considered
It's quite easy to rig this up in the terminal, for prototyping and playing, but there's a non-zero amount of friction.

Additional context
Many playgrounds allow you to save/share state. This would be a "nice-to-have", but not strictly necessary, IMO.

[Xophmeister]

Reporting findings, so far, so they don't get lost:

WASM vs. REST

A REST API would be very easy to write. However, that would then require a long-term commitment on server infrastructure and all that entails.

A WASM target is harder to write, presuming it can be done, but then it can be statically hosted with a trivial maintenance burden.

Is WASM possible?

The Tree-sitter playground is using WASM. Also, the Rust Sitter project contains this quote in its README:

Note: By default, Rust Sitter uses a fork of Tree Sitter with a pure-Rust runtime to support wasm32-unknown-unknown. To use the standard C runtime instead, disable default features and enable the tree-sitter-standard feature

That highly suggests that, not only is it possible, but gives two departure points to see how it's done.

Notes and Resources

• Rust and WebAssembly
  This contains a bunch of resources, including a book. It was a struggle to get the book exercises to work as described -- things have moved on, since it was written -- but it's possible, albeit leaving one with a somewhat unsatisfied feeling from the superficial understanding.

• Nix
  Getting the Rust WASM book exercises to work with Nix was a particular challenge. This gist helped, but may also be subject to drift. There is also tweag:rust-wasm-nix...

[torhovland]

Looking a bit more into what Rust Sitter does, I don't think it applies to us.

They are using the tool C2Rust to transpile Tree-sitter to Rust code. It then becomes easy to build Wasm using cargo build --target wasm32-unknown-unknown.

We could have done the same, but the problem is all the various Tree-sitter grammars that also have to build C code as well as depend on the Tree-sitter crate. Rust Sitter doesn't have that problem, because they don't depend on any grammars. They are a tool for creating grammars.

The Tree-sitter crate actually consumes grammars as C code. Reworking this as well as all the grammars to Wasm sounds non-trivial.

Will now take a look at how the Tree-sitter playground works.

[torhovland]

The Tree-sitter Playground is nice because it is built as one tree-sitter.wasm file without the grammars. And each time you select a language from a drop-down list, a corresponding tree-sitter-<language>.wasm is downloaded.

[aspiwack]

Let me point out that @TerrorJack has extensive experience with Wasm, in particular with compiling C code with Wasm. So don't hesitate to ask him for advice.

[torhovland]

I think the way to do this would be something like this:

• Make Tree-sitter and all the grammars optional dependencies, so we can compile without them when building Wasm.
• Replace the calls to Tree-sitter and grammars with stubs so the code compiles.
• Replace the stubs with target-specific (if Wasm) bindings to some Javascript glue code that can invoke code from the Tree-sitter and grammar Wasm modules.

That's the general idea, anyway. Will update this strategy when I know why it doesn't work 😀

[torhovland]

A basic version is now implemented and deployed to https://topiary.tweag.io/playground. Let's log any suggested improvements as separate issues.

[torhovland]

Follow-up issues are logged in #356, #357, #358, #359, #360, #361, #362, and #365.

@Xophmeister Can we close this issue?

[Xophmeister]

Definitely 👍