Add plugin support

[dosisod]

I am currently writing a tree-sitter grammar for a DSL (domain specific language), and would like to add Difftastic support to it as well. This DSL won't be super useful to the general Difftastic user, though, from what I can tell, the only way to add a new parser is to merge the code in here, or fork it and add it to the fork.

I propose that we allow for a "plugin" system of sorts: Basically, you create a .so file which contains:

• The TreeSitterConfig data (which calls tree_sitter_xyz())
• A guess function which returns true/false if the passed file should use this new plugin

And then you run difft like so:

$ difft --load plugin.so old.dsl new.dsl

At least, that's how I would go about it (from what I can see from briefly looking at the code). Would this be something you would be interested in? If we decide it is worthwhile I would totally be willing to implement it, though I will say I am somewhat new to Rust.

[luolong]

I would like to add that there's a growing number of tools that support Tree-Sitter grammars in one way or another and it would be nice to be able to install those grammars in a single place on OS and have tools pick them up dynamically somehow, with all tools having same (or mostly overlapping) set of supported languages.

Currently each tool solves this slightly differently - some, like difftastic embed languages at compile time, while others (Neovim for example), supports some sort of dynamic loading and even compilation of language definitions.

It would be nice for the tools to agree on some sort of shared common ground and plug-in system for sharing grammar definitions.

[dosisod]

@luolong Here is some tools I found which use tree-sitter, all of which have their own way about doing module loading:

• Zee Editor: Loaded from ~/.config/zee/grammars
• Diffsitter: Uses config file which points to .so files
• Semantic (used on GitHub): Manual source integration, no plugins from what I can tell

And then of course a plethora of libraries which solely rely on tree-sitter itself to provide the grammars (and thus don't support modular loading of grammars). I feel like every program has their own way of doing it because tree-sitter itself does not provide it. Although we could talk to every program author that uses tree-sitter and see if we can come to some sort of consensus, I feel like a better (and more sustainable long term) solution is to start a conversation with the tree-sitter devs themselves, and see how they would approach this.

The only issue though is that we have additional code we need to run for each grammar, so we still need a way to dynamically load the code for that, in addition to the tree-sitter provided grammar.

[Wilfred]

https://crates.io/crates/tree-sitter-loader looks very relevant here.