Change liquidjs parameter parsing to use Liquid’s Tokenizer

[zachleat]

https://liquidjs.com/tutorials/parse-parameters.html#Parse-Parameters-as-Values
https://liquidjs.com/tutorials/parse-parameters.html#Parse-Key-Value-Pairs-as-Named-Parameters

Currently implementation predated those features and is using a one-off parser using moo: https://github.com/11ty/eleventy/blob/0f44f756a30247515033945e74e171a293bc75d5/src/Engines/Liquid.js

I think due to the size of this change, we’ll probably want a configuration escape hatch for folks to swap back to the old parameter parser

Excellent prior art #1263 #1733

[AleksandrHovhannisyan]

Hey, just checking to see if you're accepting help/contributions for this issue. I noticed that 11ty's Liquid engine is currently only using moo in two places:

eleventy/src/Engines/Liquid.js

Line 24 in 0f44f75

this.argLexer = moo.compile(Liquid.argumentLexerOptions);

eleventy/src/Engines/Liquid.js

Line 100 in 0f44f75

lexer = moo.compile(Liquid.argumentLexerOptions);

By size of the change, do you mean that it's high impact, or are there other changes that would need to be made as well?

[jgerigmeyer]

@zachleat Any thoughts on @AleksandrHovhannisyan's question above? Would you be interested in contributions for this issue? Thanks!

[zachleat]

Starting in Eleventy v3.0.0-alpha.18 and newer you can use eleventyConfig.setLiquidParameterParsing("builtin"); to use the parameter parser provided by Liquid.js. Notably this is an opt-in feature and is not enabled by default in 3.0.

The biggest difference y’all will see with this swap is that with Liquid’s provided tokenizer commas are not allowed to separate parameters. In the one-off parameter parser Eleventy supplied, commas were optional. This commas-optional feature is advertised prominently in the docs: https://www.11ty.dev/docs/shortcodes/ Commas are now optional thanks to harttle/liquidjs#724

[zachleat]

Related: harttle/liquidjs#724

[zachleat]

Temporary docs preview URL deploying here: https://11ty-website-git-v3-11ty.vercel.app/docs/languages/liquid/#shortcode-parameter-parsing

[marioortizmanero]

Could the documentation include an example of a Liquid shortcode that uses named parameters? I can't get it to work with:

eleventyConfig.setLiquidParameterParsing("builtin"); // in the config

eleventyConfig.addShortcode("example", function (name, age) {
  return `<p>Name: ${name}, Age: ${age}</p>`;
}); // in the config, later

{% example name: "abc", age: 3 %} // in the markdown file

and I wonder if I'm doing something wrong. Particularly, I'm not sure what the function definition of the shortcode should look like: multiple parameters, a dictionary, something else? The parameters name and age are always undefined. Alternatively, this might be a bug in the implementation?

[aliou]

@marioortizmanero I am having the same issue. Looking at the LiquidJS docs and the commit introducing Liquid's Tokenizer, it seems like named parameters are not supported for Liquid shortcodes (yet?).

For now, I ended up doing a variation of the figure block mentioned in this article. I hope this helps.

[tettoffensive]

@marioortizmanero @zachleat Yea. this isn't working for me either. I get undefined as well.