Handling single quotes

[d0ssant0s]

Hi, how should we handle single quotes in particle? For example, how could make the INI test example from _examples/ini/main_test.go pass with a single quoted string? I've tried to play around with a custom lexer (please see below), but I suspect I'm yet to understand how it is supposed to work. Could someone give me a hint?

func TestExe(t *testing.T) {
	ini := &INI{}
	err := parser.ParseString("", `
global = 1

[section]
value = 'str'
`, ini)
	require.NoError(t, err)
	repr.Println(ini)
}

var iniLexer = lexer.MustSimple([]lexer.SimpleRule{
	{`Ident`, `[a-zA-Z][a-zA-Z_\d]*`},
	{`String`, `"(?:\\.|[^"])*"`},
	{`StringSingle`, `'(?:\\.|[^'])*'`},
	{`Float`, `\d+(?:\.\d+)?`},
	{`Punct`, `[][=]`},
	{"comment", `[#;][^\n]*`},
	{"whitespace", `\s+`},
}) 

var parser = participle.MustBuild(&INI{},
	participle.Lexer(iniLexer),
	participle.Unquote("String", "StringSingle"),
)

Thank you in advance!

[alecthomas]

This looks about right, except Unquote won't work with single quoted strings. You'll have to use the Map option and write a custom unquoter, which just strips the single quotes.

[d0ssant0s]

Hi, thank you for the hint, but now with:

func StripSingleQuotes(types ...string) participle.Option {
	return participle.Map(func(token lexer.Token) (lexer.Token, error) {
		token.Value = strings.ReplaceAll(token.Value, "'", "")
		return token, nil
	}, types...)
}

var parser = participle.MustBuild(&INI{},
	participle.Lexer(iniLexer),
	StripSingleQuotes("StringSingle"),
	participle.Unquote("String"),
)

I'm getting the following error:

$ go test -v
=== RUN   TestExe
    main_test.go:18: 
        	Error Trace:	main_test.go:18
        	Error:      	Received unexpected error:
        	            	5:9: unexpected token "str" (expected Value)
        	Test:       	TestExe
--- FAIL: TestExe (0.00s)
FAIL
exit status 1
FAIL	github.com/alecthomas/participle/v2/_examples/ini	0.005s