Support composite grammars.

[otac0n]

It would be nice to have a grammar library where you could import a grammar, something like this (syntax tentative):

@include "csharp.peg"

Now that the grammar is included, you would then be able to do something like this:

code = "{" body:csharp::FunctionBody "}" { body }

This would allow reuse and embedding of other grammars.

[otac0n]

Another way to do this would be to support external and/or custom implemented rules.

something like this:

@members
{
    private readonly cSharpParser = new CSharpParser();
}
...
code = "{" body:@this.cSharpParser.Expression "}" { body }

[otac0n]

I'm now convinced that this should be done similar to the following:

@using Roslyn.Compilers.CSharp

start
  = _ name _ "{" _ csExpression _ "}" _

name
  = [a-zA-Z] [a-zA-Z0-9]*

csExpression <ExpressionSyntax> = #CUSTOM(ref Cursor cursor) {
    var startCursor = cursor;
    var parsed = Syntax.ParseExpression(cursor.Subject, cursor.Location);
    if (parsed.FullSpan.Length == 0)
    {
        return null;
    }
    cursor = cursor.Advance(parsed.FullSpan.Length);
    // If using lexical rules, you could now walk the "parsed" value to add LexicalElements to the cursor.
    return this.ReturnHelper(startCursor, ref cursor, () => parsed);
}

EOF
  = !.

_
  = '' [ \t\r\n]*

And, of course, other Pegasus parsers could be invoked more simply once #42 is implemented:

otherParserRule <Type> = #CUSTOM(ref Cursor cursor) {
    return this.otherParser.ParseBlah(ref cursor);
}

[otac0n]

Slightly better syntax idea, using the previous example.

otherParserRule <Type> = #(ref cursor) {
    return this.otherParser.ParseBlah(ref cursor);
}

Allowing this anywhere would be nifty and would very easily lead to higher-order rules (#32).

[otac0n]

How do we deal with left-recursion and zero-width detection, given the above scheme?

[otac0n]

I think we will assume that custom rules will potentially be zero-width and that they may be assertions. We will allow reentrancy at the programmers own risk. Know what you are doing, but nesting languages should be allowed. It is the programmers responsibility to handle left-recursion properly.

[otac0n]

Here's a proposed syntax for the most complex case I can imagine (calling another parser, and copying its result into the lexical collection):

nestedExpression <OtherLanguageExpression> -lexical =
    "{"
    expr:#PARSE{{
        OtherLanguageExpression expression;
        var success = this.otherLanguageParser.TryParseExpression(state.Subject, state.Index, out expression);
        if (!success)
        {
            return null;
        }

        var startCursor = state;
        var lexical = state["_lexical"] as ListNode<LexicalElement>;
        /* snip about 15 lines of code that copies the lexical structure of `expression` into `state` */
        return this.ReturnHelper(startCursor, ref state, _ => expression, name: "nestedExpressionInner");
    }}
    "}"
    { expr }

It would simplify significantly for other Pegasus parsers that eventually expose -public rules:

integer <int> = #PARSE{ this.otherPegasusParser.parseFoo(ref state) }

[otac0n]

This is completed in the develop branch.