How do I get a Lemon parser to terminate itself on a newline?

Question

Following this old tutorial, I am trying to get a lemon parser to automatically terminate parsing on an EOL token. The relevant part of the parser looks like this:

start ::= in .
in ::= .
in ::= in commandList EOL .
{
    printf("start ::= commandList .\n");
    printf("> ");
}

Here's how I'm executing the parser with tokens scanned by Flex:

int lexCode;
do {
    lexCode = yylex(scanner);
    Parse(shellParser, lexCode, yyget_text(scanner));
    // XXX This line should not be necessary; EOL should automatically
    // terminate parsing. :-(
    if (lexCode == EOL) Parse(shellParser, 0, NULL);
} while (lexCode > 0);

I'd like to eliminate the need to check for the EOL token here, and just let the parser figure out when it's done. How do I do that?

Thanks!

Answer 1

In EBNF terms your definition of in is

in ::= (commandList EOL)*

Which allows multiple EOLs. What you want is

in ::= commandList* EOL

Which should work out to

start ::= in EOL .
in ::= .
in ::= in commandList .

Note that this does not allow for completely empty input (not even an EOL); you can tweak things as necessary if this is a problem.