help with lexer

[Archie Cobbs <archie@whistle.com>]

Hmm.. I'm not sure how to do what I want to do with SableCC... 

I have a line-based protocol running over TCP and want to create
a parser that the server will use. There is exactly one command
per line, where a line is terminated with \n.

Ther is one twist: backslash continuations are allowed... that is,
if a newline immediately follows a backslash, both characters are
ignored and the command continues on the next line.

I want the server to be able to parse the client's input, with
these things being true:

 o Interactivity - the server should not read any more characters
   than absolutely necessary to determine the correct token (ie,
   it should not read past the terminating newline character).

 o Error recovery - if there's any lexical or parse error, the server
   should read and discard characters up to & including the next newline
   character (with flex, you'd define an "error command" that matches
   the error token).

 o Repeatabilty - the server should be able to repeatedly call the
   parsing engine to parse out consecutive AST's (ie, commands) from
   the input stream.

I'm having trouble doing this with SableCC, mainly the last point,
because it always expects to parse the first production followed by
an EOF token. In bison, you could use YYACCEPT after parsing a
"command" production to make the parser accept & stop. With SableCC,
I'm trying to insert an EOF token after reading the newline, but
it seems impossible to insert an EOF token in the filter() routine
without violating the interactivity requirement.

Any ideas appreciated! Mainly, what would be the most correct/elegant
approach with SableCC to a design problem like this in terms of
satisfying those three requirements.

Thanks,
-Archie

___________________________________________________________________________
Archie Cobbs   *   Whistle Communications, Inc.  *   http://www.whistle.com