[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

epsilon tokens



Hi!

I'm a novice SableCC user and have run into the following tidbit:
I need to occasionally recognize epsilons (zero-length tokens)
in a grammar (yes, there IS a reason for that).  The problem
can be extracted to and illustrated by the following toy grammar:

==== eps.sablecc ====

Package eps;

States
 initial, rest;

Tokens
 {initial -> rest} eps = ;
 {rest} char = [0x0000 .. 0xffff];

Productions
 whole = eps char*;

==== ends here ====

Here, I just want to have an epsilon token in front of everything
else, and the parser doesn't have any other rules to go on, so
this seems pretty deterministic.  Indeed, SableCC 2.17.3 and javac
compile this grammar without a hitch.  On use, however, I always
get the following error message:

eps.lexer.LexerException: [1,1] Unknown token: e, state: 0
 at eps.lexer.Lexer.getToken(Lexer.java:329)
 at eps.lexer.Lexer.next(Lexer.java:105)
 at eps.parser.Parser.parse(Parser.java:262)
 at eps.tool.TreeBuilder.getNode(TreeBuilder.java:43)
 at eps.tool.PrintTree.main(PrintTree.java:35)

And the sample text I'm parsing starts with an "e" character.

Any ideas or pointers on what I am missing here?  I scoured
the great "thesis.html" documentation and peeked a bit into
the sources, but came up empty.

Thanks,
VEROK Istvan