Re: SableCC-2.6 bug?

[Etienne Gagnon <egagnon@sprynet.com>]

Amancio Hasty wrote:
> 
> ...
> It appears that SableCC gets confused if the production rule
> includes a token who is a proper subset of another token in the
> rule.

SableCC is behaving properly. Let me restate that SableCC lexers match
the longest token, and then, if there are two tokens that can be
matched, it chooses the first listed token in the Tokens section.

This means, concretely, that "aaa" is matched as a TAlpa, because "aaa"
could be both a TAlpa or a TLetter. But TAlpa is listed before TLetter,
so TLetter is matched.

It is important to see that the lexer operates independently from the
parser. This means that the lexer won't look at the grammar and see that
a TLetter is expected and try to match it.

It is also important to note that TLetter will never be matched, because
TAlpa completely hides it.

Have fun!

Etienne