[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Pushback error!

Israel Antezana Rojas wrote:
> > You will need to increase the size of the pushback buffer.
> Yes, thank you I make the buffer size as big as the input file.
> > Question: Are you using lexer states?
> Yes. Why?

Because... I made a mistake when I designed the "implementation" of
lexer states.  This has an infortunate consequence:  the lexer can look
ahead many more characters than it is necessary when scanning tokens
defined using lexer states.  A nasty side effect is that you sometimes
need a very big pushback buffer.  In summary, when you use lexer states,
you suffer from:
(1) slow scanning (because lookaheads are rescanned for every token).
(2) high memory requirement (large pushback buffer).

I should have a fix for this tomorrow.  So: keep your states as they

I apologize for the inconvenience.  The good side of this is that I
learned something new that wasn't written in a book.  Sometimes,
experiments (and mistakes) are a very good way of learning;-) 

Once you get the fix, you should be able to reduce considerably the size
of the pushback buffer. 

Etienne M. Gagnon, M.Sc.                     e-mail: egagnon@j-meg.com
Author of SableCC:                 http://www.sable.mcgill.ca/sablecc/