Re: PushbackReader

[Nick Kramer <nkramer@jprc.com>]

>I do like to use Java standard APIs as much as possible. Knowing that I
>needed a reader with "push back" capability, I thought that the
>PushbackReader reader was exactly what I was looking for. It turns out that
>the PushbackReader class uses a fixed size "pushback" buffer, and that this
>size must be passed to the constructor.

This may be a bit heretical, but have you considered abandoning
streams/readers entirely, and just using a single large String as the lexer
input?  We're already building the entire parse tree in memory, which is
many times larger than the input file.  And using a raw String will
probably be faster than reading a single character at a time from a stream.

(I recently wrote a simple file comparison utility in Java.  At first, I
just compared the inputs one character at a time, using BufferedStreams and
all that.  Slow as hell.  I then implemented my own buffering on top of
BufferedStream, and got a speed-up of about 20x.  My theory is that the
overhead of doing a method call for each character was causing the dreadful
performance.)

>Normally, you should not be concerned with this problem. I will fix this in
>the next version of SableCC. I will then take a simple Reader as parameter
>to the Lexer constructor. I'll also implement an internal variable size
>pushback buffer.

Fair enough.  In the mean time, it's easy enough to crank up the buffer size.

-Nick Kramer