[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: case insensitive tokens??



Hi.

>I made a slight modification to the lexer.txt file in the sablecc source
>directory. I replaced most of the "private" occurrence with "protected".
>This allows me to override the getchar method. If I do this I can check
>for the (lexer) state I'm in and do a conversion to lower case.


If you think that this is safe, then go for it... I have not designed
getchar to be overridden. So, you take your own risks there;-)

This is a clear example of the benefits of having the source code, and the
rights to modify it (under the conditions of the GNU GPL). When something
does not suit your needs, you can alway modify the software;-)

>I am probably going to make more extensive changes to these files. I
>want to get case insensitivity in tokens in a sub-set of lexer states. I
>think that at some point that I will have to keep the original text
>also. This will require me to change both the util.txt and lexer.txt
>files.


I am still not sure that what you are doing is safe. A small hint: you
should probably make sure that you "pushback" characters in their initial
upper/lowercase state to prevent "future" tokens in possibly different
states to receive uppercased characters...

>If you add the double quote token to SableCC, please be sure to retain a
>copy of the original text.


I can only garantee that I will try to keep a similar protected & public
interface. (I might change protected variables into get/set methods with
identical names, though).

>BTW, this use of private in library classes is a bit problematic. It
>severely restricts the ability of users of the class to extend the
>library.


Our point of view is different here, but as you can modify the software as
you wish, there is no problem:-)))

Have fun!

Etienne