[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: New Lexer

Philippe wrote:
> Another question: Why do you use a pushback reader?
> I want to be sure that I do not need a pushback reader (because this will
> be difficult to install on a network link).

The pushback reader is used by the lexer to "unread" characters.


  end = 'end';
  endif = 'endif';

Now, feed this lexer the following string: "endimage"

It is easy to see that the lexer won't know that it must return a "TEnd"
token before reading the "m". When it does realize this, it must
pushback the "m" and the "i". The next time, the lexer will be able to
scan "image" (and not "age").

I assume that your native lexer will take care of all these details. So,
all you need to do is override the getToken() method. You shouldn't
worry about anything else.


public class SomeLexer extends Lexer
  final static int BLAH = 0;
  final static int OTHER_BLAH = 1;

  // Constructor

  protected Token getToken()
    native.scanNextToken(); // scan the next token

      case BLAH:
        return new TBlah(native.getLine(), native.getColumn(),
      case OTHER_BLAH:
        return new TBlah(native.getLine(), native.getColumn(),
    throw new RuntimeException("Shouldn't reach this point;-)");

You should be able to work the details out:-)