[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: New Lexer



> So my question, what must I do to write my own lexer?


The easiest way to write your own lexer is to let SableCC write most of
it for you;-) Let me explain.

You write your grammar as usual, but you leave the right-hand side of
your token definitions empty.

Package
  ...
Helpers
  ...
Token
  blah = ;
  other_blah = ;
  ...
Ignored Tokens
  ... //(if any)
Productions
  ...

What SableCC will do for you is to create a Lexer class (and Parser,
...), as well as all the TOKEN CLASSES. This can save you much time;-)

Then, all you need to do is to create a new SomeLexer class that
overrides the next() and peek() methods. Be sure to instatiate the
appropriate Token class (e.g. TBlah(), TOtherBlah()) for each scanned
token. 

In reality, I should have made the "getToken" method "protected". This
is the methods that you really need to override. I will do this today.
So, you won't need to override next and peek. Only getToken;-)

Does it all make sense to you?

Etienne