Patch to make ParserException accept and return Tokens.

[willh@msoft.com (Will Hartung)]

We discussed this a little before, and I just got around to getting it
done.

I needed to add an "include" functionality to the Sable generated
lexer, and that I was able to do by subclassing the generated Lexer
with my own version.

The problem was that now that I have "include" functionality, I don't
have any information for error messages that tell me what file the
error was in. This is mostly because the filename information isn't
known to the lexer or the parser.

Etienne and I talked a bit about this and the best way to help enable
this was to let the Parser send the token back with it's generated
exception, then you can use the token for assorted things to get the
information you need.

What I ended up doing was I have a HashTable similar to the setIn and
setOut tables in the parser, and I have the myLexer.filter() (which
knows about filenames) push every token into this HashTable with the
file name.

When the parser gets an exception, it'll return the token, and I can
look up the filename using that token.

Anyway, below is a diff that should be suitable for a patch to enable
Sablecc to pass the token back with the parser exception. It's pretty
short.

Most of the code was in the parser.txt file "macro", but it turns out
I had actually modify a bit of java to add an import to the
ParserException class.

Anyway, here's the patch. I know I should be using the CVS stuff, but
I'm just not familiar with the whole process, and don't have the time
to get all the wiley components set up here to hook up to it. These
changes are minor, and there's more "patch" infomation than real
information in this thing. It should be fairly trivial to add by hand
if you need to, or to add to an older version.

Also, it's doesn't help with actual Lexer errors yet.

Enjoy!

Will Hartung
(willh@msoft.com)

-----------
diff -c -r sablecc-2.14.2/src/org/sablecc/sablecc/GenParser.java
sablecc-2.14.exp/src/org/sablecc/sablecc/GenParser.java
*** sablecc-2.14.2/src/org/sablecc/sablecc/GenParser.java Mon Dec 25
08:38:06 2000
--- sablecc-2.14.exp/src/org/sablecc/sablecc/GenParser.java Fri Jan 19
14:16:27 2001
***************
*** 769,775 ****

          try
          {
!             macros.apply(file, "ParserException", new String[] {pkgName});
          }
          catch(IOException e)
          {
--- 769,776 ----

          try
          {
!             macros.apply(file, "ParserException", new String[] {pkgName,
!                 ids.pkgName.equals("") ? "node" : ids.pkgName + ".node"});
          }
          catch(IOException e)
          {
diff -c -r sablecc-2.14.2/src/org/sablecc/sablecc/parser.txt
sablecc-2.14.exp/src/org/sablecc/sablecc/parser.txt
*** sablecc-2.14.2/src/org/sablecc/sablecc/parser.txt Mon Dec 25 08:38:06
2000
--- sablecc-2.14.exp/src/org/sablecc/sablecc/parser.txt Fri Jan 19 14:30:44
2001
***************
*** 30,35 ****
--- 30,36 ----
      private int last_shift;
      private int last_pos;
      private int last_line;
+  private Token last_token;
      private final TokenIndex converter = new TokenIndex();
      private final int[] action = new int[2];

***************
*** 211,216 ****
--- 212,218 ----

              last_pos = lexer.peek().getPos();
              last_line = lexer.peek().getLine();
+    last_token = lexer.peek();

              int index = index(lexer.peek());
              action[0] = actionTable[state()][0][1];
***************
*** 272,278 ****
                          return node;
                      }
                  case ERROR:
!                     throw new ParserException(
                          "[" + last_line + "," + last_pos + "] " +
                          errorMessages[errors[action[1]]]);
              }
--- 274,280 ----
                          return node;
                      }
                  case ERROR:
!                     throw new ParserException(last_token,
                          "[" + last_line + "," + last_pos + "] " +
                          errorMessages[errors[action[1]]]);
              }
***************
*** 402,413 ****

  package $0$;

  public class ParserException extends Exception
  {
!     public ParserException(String  message)
      {
          super(message);
      }
  }

  $
--- 404,424 ----

  package $0$;

+ import $1$.*;
+
  public class ParserException extends Exception
  {
!  Token token;
!     public ParserException(Token token, String  message)
      {
          super(message);
+   this.token = token;
      }
+
+  public Token getToken()
+  {
+   return token;
+  }
  }

  $