Tokens and free text

[hutch@RedRock.com (Bob Hutchison)]

Hi,

Another question...

Consider the following input:

"to: i want this to work"

The quotes are not part of the input.

I've got this small grammar:


Package simple;

/*******************************************************************
 * Helpers                                                         *
 *******************************************************************/
Helpers

   c_colon    = ':';
   c_cr       = 0x000d;
   c_lf       = 0x000a;
   char       = [0x00..0xff];

/*******************************************************************
 * Tokens                                                          *
 *******************************************************************/
Tokens

   colon              = c_colon;
   cr                 = c_cr;
   lf                 = c_lf;
   crlf               = c_cr c_lf;
   text_chars         = char;

   to_header          = 'to';
   from_header        = 'from';

/*******************************************************************
 * Productions                                                     *
 *******************************************************************/
Productions

   message         = lines*;
   lines           = header colon text* crlf;
   header          =   {to}    to_header
                     | {from}  from_header
                     ;

   text            =   {text}  text_chars
                     | {to}    to_header
                     | {from}  from_header
                     | {cr}    cr
                     | {lf}    lf
                     | {colon} colon
                     ;


The problem I have with this is the definition of the 'text' production.
It seems I have to list every token here to recognise the text of the
token.

Now I really hope I'm missing something... or is this the way it is?

thanks,
Bob


---
Bob Hutchison, hutch@RedRock.com, (416) 878-3454
RedRock, Toronto, Canada