RE: SableCC hangs when 'compiling' a grammar

[Mariusz Nowostawski <mariusz@marni.otago.ac.nz>]

Hi Anton,

> I've been examining the sablecc a bit with jdb and i've added some debug
> print statements to the DFA optimize() method.
>
> It looks like the sablecc is 'hanging' there. Actually it is not hanging at
> all, i think. No output is produced in 'DFA.optimize()' and this method
> takes a very very long time to complete. In 'jdb' is suspended and resumed
> the main thread, set some breakpoints in this method and there seem not be
> any dead-locks (any more).
>
> I printed out the number of states that outer loop (for (i)) in this method
> goes through for my example grammar:
>
>  - Optimizing 133629 states.
>
> I don't know that the average number of state-transitions processed in its
> inner loop (for (j)), but it does not surprise me anymore that this method
> takes a long time to complete.

Great! So we have narrowed down the issue. Good work.
So now I believe it is simply a matter of OutOfMemory exception.

There are two cases now as far as I can see it:

1. It is bad that the grammar has 133629 states to be optimized, and there
   is something wrong with the way the grammar is structured. In other
   words optimizing such a huge space of states should not happen.

2. It is perfectly normal to have such a huge number of states to be
   optimized, and it is SableCC fault not to cope with it. Some memory
   usage optimization is needed.


I will look into 2., and I will try to reduce the memory sablecc uses for
otimizing the states. Any reduction there is good, whatever the result in
this particular case is, so I am happy to do it.  Give me couple hours to
play with the memory usage.

If you could look a little more into 1. and maybe reduce the number of
states, that would also help progressing. Please do check how many states
Java grammar or criptonite grammar produce, just to have a point of
reference.


best regards

Mariusz