[abc] J-LO benchmark results

Hi.

I just evaluated the benchmark results for JHotDraw with J-LO
instrumentation. They seem pretty interesting:

http://www.sable.mcgill.ca/~ebodde/data/jlo-benchmark-results.htm

Total runtime was 4565207ms, which is about 76 minutes, but more
interesting is the memory and runtime-behaviour: First of all it seems
completely related, which makes sense because more memory consumption
means more that more objects are bound which then means that J-LO has to
iterate over larger formulae. However, I cannot explain what those peaks
actually come from. Have you guys seen similar peaks in your
implementation?

It could obviously be related to garbage collection. (Note that J-LO can
only discover that an interator cannot be "misused" any more when it is
really GC'ed - before that happens, J-LO still has to take it into
account and iterator over it. I think that also counts for
tracematches.) However, when I interpreted the JHotDraw code corretly,
we should actually do a GC after each iterations, so this seems really
odd. Having said that, I know that at least IBM JVMs partially override
calls to System.gc() with a noop because people were sometimes misusing
it which was counterproductive. I don't knowif Sun JVMs behave in a
similar way.

Another interesting is apparently that dispite the fact that J-LO seems
actually memory-safe (it never exceeds a certain limit of memory
consumption), it is definetely generating a HUGE footprint. I can only
conclude that this is due to the fact that the benchmark uses so many
shapes, because when I did tests first with 2-3 shapes, it never went
above 2MB. But since for any binding the program generates, we basically
have to generate a full copy of the subformula stating "for this binding
now please do not touch the collection", this takes up rather much
memory.

So, I think that was it from me for today.

Eric

--
Eric Bodden
Sable Research Group, McGill University
Montreal, Canada