Re: [abc] J-LO benchmark results

Great! It's good to see that J-LO does not suffer a memory leak here.
I think there have to be a couple of sentences in the paper that explain
this: because J-Lo does not have a facility for using bindings in
advice, you can make all variable bindings a weak ref. The same does
not hold true for tracematches. Also, for tracematches there is a
potential problem of disjuncts leaking. Why does that not happen in
JLO, do you know?

In any event, I think this is pretty strong evidence for the conclusion
I prematurely wrote into the paper, namely that the #1 difference between
the tm implementation and J-LO is the level of specialisation.
Do you agree with this?

Hm, calling gc does make a difference for the tm picture, making it
a nice smooth line taking away any strange peaks. So I don't quite
understand what is going on with your peaks.

-Oege

On Fri, 3 Mar 2006, Eric Bodden wrote:

> Hi.
>
> I just evaluated the benchmark results for JHotDraw with J-LO
> instrumentation. They seem pretty interesting:
>
> http://www.sable.mcgill.ca/~ebodde/data/jlo-benchmark-results.htm
>
> Total runtime was 4565207ms, which is about 76 minutes, but more
> interesting is the memory and runtime-behaviour: First of all it seems
> completely related, which makes sense because more memory consumption
> means more that more objects are bound which then means that J-LO has to
> iterate over larger formulae. However, I cannot explain what those peaks
> actually come from. Have you guys seen similar peaks in your
> implementation?
>
> It could obviously be related to garbage collection. (Note that J-LO can
> only discover that an interator cannot be "misused" any more when it is
> really GC'ed - before that happens, J-LO still has to take it into
> account and iterator over it. I think that also counts for
> tracematches.) However, when I interpreted the JHotDraw code corretly,
> we should actually do a GC after each iterations, so this seems really
> odd. Having said that, I know that at least IBM JVMs partially override
> calls to System.gc() with a noop because people were sometimes misusing
> it which was counterproductive. I don't knowif Sun JVMs behave in a
> similar way.
>
> Another interesting is apparently that dispite the fact that J-LO seems
> actually memory-safe (it never exceeds a certain limit of memory
> consumption), it is definetely generating a HUGE footprint. I can only
> conclude that this is due to the fact that the benchmark uses so many
> shapes, because when I did tests first with 2-3 shapes, it never went
> above 2MB. But since for any binding the program generates, we basically
> have to generate a full copy of the subformula stating "for this binding
> now please do not touch the collection", this takes up rather much
> memory.
>
> So, I think that was it from me for today.
>
> Eric
>
> --
> Eric Bodden
> Sable Research Group, McGill University
> Montreal, Canada
>
>
>
Received on Sat Mar 04 12:37:48 2006