Re: [abc] oopsla paper

While I'm still rather at a loss for what to do, here the running times
for the benchmark on my laptop:

JHotDraw with minimised animation window:
- Pure java: 5s
- Naive aspects: 22s
- Seminaive aspects: 11.2s
- Non-naive aspects: 11.1s
- Tracematches: between 220 and 256 s.

Clearly it would be better to quote these numbers rather than the ones I
obtained on my desktop, since we "only" have a slowdown of about 50...
but the current numbers in the paper show a slowdown of less than 30.

What should we do? Laurie? Oege?

- P

Pavel Avgustinov wrote:

>Pavel Avgustinov wrote:
>
>
>
>>I plan to call it a day at some point this afternoon and replace the
>>numbers by newly generated ones on my own machine. I'll check the
>>section for any re-wording that needs to be done. There should be some
>>improvement, though nothing dramatic -- we may be 15 times slower
>>instead of 20, that order of magnitude.
>>
>>
>>
>>
>Slight problem with that...
>
>When trying to measure how long the pure-java version of the benchmark
>took, I was surprised to find it fluctuating, seemingly randomly, betwee
>4.4 seconds and 28.2 seconds -- always these two values, with a margin
>of error of 300ms or so. Investigation revealed that the running time
>depended on whether the X server or the JVM was prioritised by the OS --
>if the JVM got more CPU time, it finished in 4.4 seconds, if the X
>server got more it was 28.2.
>
>I tried the following experiment: I minimised the MDI child window
>containing the animation before selecting Animation->Start from the
>menu. The results are somewhat catastrophic... The pure java version
>finishes in 512-520 ms, reproducible. Naive aspects runs in 13.5 ms and
>accumulates a huge heap. Seminaive and non-naive aspects run in 7.2-7.4
>seconds, with the expected slight upwards trend in memory usage for
>seminaive and no such trend for non-naive aspects.
>
>Tracematches, on a minimised animation window, took me 194 seconds.
>
>We thus have a slowdown of almost a factor 400, when ignoring the X
>server overheads for the animation.
>
>I'm a bit at a loss for what to do. I can't seem to get reliable numbers
>for the pure java version with a non-minimised animation window, but the
>numbers with minimised windows seem to destroy our 'happy story'. We
>could leave the numbers as they are and investigate performance further
>until OOPSLA... or try to obtain some sort of numbers with non-minimised
>animation windows, these would be reasonably close to the ones in the
>paper..
>
>Here a summary of the timings I've taken:
>
>JHotDraw, minimising MDI animation window before starting animation:
>- pure Java: 0.5 s
>- naive aspects: 13.5 s
>- seminaive aspects: 7.3 s
>- non-naive aspects: 7.3 s
>- tracematches: 194 s
>
>JHotDraw, without minimising MDI animation window before starting
>animation, keeping it visible at all times:
>- pure Java: Fluctuates between 4.4 s and 28.2 s (discreetly, not
>continuously)
>- tracematches: 300-315 s.
>
>Any suggestions on a course of action are very welcome. If you can run
>the benchmark yourself, it might be helpful to compare results.
>
>- P
>
>
>
>
>
Received on Thu Aug 11 18:35:24 2005