Re: [abc] Compile time benchmarks

What exactly is the elapsed time and is it important?

Laurie

+-------------------------------------------------------------+
| Laurie Hendren, Professor, School of Computer Science |
| McGill University |
| 318 McConnell Engineering Building tel: (514) 398-7391 |
| 3480 University Street fax: (514) 398-3883 |
| Montreal, Quebec H3A 2A7 hendren@cs.mcgill.ca |
| CANADA http://www.sable.mcgill.ca/~hendren |
+-------------------------------------------------------------+

On Thu, 16 Dec 2004, Sascha Kuzins wrote:

>
> Hi all,
>
> I assembled a script to measure compile times of ajc and abc. The script
> is in the cvs folder 'benches' and is called buildall.pl.
> It compiles the specified benchmark a number of times and computes the
> average (removing smallest and greates values first, based on total time
> (user+sys), not elapsed time).
> Before each compilation, it cleans the output folders. I've re-used the
> existing scripts to compile and clean the benchmarks, so I assumed these
> work correctly.
>
> This is the command line:
> ./buildall.pl `cat paper_benchmarks` > buildtimes
>
> The results of a first test run are shown below.
>
> ajc: bean: Total(sys+user) 3.69 Elapsed 3.68 (chopped times (total):
> 3.68, 3.69)
> abc: bean: Total(sys+user) 4.26 Elapsed 7.93 (chopped times (total):
> 3.64, 4.28)
>
> ajc: figure: Total(sys+user) 3.37 Elapsed 11.3 (chopped times (total):
> 3.36, 3.37)
> abc: figure: Total(sys+user) 3.835 Elapsed 15.125 (chopped times
> (total): 3.74, 3.85)
>
> ajc: dcm_aspects: Total(sys+user) 5.68 Elapsed 20.795 (chopped times
> (total): 5.68, 5.69)
> abc: dcm_aspects: Total(sys+user) 8.82 Elapsed 29.59 (chopped times
> (total): 8.73, 8.86)
>
> ajc: simulator_null_checks: Total(sys+user) 4.925 Elapsed 34.515
> (chopped times (total): 4.81, 4.93)
> abc: simulator_null_checks: Total(sys+user) 8.18 Elapsed 42.685 (chopped
> times (total): 8.07, 8.3)
>
> ajc: adaptj-pooling-FFT: Total(sys+user) 8.915 Elapsed 51.55 (chopped
> times (total): 7.67, 8.98)
> abc: adaptj-pooling-FFT: Total(sys+user) 50.07 Elapsed 102.865 (chopped
> times (total): 49.86, 51.31)
>
>
> If everybody is happy with the format and method, I will run it again
> for all benchmarks and with a higher number of rounds.
>
> Sascha
>
>
>
>
>
>
>
>
>
>
Received on Thu Dec 16 17:45:29 2004