Re: [abc] oopsla outline

Ondrej Lhotak wrote:

>Are there some common benchmarking scripts I should be using,
>or is everyone rolling their own?
>
>
In principle, I have a set of scripts to run benchmarks automatically.
They rely on a file specialised to each benchmark, however, so if you
just provide *some* scripts to compile and run the benchmark (preferably
separately), then I could easily use those by calling them from the
specialised script.

If you want the exact details, the assumption is that there is a script
called run-benchmark.sh in the root directory of the example. This
script should

- accept three flags: --clean to get rid of previously generated output
(*not* compiled code, just the results), --compile to remove previous
compilation results and recompile, and <anything-else>. This would then
be used as a 'tag'; it becomes part of the output filename, to allow
several iterations to be run.
- produce timing/memory measurements in some output directory, usually
<benchmark-root>/output/Benchmar-description.$tag.out, where $tag is the
<anything-else> value above.

If you want a template for such a run-benchmark script that already does
the command-line argument handling, I can give you one, but, as I said,
just providing 'compile' and 'run' scripts (where 'run' produces
measurements in some predefined format) would make it quite easy for me
to just reuse these.

- P
Received on Sat Mar 04 13:09:39 2006