[Soot-list] different experimental results using SPARK

[Ondrej Lhotak]

On Tue, Sep 26, 2006 at 06:51:23PM -0400, Aiwu Shi wrote:
>    hi,
>        I am doing points-to analysis using Spark,  but I got different
>    results, compared to Ondrej Lhotak in his master thesis titled " Spark: a
>    flexible points-to analysis framework for Java".
>     
>    my code snippet below  for seting the spark options
>      sparkOptions.put("enabled", "true");
>      sparkOptions.put("on-fly-cg", "false");
>      sparkOptions.put("propagator", "worklist");
>      sparkOptions.put("set-impl", "hybrid");
>       SparkTransformer.v().transform("cg.spark",sparkOptions);
>     
>    setting above is "ot-cha-fs" , but when I set "on-fly-cg" as "true" and
>    get "ot-otf-fs"
>     
>    my experimental results show "ot-otf-fs" is faster than "ot-cha-fs", which
>    is different from the master thesis of ondrej.
>     
>    I dont know why? I set wrong options?

The short answer is that Spark has evolved since the time I did the
experiments for my thesis.

The two most significant issues that may explain this difference are the
following.

First, like many "on-the-fly call graph" analyses, Spark
used to be only *partly* on-the-fly, in the sense that it built
constraints to model intra-procedural pointer assignments for *all*
procedures, and then added constraints for inter-procedural pointer
flow as the call graph was constructed. Now, Spark is *fully* on-the-fly,
in that it only builds intra-procedural constraints for procedures that
are found to be reachable in the call graph.

Second, the timing reported in my thesis were only for that points-to
analysis, and did not include other things that Soot does, such as
reading class files and converting them to its Jimple intermediate
representation. If you are measuring the total time that Soot runs,
it does not surprise me that the on-the-fly call graph would be faster,
because, as I said above, it builds intra-procedural constraints 
for fewer methods, meaning that fewer methods need to be read and
converted to Jimple, which can save quite a bit of time.

If you are seeing the points-to analysis time itself faster with otf
than cha, I would find that surprising (although theoretically, it's
not impossible, depending on the benchmark). Note that for an accurate
measurement of the points-to analysis time, the pre-jimplify option is
needed to separate jimplification from points-to analysis.

Ondrej

>     
>        thanks
>       
>        Aiwu

> _______________________________________________
> Soot-list mailing list
> Soot-list at sable.mcgill.ca
> http://mailman.cs.mcgill.ca/mailman/listinfo/soot-list