[Soot-list] Call Graph issues and advice needed
Marc-André Laverdière-Papineau
marc-andre.laverdiere-papineau at polymtl.ca
Fri Apr 26 21:09:42 EDT 2013
Hello,
Lots of stuff happened in this thread - I'm late for the party :)
When it comes to an IFDS analysis, the solution when the order is
unknown is that you should put it in a loop with a random.
switch (random.nextInt()) {
case 0: ... break;
case 1: ... break;
...
default: return
}
When it comes to the entry points, what Bernhard and I are doing is to
add a step in the wjpp phase that creates the main and related classes
and then sets the new main in the Scene. So we don't need to use the
special API for the entry points and so we can use Soot main.
As for the link you shared, thanks for reminding me! I had totally
forgotten about it. I am sorry I confused everyone.
My previous experiments with a flexible entry point framework gave me
all sorts of trouble, including a major slowdown in Spark.
In case you are interested to continue where I left off, you can have a
look at the flexible entry point detection framework:
https://github.com/malaverdiere/soot/tree/flexible-entrypoints
As for the non-static entry point stuff, I'm going to ask Bernhard to
elaborate on that, because he dealt with it a whole lot more than I did.
IIRC it was because of the this pointer.
My guess is that you could patch Spark to deal with it by using CHA to
resolve the this pointer whenever you have a non-static entry point.
After all, you don't have any extra information at that point to know
what you're pointing to...
Marc-André Laverdière-Papineau
Doctorant - PhD Candidate
On 04/26/2013 10:50 AM, Henddher Pedroza wrote:
> Regarding
>
>>> @Henddher , Do you some links for using src-prec apk, currently I am
>>> setting entry points manually and then calling
>>> PackManager.runpacks(). Although with this method I am able to
>>> generate the Call graph and run the analysis , but I have a tough
>>> time debugging, when some exception occurs. I wish to use the main of
>>> soot.
>
> I am not using soot.Main either. I am using the driver I cited below. If
> I am not mistaken, if one wants to specify entry points, one needs to
> use PackManager.runPacks()
> (http://www.bodden.de/2012/07/26/soot-custom-entry-points/)
>
> On top of that, I am trying 2 different ways both unsuccessful for large
> Android apps:
> 1. using -src-prec apk and -android-jars to android-platform
> android.jars (from git)
> 2. excluding -src-prec (which defaults to .class) and adding specific
> android.jar (also from git) to soot-classpath (-cp
> appclasses/:./android-platform/.../android.jar)
>
> On Apr 26, 2013, at 9:43 AM, Henddher Pedroza <hpedro2 at uic.edu
> <mailto:hpedro2 at uic.edu>> wrote:
>
>> Thank you all,
>>
>> I echo Ashish, is there a fundamental problem with specifying entry
>> points that are not static?
>> (I am specifying activity.onCreate which is not static, and for simple
>> apps, cg produces a good graph that ifds can use. For larger apps, cg
>> fails)
>>
>> Can anyone comment on the "all-reachable" option for cg phase?
>> What type of graph would you use and why?
>>
>> @Rohan, I was suspecting that creating a "stub" main would cause
>> issues for IFDS resolution since the graph is not accurate. I am glad
>> you confirmed it. Thank you.
>>
>> Regards,
>>
>> - Henddher
>>
>> On Apr 26, 2013, at 3:50 AM, ASHISH MISHRA
>> <ashish123.mishragkp at gmail.com <mailto:ashish123.mishragkp at gmail.com>>
>> wrote:
>>
>>> Can any one please elaborate the static Entry points problem, I am
>>> sorry if the question is too trivial, but I can set even a non Static
>>> method as a possible entry point (if I understand it correctly).
>>>
>>> @Henddher , Do you some links for using src-prec apk, currently I am
>>> setting entry points manually and then calling
>>> PackManager.runpacks(). Although with this method I am able to
>>> generate the Call graph and run the analysis , but I have a tough
>>> time debugging, when some exception occurs. I wish to use the main of
>>> soot.
>>>
>>> ragards
>>> Ashish
>>>
>>>
>>> On Thu, Apr 25, 2013 at 8:12 PM, Marc-André Laverdière-Papineau
>>> <marc-andre.laverdiere-papineau at polymtl.ca
>>> <mailto:marc-andre.laverdiere-papineau at polymtl.ca>> wrote:
>>>
>>> Hello,
>>>
>>> I tried using that API for entry points in the past and I would
>>> just set
>>> the entry points and then run Soot Main.
>>>
>>> That being said, I hit the static method problem. The way I worked
>>> around it (with Bernhard's help) was to generate a stub that had
>>> a main
>>> and make it instantiate the classes and call the entry points. I
>>> realize
>>> that it is a lot of work.
>>>
>>> If you are very brave, you could try to do something inside Spark to
>>> handle non-static entry points. I think few people know that code
>>> well,
>>> so the workaround is the preferred solution.
>>>
>>> I haven't used this all-reacheable option, so I am not familiar
>>> with it.
>>> If you can confirm there is a bug, it'd be great if you are able to
>>> write a patch.
>>>
>>> Marc-André Laverdière-Papineau
>>> Doctorant - PhD Candidate
>>>
>>> On 04/24/2013 05:32 PM, Henddher Pedroza wrote:
>>> > Thank you Marc-Andre.
>>> >
>>> > Does each entry point need to be a static method?
>>> > Couldn't a non-static arbitrary method of any class be marked as an
>>> > entry point?
>>> >
>>> > Does the driver cited below have anything wrong?
>>> > (I am using my own because I need to manipulate entry points. The
>>> > standard entry points in android do not seem to be handled
>>> automatically
>>> > when I use -src-prec apk. If the entry points for android were
>>> known, or
>>> > could be specified in another way, I would be able to use
>>> soot.Main I guess)
>>> >
>>> > What about the option "all-reachable:true"? What is it expected
>>> to do?
>>> >
>>> > On Apr 24, 2013, at 3:46 PM, Marc-André Laverdière-Papineau
>>> > <marc-andre.laverdiere-papineau at polymtl.ca
>>> <mailto:marc-andre.laverdiere-papineau at polymtl.ca>
>>> > <mailto:marc-andre.laverdiere-papineau at polymtl.ca
>>> <mailto:marc-andre.laverdiere-papineau at polymtl.ca>>> wrote:
>>> >
>>> >> Hello,
>>> >>
>>> >> 1) For Paddle, did you include the separate Paddle jar?
>>> >
>>> > I didn't know about this. I will search for it.
>>> >
>>> >> 2) For all-reachable, I would encourage you to dump the CG
>>> using Probe
>>> >> and double-check that its really due to the call graph.
>>> >> http://plg.uwaterloo.ca/~olhotak/probe/
>>> >> Also, I would encourage you to use the soot main instead of
>>> rolling your
>>> >> own driver. I would say that this is the source of so many
>>> problems we
>>> >> see reported on the ML that its always the first thing that gets
>>> >> suggested.
>>> >>
>>> >
>>> > I will look into this.
>>> >
>>> >> By the way, note that you need entry points no matter what,
>>> because your
>>> >> CG needs a root. There is a catch: the entry points need to be
>>> static,
>>> >> otherwise Spark will produce a CG with missing edges.
>>> >>
>>> >
>>> > Is static a mandatory requirement for entry points?
>>> >
>>> >> 3) Those call on CG objects... are they done inside a transformer?
>>> >> Outside of one, you are guaranteed to get an empty CG.
>>> >
>>> > They are inside internalTransform after the instantiation of CG:
>>> >
>>> > this.cg <http://this.cg/> = new JimpleBasedInterproceduralCFG();
>>> > ...
>>> > cg.getCallersOf(sm); // sm is my method of interest
>>> >
>>> >
>>> >>
>>> >> 4) I don't know about the Android side of things, sorry.
>>> >>
>>> >> 5) I saw this NPE in getBodyFromMethodSource before. IIRC, it
>>> happens
>>> >> when trying to load a body from an excluded class.
>>> >>
>>> >
>>> > I am not explicitly excluding classes.
>>> >
>>> >> I hope this helps. Regards,
>>> >>
>>> >> Marc-André Laverdière-Papineau
>>> >> Doctorant - PhD Candidate
>>> >>
>>> >> On 04/24/2013 11:00 AM, Henddher Pedroza wrote:
>>> >>> Hello all:
>>> >>>
>>> >>> I am trying to obtain the most complete CFG there can be for
>>> >>> Interprocedural analysis to use with Heros. I am analyzing
>>> Android apps.
>>> >>>
>>> >>> *) cg.paddle does not seem to be complete. Apparently
>>> PaddleTransformer
>>> >>> is missing. Exception in thread "main"
>>> java.lang.RuntimeException: Could
>>> >>> not find soot.jimple.paddle.PaddleTransformer. Did you
>>> include Paddle on
>>> >>> your Java classpath?
>>> >>> at soot.jimple.paddle.PaddleHook.instantiate(PaddleHook.java:51)
>>> >>> *) Neither cg.spark and cg.cha seem to obey the
>>> "all-reachable:true"
>>> >>> option.
>>> >>>
>>> >>> I am using JimpleBasedInterproceduralCFG and it seems like
>>> the cg is not
>>> >>> complete. I create this instance within the body of
>>> internalTransform of
>>> >>> my SceneTransformer phase added to the "wjtp" pack.
>>> >>>
>>> >>> I have my own driver:
>>> >>> // command line: java -cp lib/soot.jar:bin/ -Xmx4000m -ea
>>> MyDriver -f J
>>> >>> -d ./sootOutput/ -cp $APPCP -process-dir $APK
>>> -no-bodies-for-excluded
>>> >>> -allow-phantom-refs -w -pp -src-prec apk -android-jars
>>> >>> ../Documents/android-sdk-macosx/platforms/ -ire -p cg
>>> >>> all-reachable:true -p cg.spark enabled:true
>>> >>>
>>> >>> // MyDriver.java
>>> >>> public static void main(String[] args) {
>>> >>> Options.v().parse(Arrays.copyOf(args, args.length));
>>> >>>
>>> >>> Scene.v().loadNecessaryClasses();
>>> >>>
>>> >>> addMyEntryPointsTo(Scene.v().getEntryPoints()); // here
>>> I add my
>>> >>> entry points because "all-reachable:true" seems to do nothing
>>> >>> PackManager.v().getPack("wjtp").add(newTransform("wjtp.ifds",
>>> >>> newMyAnalysis()));
>>> >>> PackManager.v().runPacks();
>>> >>> }
>>> >>>
>>> >>> Any advice of how I should instruct Soot to create the CG for
>>> >>> interprocedural?
>>> >>>
>>> >>> I am experiencing multiple problems:
>>> >>>
>>> >>> 1. When I query all the callers (callsites) of a given callee
>>> method, I
>>> >>> get an empty set (i.e. cg.getCallersOf(callee).isEmpty() is
>>> true). But,
>>> >>> if I specify one of the known callers (i.e.
>>> knownCallerSootMethod) as an
>>> >>> entry point
>>> (Scene.v().getEntryPoints().add(knownCallerSootMethod)),
>>> >>> then cg.getCallersOf(callee) returns the callsite unit where
>>> my callee
>>> >>> is invoked. The fact that I have to add knownCallerSootMethod
>>> to entry
>>> >>> points tells me that the knownCallerSootMethod is not an
>>> entry point
>>> >>> automatically found by Soot even though is an entry point for
>>> Android
>>> >>> app (aka Acvitity.onCreate(Bundle)). Doesn't Soot know the
>>> standard
>>> >>> entry points for android apps when the -src-prec option is "apk"?
>>> >>> And, why does "all-reachable:true" option seem to do nothing
>>> in my case?
>>> >>>
>>> >>> 2. The cg constructed seems to be incomplete even when I
>>> choose .class
>>> >>> file analysis instead of .apk. IOW, cg.getCallersOf(callee)
>>> returns
>>> >>> empty even when I use -src-prec class AND add the
>>> android-platform path
>>> >>> (not stubs) to soot classpath.
>>> >>>
>>> >>> 3. For some apps (.apk), the construction of cg fails with
>>> NPE depending
>>> >>> on which entry points I explicitly add. Note that my command line
>>> >>> specifies "cg.spark" (a) but similar results (NPE) are
>>> produced when I
>>> >>> use cg.cha (b).
>>> >>>
>>> >>> a)
>>> >>> Exception in thread "main" java.lang.NullPointerException
>>> >>> at soot.SootMethod.getBodyFromMethodSource(SootMethod.java:89)
>>> >>> at soot.SootMethod.retrieveActiveBody(SootMethod.java:322)
>>> >>> at
>>> >>>
>>> soot.jimple.toolkits.callgraph.OnFlyCallGraphBuilder.processNewMethod(OnFlyCallGraphBuilder.java:532)
>>> >>> at
>>> >>>
>>> soot.jimple.toolkits.callgraph.OnFlyCallGraphBuilder.processReachables(OnFlyCallGraphBuilder.java:427)
>>> >>> at
>>> soot.jimple.spark.solver.OnFlyCallGraph.build(OnFlyCallGraph.java:55)
>>> >>> at
>>> >>>
>>> soot.jimple.spark.builder.ContextInsensitiveBuilder.build(ContextInsensitiveBuilder.java:77)
>>> >>> at
>>> >>>
>>> soot.jimple.spark.SparkTransformer.internalTransform(SparkTransformer.java:84)
>>> >>> at soot.SceneTransformer.transform(SceneTransformer.java:39)
>>> >>> b)
>>> >>> Exception in thread "main" java.lang.NullPointerException
>>> >>> at soot.SootMethod.getBodyFromMethodSource(SootMethod.java:89)
>>> >>> at soot.SootMethod.retrieveActiveBody(SootMethod.java:322)
>>> >>> at
>>> >>>
>>> soot.jimple.toolkits.callgraph.OnFlyCallGraphBuilder.processNewMethod(OnFlyCallGraphBuilder.java:532)
>>> >>> at
>>> >>>
>>> soot.jimple.toolkits.callgraph.OnFlyCallGraphBuilder.processReachables(OnFlyCallGraphBuilder.java:427)
>>> >>> at
>>> >>>
>>> soot.jimple.toolkits.callgraph.CallGraphBuilder.build(CallGraphBuilder.java:84)
>>> >>> at
>>> >>>
>>> soot.jimple.toolkits.callgraph.CHATransformer.internalTransform(CHATransformer.java:43)
>>> >>> at soot.SceneTransformer.transform(SceneTransformer.java:39)
>>> >>> at soot.Transform.apply(Transform.java:89)
>>> >>> at soot.RadioScenePack.internalApply(RadioScenePack.java:57)
>>> >>> at
>>> >>>
>>> soot.jimple.toolkits.callgraph.CallGraphPack.internalApply(CallGraphPack.java:49)
>>> >>> at soot.Pack.apply(Pack.java:114)
>>> >>>
>>> >>> Any suggestion is welcome.
>>> >>>
>>> >>> - Henddher
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> _______________________________________________
>>> >>> Soot-list mailing list
>>> >>> Soot-list at sable.mcgill.ca <mailto:Soot-list at sable.mcgill.ca>
>>> <mailto:Soot-list at sable.mcgill.ca <mailto:Soot-list at sable.mcgill.ca>>
>>> >>> http://mailman.cs.mcgill.ca/mailman/listinfo/soot-list
>>> >>>
>>> >> _______________________________________________
>>> >> Soot-list mailing list
>>> >> Soot-list at sable.mcgill.ca <mailto:Soot-list at sable.mcgill.ca>
>>> <mailto:Soot-list at sable.mcgill.ca <mailto:Soot-list at sable.mcgill.ca>>
>>> >> http://mailman.cs.mcgill.ca/mailman/listinfo/soot-list
>>> >
>>> _______________________________________________
>>> Soot-list mailing list
>>> Soot-list at sable.mcgill.ca <mailto:Soot-list at sable.mcgill.ca>
>>> http://mailman.cs.mcgill.ca/mailman/listinfo/soot-list
>>>
>>>
>>>
>>>
>>> --
>>> Regards,
>>> Ashish Mishra
>>> Graduate Student,
>>> Computer Science and Automation Department,IISc
>>> Cell : +91-9611194714
>>> Mailto : ashishmishra at csa.iisc.ernet.in
>>> <mailto:ashishmishra at csa.iisc.ernet.in>
>>>
>>> _______________________________________________
>>> Soot-list mailing list
>>> Soot-list at sable.mcgill.ca <mailto:Soot-list at sable.mcgill.ca>
>>> http://mailman.cs.mcgill.ca/mailman/listinfo/soot-list
>>
>
>
>
> _______________________________________________
> Soot-list mailing list
> Soot-list at sable.mcgill.ca
> http://mailman.cs.mcgill.ca/mailman/listinfo/soot-list
>
More information about the Soot-list
mailing list