[Soot-list] Call Graph issues and advice needed

Fri Apr 26 10:43:41 EDT 2013

Thank you all,

I echo Ashish, is there a fundamental problem with specifying entry points that are not static?
(I am specifying activity.onCreate which is not static, and for simple apps, cg produces a good graph that ifds can use. For larger apps, cg fails)

Can anyone comment on the "all-reachable" option for cg phase?
What type of graph would you use and why? 

@Rohan, I was suspecting that creating a "stub" main would cause issues for IFDS resolution since the graph is not accurate. I am glad you confirmed it. Thank you.

Regards,

- Henddher

On Apr 26, 2013, at 3:50 AM, ASHISH MISHRA <ashish123.mishragkp at gmail.com> wrote:

> Can any one please elaborate the static Entry points problem, I am sorry if the question is too trivial, but I can set even a non Static method as a possible entry point (if I understand it correctly).
> 
> @Henddher , Do you some links for using src-prec apk, currently I am setting entry points manually and then calling PackManager.runpacks(). Although with this method I am able to generate the Call graph and run the analysis , but I have a tough time debugging, when some exception occurs. I wish to use the main of soot.
> 
> ragards
> Ashish
> 
> 
> On Thu, Apr 25, 2013 at 8:12 PM, Marc-André Laverdière-Papineau <marc-andre.laverdiere-papineau at polymtl.ca> wrote:
> Hello,
> 
> I tried using that API for entry points in the past and I would just set
> the entry points and then run Soot Main.
> 
> That being said, I hit the static method problem. The way I worked
> around it (with Bernhard's help) was to generate a stub that had a main
> and make it instantiate the classes and call the entry points. I realize
> that it is a lot of work.
> 
> If you are very brave, you could try to do something inside Spark to
> handle non-static entry points. I think few people know that code well,
> so the workaround is the preferred solution.
> 
> I haven't used this all-reacheable option, so I am not familiar with it.
> If you can confirm there is a bug, it'd be great if you are able to
> write a patch.
> 
> Marc-André Laverdière-Papineau
> Doctorant - PhD Candidate
> 
> On 04/24/2013 05:32 PM, Henddher Pedroza wrote:
> > Thank you Marc-Andre.
> >
> > Does each entry point need to be a static method?
> > Couldn't a non-static arbitrary method of any class be marked as an
> > entry point?
> >
> > Does the driver cited below have anything wrong?
> > (I am using my own because I need to manipulate entry points. The
> > standard entry points in android do not seem to be handled automatically
> > when I use -src-prec apk. If the entry points for android were known, or
> > could be specified in another way, I would be able to use soot.Main I guess)
> >
> > What about the option "all-reachable:true"? What is it expected to do?
> >
> > On Apr 24, 2013, at 3:46 PM, Marc-André Laverdière-Papineau
> > <marc-andre.laverdiere-papineau at polymtl.ca
> > <mailto:marc-andre.laverdiere-papineau at polymtl.ca>> wrote:
> >
> >> Hello,
> >>
> >> 1) For Paddle, did you include the separate Paddle jar?
> >
> > I didn't know about this. I will search for it.
> >
> >> 2) For all-reachable, I would encourage you to dump the CG using Probe
> >> and double-check that its really due to the call graph.
> >> http://plg.uwaterloo.ca/~olhotak/probe/
> >> Also, I would encourage you to use the soot main instead of rolling your
> >> own driver. I would say that this is the source of so many problems we
> >> see reported on the ML that its always the first thing that gets
> >> suggested.
> >>
> >
> > I will look into this.
> >
> >> By the way, note that you need entry points no matter what, because your
> >> CG needs a root. There is a catch: the entry points need to be static,
> >> otherwise Spark will produce a CG with missing edges.
> >>
> >
> > Is static a mandatory requirement for entry points?
> >
> >> 3) Those call on CG objects... are they done inside a transformer?
> >> Outside of one, you are guaranteed to get an empty CG.
> >
> > They are inside internalTransform after the instantiation of CG:
> >
> > this.cg = new JimpleBasedInterproceduralCFG();
> > ...
> > cg.getCallersOf(sm); // sm is my method of interest
> >
> >
> >>
> >> 4) I don't know about the Android side of things, sorry.
> >>
> >> 5) I saw this NPE in getBodyFromMethodSource before. IIRC, it happens
> >> when trying to load a body from an excluded class.
> >>
> >
> > I am not explicitly excluding classes.
> >
> >> I hope this helps. Regards,
> >>
> >> Marc-André Laverdière-Papineau
> >> Doctorant - PhD Candidate
> >>
> >> On 04/24/2013 11:00 AM, Henddher Pedroza wrote:
> >>> Hello all:
> >>>
> >>> I am trying to obtain the most complete CFG there can be for
> >>> Interprocedural analysis to use with Heros. I am analyzing Android apps.
> >>>
> >>> *) cg.paddle does not seem to be complete. Apparently PaddleTransformer
> >>> is missing. Exception in thread "main" java.lang.RuntimeException: Could
> >>> not find soot.jimple.paddle.PaddleTransformer. Did you include Paddle on
> >>> your Java classpath?
> >>> at soot.jimple.paddle.PaddleHook.instantiate(PaddleHook.java:51)
> >>> *) Neither cg.spark and cg.cha seem to obey the "all-reachable:true"
> >>> option.
> >>>
> >>> I am using JimpleBasedInterproceduralCFG and it seems like the cg is not
> >>> complete. I create this instance within the body of internalTransform of
> >>> my SceneTransformer phase added to the "wjtp" pack.
> >>>
> >>> I have my own driver:
> >>> // command line: java -cp lib/soot.jar:bin/ -Xmx4000m -ea MyDriver -f J
> >>> -d ./sootOutput/ -cp $APPCP -process-dir $APK -no-bodies-for-excluded
> >>> -allow-phantom-refs -w -pp -src-prec apk -android-jars
> >>> ../Documents/android-sdk-macosx/platforms/ -ire -p cg
> >>> all-reachable:true -p cg.spark enabled:true
> >>>
> >>> // MyDriver.java
> >>> public static void main(String[] args) {
> >>>   Options.v().parse(Arrays.copyOf(args, args.length));
> >>>
> >>>   Scene.v().loadNecessaryClasses();
> >>>
> >>>      addMyEntryPointsTo(Scene.v().getEntryPoints()); // here I add my
> >>> entry points because "all-reachable:true" seems to do nothing
> >>> PackManager.v().getPack("wjtp").add(newTransform("wjtp.ifds",
> >>> newMyAnalysis()));
> >>>   PackManager.v().runPacks();
> >>> }
> >>>
> >>> Any advice of how I should instruct Soot to create the CG for
> >>> interprocedural?
> >>>
> >>> I am experiencing multiple problems:
> >>>
> >>> 1. When I query all the callers (callsites) of a given callee method, I
> >>> get an empty set (i.e. cg.getCallersOf(callee).isEmpty() is true). But,
> >>> if I specify one of the known callers (i.e. knownCallerSootMethod) as an
> >>> entry point (Scene.v().getEntryPoints().add(knownCallerSootMethod)),
> >>> then cg.getCallersOf(callee) returns the callsite unit where my callee
> >>> is invoked. The fact that I have to add knownCallerSootMethod to entry
> >>> points tells me that the knownCallerSootMethod is not an entry point
> >>> automatically found by Soot even though is an entry point for Android
> >>> app (aka Acvitity.onCreate(Bundle)). Doesn't Soot know the standard
> >>> entry points for android apps when the -src-prec option is "apk"?
> >>> And, why does "all-reachable:true" option seem to do nothing in my case?
> >>>
> >>> 2. The cg constructed seems to be incomplete even when I choose .class
> >>> file analysis instead of .apk. IOW, cg.getCallersOf(callee) returns
> >>> empty even when I use -src-prec class AND add the android-platform path
> >>> (not stubs) to soot classpath.
> >>>
> >>> 3. For some apps (.apk), the construction of cg fails with NPE depending
> >>> on which entry points I explicitly add. Note that my command line
> >>> specifies "cg.spark" (a) but similar results (NPE) are produced when I
> >>> use cg.cha (b).
> >>>
> >>> a)
> >>> Exception in thread "main" java.lang.NullPointerException
> >>> at soot.SootMethod.getBodyFromMethodSource(SootMethod.java:89)
> >>> at soot.SootMethod.retrieveActiveBody(SootMethod.java:322)
> >>> at
> >>> soot.jimple.toolkits.callgraph.OnFlyCallGraphBuilder.processNewMethod(OnFlyCallGraphBuilder.java:532)
> >>> at
> >>> soot.jimple.toolkits.callgraph.OnFlyCallGraphBuilder.processReachables(OnFlyCallGraphBuilder.java:427)
> >>> at soot.jimple.spark.solver.OnFlyCallGraph.build(OnFlyCallGraph.java:55)
> >>> at
> >>> soot.jimple.spark.builder.ContextInsensitiveBuilder.build(ContextInsensitiveBuilder.java:77)
> >>> at
> >>> soot.jimple.spark.SparkTransformer.internalTransform(SparkTransformer.java:84)
> >>> at soot.SceneTransformer.transform(SceneTransformer.java:39)
> >>> b)
> >>> Exception in thread "main" java.lang.NullPointerException
> >>> at soot.SootMethod.getBodyFromMethodSource(SootMethod.java:89)
> >>> at soot.SootMethod.retrieveActiveBody(SootMethod.java:322)
> >>> at
> >>> soot.jimple.toolkits.callgraph.OnFlyCallGraphBuilder.processNewMethod(OnFlyCallGraphBuilder.java:532)
> >>> at
> >>> soot.jimple.toolkits.callgraph.OnFlyCallGraphBuilder.processReachables(OnFlyCallGraphBuilder.java:427)
> >>> at
> >>> soot.jimple.toolkits.callgraph.CallGraphBuilder.build(CallGraphBuilder.java:84)
> >>> at
> >>> soot.jimple.toolkits.callgraph.CHATransformer.internalTransform(CHATransformer.java:43)
> >>> at soot.SceneTransformer.transform(SceneTransformer.java:39)
> >>> at soot.Transform.apply(Transform.java:89)
> >>> at soot.RadioScenePack.internalApply(RadioScenePack.java:57)
> >>> at
> >>> soot.jimple.toolkits.callgraph.CallGraphPack.internalApply(CallGraphPack.java:49)
> >>> at soot.Pack.apply(Pack.java:114)
> >>>
> >>> Any suggestion is welcome.
> >>>
> >>> - Henddher
> >>>
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> Soot-list mailing list
> >>> Soot-list at sable.mcgill.ca <mailto:Soot-list at sable.mcgill.ca>
> >>> http://mailman.cs.mcgill.ca/mailman/listinfo/soot-list
> >>>
> >> _______________________________________________
> >> Soot-list mailing list
> >> Soot-list at sable.mcgill.ca <mailto:Soot-list at sable.mcgill.ca>
> >> http://mailman.cs.mcgill.ca/mailman/listinfo/soot-list
> >
> _______________________________________________
> Soot-list mailing list
> Soot-list at sable.mcgill.ca
> http://mailman.cs.mcgill.ca/mailman/listinfo/soot-list
> 
> 
> 
> -- 
> Regards,
> Ashish Mishra
> Graduate Student,
> Computer Science and Automation Department,IISc
> Cell : +91-9611194714
> Mailto : ashishmishra at csa.iisc.ernet.in
> 
> _______________________________________________
> Soot-list mailing list
> Soot-list at sable.mcgill.ca
> http://mailman.cs.mcgill.ca/mailman/listinfo/soot-list

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.cs.mcgill.ca/pipermail/soot-list/attachments/20130426/dab15cab/attachment.html 