[Soot-list] Soot/Spark GC overhead limit exceeded and the Android Framework

[Steven Arzt]

Hi Jason,

 

Your main() method is  static method that correctly creates a new instance of your “OneManager” class and then exercises it.  Your “custom entry point” (which is misleading nomenclature as the mere fact that you explicitly specify the entry point instead of having Soot pick the main method for you does not  change anything with respect to Soot) still has the old problem: You are calling into an instance method without having an instance. You are trying to analyze “closedAPI()”, but there is no base instance of “OneManager”, so there will not be any outgoing edges from any local field. There is simply no way for SPARK to find out where the call to “(this.)mService.closedAPI()” should go as “this” was never initialized, the constructor was never run, and “mService” is just null.

 

If you want to analyze instance methods, you need to generate a dummy “main” method which you can then use as your entry point. This dummy main method must create proper instances and then call the methods you want to analyze. You can for instance use the DefaultEntryPointCreator (which is included in FlowDroid) for that job. It takes a list of method signatures and then creates a dummy main method from them.

 

Best regards,

  Steven

 

Von: soot-list-bounces at CS.McGill.CA [mailto:soot-list-bounces at CS.McGill.CA] Im Auftrag von Jason Ott
Gesendet: Dienstag, 7. April 2015 01:06
An: soot-list at CS.McGill.CA
Betreff: Re: [Soot-list] Soot/Spark GC overhead limit exceeded and the Android Framework

 

Thank you!

 

I have a few other questions regarding entry points and call graph creation:

1.	If soot/SPARK cannot figure out the path does it stop and not generate any edges?  I have a toy example (see below) for which the new'd object creates the appropriate graph, but the custom entry point creates a graph with no edges.
2.	Which leads me to my next question: what specifically causes confusion for SPARK in the custom entry point?

1.	Is it the use of this.methodX() versus methodX()?  (Does the simple use of "this" keyword kill the graph?

3.	Does subtype polymorphism have any affect on the call graph whether new'd or custom entry?

I have tried answering these problems myself, but it is difficult when the custom entry point of the graph doesn't generate any edges for me to analyze.  Thanks!

 

I am generating the callgraph with these two commands:

new'd object:

*	-p cg.spark on -cp /path/to/classes/:. -trim-cfgs -allow-phantom-refs -w -verbose run main

Custom Entry point:

*	-p cg.spark on -cp /path/to/classes/:. -trim-cfgs -allow-phantom-refs -w -verbose Managers.OneManager closedAPI

 

Toy example:

OneManager.java

public class OneManager {
    public OneService mService;

    public OneManager() {
        mService = new OneService();

    }
    public void openAPI() {
        mService.openAPI();

    }
    public void closedAPI() {
        mService.closedAPI();

    }
}

OneService.java

public class OneService extends MasterService {
    public boolean mHasRights = false;

    public void overridenMethod() {
        //don't call super here
    }
    public void notOverriddenMethod() {
        super.notOverriddenMethod();
    }
    public void closedAPI() {
        callChainOne();
    }
    public void openAPI() {
        super.callChainOne();
    }
    public void callChainOne(){
        callChainTwo();
    }
    public void callChainTwo() {
        callChainThree();
    }
    public void callChainThree() {
    }
}
MasterService.java
public class MasterService {
    public void overriddenMethod() {
    }
    public void notOverriddenMethod() {
        
    }
    public void callChainOne() {
        this.callChainTwo();
    }
    public void callChainTwo() {
        this.callChainThree();
    }
    public void callChainThree() {
        return;
    }
}

    Run.java

public class run {
    public static void main(String[] args) {
        OneManager om = new OneManager();
        om.closedAPI();
    }
}

 

My SootAnalyzer is super simple:

 public class Main {
/**
* @param args
*/
public static void main(String[] args) {
String entryPointClass = args[args.length-2];
String entryPointMethod = args[args.length-1];

System.out.println(entryPointClass + "." + entryPointMethod);

args = Arrays.copyOf(args, args.length-2);

for(String s : args) {
//System.out.println(s);
}
long start = System.currentTimeMillis();
Options.v().parse(args);
//SootClass c = Scene.v().forceResolve("android.hardware.usb.UsbManager", SootClass.BODIES);
SootClass c = Scene.v().forceResolve(entryPointClass, SootClass.BODIES);
c.setApplicationClass();
Scene.v().loadNecessaryClasses();
SootMethod method = c.getMethodByName(entryPointMethod);

List entryPoints = new ArrayList();
entryPoints.add(method);

Scene.v().setEntryPoints(entryPoints);
PackManager.v().runPacks();

//soot.Main.main(args);
soot.jimple.toolkits.callgraph.CallGraph callgraph = Scene.v().getCallGraph();


FileHandler fh = new FileHandler(entryPointClass + "-" + entryPointMethod);

for(Edge e : callgraph) {
fh.write(e.toString());
fh.write("\n");
}
fh.close();
System.out.println("[TestSpark] Call graph size " + callgraph.size());

System.out.println(String.format("It took: %d seconds to generate a callgraph for: %s",
TimeUnit.MILLISECONDS.toSeconds(System.currentTimeMillis() - start),
entryPointClass +"."+ entryPointMethod));
}
}

 

FileHandler is simple a wrapper class for handling I/O, it works just fine. 

 

On Thu, Apr 2, 2015 at 11:15 AM, Jason Ott <jott002 at ucr.edu> wrote:

So I'm trying to run soot with spark on the Android Framework using a custom entry point.

 

Here is my the code that runs soot:

   

    public class Main {

        public static void main(String[] args) {

 

            String entryPointClass = args[args.length-2];

            String entryPointMethod = args[args.length-1];

 

            System.out.println(entryPointClass + "." + entryPointMethod);

 

            args = Arrays.copyOf(args, args.length-2);

 

            long start = System.currentTimeMillis();

            Options.v().parse(args);

            SootClass c = Scene.v().forceResolve(entryPointClass, SootClass.BODIES);

            c.setApplicationClass();

            Scene.v().loadNecessaryClasses();

            SootMethod method = c.getMethodByName(entryPointMethod);

            List entryPoints = new ArrayList();

            entryPoints.add(method);

            Scene.v().setEntryPoints(entryPoints);

            PackManager.v().runPacks();

 

            soot.jimple.toolkits.callgraph.CallGraph callgraph = Scene.v().getCallGraph();

            System.out.println("[TestSpark] Call graph size " + callgraph.size());

 

            System.out.println(String.format("It took: %d seconds to generate a callgraph for: %s",

                    TimeUnit.MILLISECONDS.toSeconds(System.currentTimeMillis() - start),

                    entryPointClass +"."+ entryPointMethod));

            }

}

 

I would run this as follows:

java -Xmx:1024mb -jar mySpark.jar -p cg.spark on -process-dir /Users/jason/Desktop/android/pscout/android_source_dir/ -cp /Users/jason/Desktop/android/pscout/android_source_dir:. -allow-phantom-refs android.content.ClipboardManager hasText

 

The last two args end up being the api class and the api method that I want to start the graph at.  

 

When I run this jar I get a "java.lang.OutOfMemoryError: GC overhead limit exceeded" error.  I've given the JVM as mucha s 8gb and it still exhausts memory.

 

Now I must admit that to even get this to work I had to tweak the Soot source code a bit.  I had to alter the following:

*	Line 99 in soot/src//soot/asm/SootClassBuilder.java will not throw an exception with my code.  The line now prints an error and moves on.  As I was reading the code and reading other examples, this is tolerable because Soot still finds the methods in those classes (presumably, as it doesn't throw any errors about not finding methods)

Can soot handle the entire android framework?

Is my method and approach proper?

Are there any indications as to why I'm running into a GC Memory error and how to fix it?

Is my change to soot source code the root of this problem?

 

Thank you for any assistance you may provide.

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailman.CS.McGill.CA/pipermail/soot-list/attachments/20150407/9bdca878/attachment-0001.html