[Soot-list] Spark generating rta call-graph: java.lang.OutOfMemoryError: GC overhead limit exceeded

[Dario Amoroso d'Aragona]

Hi all, 
i have this problem:
when i try to generating a spark call graph i have this error:
java.lang.OutOfMemoryError: GC overhead limit exceeded

at java.util.concurrent.ConcurrentHashMap$KeySetView.iterator(ConcurrentHashMap.java:4578)
at java.util.Collections$UnmodifiableCollection$1.<init>(Collections.java:1039)
at java.util.Collections$UnmodifiableCollection.iterator(Collections.java:1038)
at soot.FastHierarchy.canStoreClass(FastHierarchy.java:324)
at soot.FastHierarchy.canStoreType(FastHierarchy.java:233)
at soot.jimple.toolkits.callgraph.VirtualCalls.resolve(VirtualCalls.java:147)
at soot.jimple.toolkits.callgraph.VirtualCalls.resolveAnySubType(VirtualCalls.java:196)
at soot.jimple.toolkits.callgraph.VirtualCalls.resolve(VirtualCalls.java:180)
at soot.jimple.toolkits.callgraph.VirtualCalls.resolve(VirtualCalls.java:130)
at soot.jimple.toolkits.callgraph.OnFlyCallGraphBuilder.addType(OnFlyCallGraphBuilder.java:549)
at soot.jimple.toolkits.callgraph.CallGraphBuilder.build(CallGraphBuilder.java:118)
at soot.jimple.spark.SparkTransformer.internalTransform(SparkTransformer.java:154)
at soot.SceneTransformer.transform(SceneTransformer.java:39)
at com.company.Project.runPacks(Project.java:130)
at com.company.Project.<init>(Project.java:60)
at com.test.TestTemporary.loadP1(TestTemporary.java:10)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)

My code work as this:

   1. I set soot option like this:

private void setSootOptions() {
        List<String> argsList = new ArrayList<>();
        argsList.add("-verbose");
        argsList.add("-w"); // whole program mode
        argsList.add("-cp"); // Soot class-path
        argsList.add(path + ";C:\\soot\\soot-3.1.0-jar-with-dependencies.jar;C:\\Program Files\\Java\\jdk1.8.0_112\\jre\\lib\\rt.jar;C:\\Program Files\\Java\\jdk1.8.0_112\\jre\\lib\\jce.jar;C:\\Program Files\\JetBrains\\IntelliJ IDEA Community Edition 2018.1.5\\lib\\junit-4.12.jar;C:\\Program Files\\JetBrains\\IntelliJ IDEA Community Edition 2018.1.5\\lib\\hamcrest-core-1.3.jar;");
        Options.v().parse(argsList.toArray(new String[0]));

    }


   1. given a directory i add all .class files in directory as application 
   class in soot (after have loaded it with Scene.v().loadClassAndSupport())
   2. i find all junit tests methods in the given project and set all as 
   entry-points
   3. i generate a spark call graph:

 private void runPacks() {
        Transform sparkTranform = new Transform("cg.spark", null);

        PhaseOptions.v().setPhaseOption(sparkTranform, "enabled:true");
        PhaseOptions.v().setPhaseOption(sparkTranform, "rta:true");
        PhaseOptions.v().setPhaseOption(sparkTranform, "verbose:true");
        PhaseOptions.v().setPhaseOption(sparkTranform, "propagator:worklist");
        PhaseOptions.v().setPhaseOption(sparkTranform, "simple-edges-bidirectional:false");
        PhaseOptions.v().setPhaseOption(sparkTranform, "on-fly-cg:false");
        PhaseOptions.v().setPhaseOption(sparkTranform, "set-impl:double");
        PhaseOptions.v().setPhaseOption(sparkTranform, "double-set-old:hybrid");
        PhaseOptions.v().setPhaseOption(sparkTranform, "double-set-new:hybrid");
        PhaseOptions.v().setPhaseOption(sparkTranform, "force-cg:true");

        Map opt = PhaseOptions.v().getPhaseOptions(sparkTranform);
        System.out.println("rta call graph building...");

        SparkTransformer.v().transform(sparkTranform.getPhaseName(), opt);
        System.out.println("...rta call graph builded");

        CallGraph c = Scene.v().getCallGraph();
        setCallGraph(c);
        System.out.println("Serialize call graph start...");
        serializeCallGraph(callGraph, path + "//" + "-call-grsph" + DotGraph.DOT_EXTENSION);
        System.out.println("...Serialize call graph completed");
     }

I have test this my code on 2 project, with one project i don't have 
problems, with the second project i have the error reported before (GC 
overhead limit exceeded). I understand that the problem is when soot,in 
call graph generating, try to analyze this 2 methods:

@Test
    public void realPassTest(){
        ArrayList<String> a = st.realMethodToTest();
        Assert.assertTrue(a.contains("real"));
    }

    @Test
    public void realFailureTest(){
        ArrayList<String> a = st.realMethodToTest();
        Assert.assertTrue(a.contains("fail"));
    }

[...]

public ArrayList<String> realMethodToTest() {
        ArrayList a = new ArrayList<>();
        a.add("real");
        a.add("method");
        a.add("toTest");
        return a;
    }

With the debugger i see that soot in generating call graph load all java 
method for ArrayList. How ca i resolve this?
The program run for 1 hour and than give me that error. I run it with java 
vm option -Xmx10g. 
I try to delete the two methods "realFailureTest" and "realPasstest" and 
all it's ok. I try to generate the spark call-graph with "rta" option false 
and "on-fly-cg" option true and all it's ok. Why?! 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.CS.McGill.CA/pipermail/soot-list/attachments/20180918/d5a2793c/attachment.html>