[Soot-list] Generating Android APK CallFlowGraph

Wed Mar 4 05:36:37 EST 2015

Hi Monika,

You get these warnings because your APK file references classes which are not included in your Android JAR file. This is usually not a problem as long as you do not wish to analyze the code of the Android operating system itself.

Best regards,

  Steven

Von: soot-list-bounces at CS.McGill.CA [mailto:soot-list-bounces at CS.McGill.CA] Im Auftrag von Monika Mashalkar
Gesendet: Mittwoch, 25. Februar 2015 13:51
An: Steven Arzt
Cc: soot-list at CS.McGill.CA; soot-list at sable.mcgill.ca; soot-list at googlegroups.com
Betreff: Re: [Soot-list] Generating Android APK CallFlowGraph

Hi Steven,

Thanks for your quick reply. I am able to run the code now. But I am getting the warnings as follow :

Warning: java.lang.ref.Finalizer is a phantom class!

Warning: android.graphics.pdf.PdfDocument$Page is a phantom class!

Warning: android.graphics.pdf.PdfDocument$PageInfo is a phantom class!

Warning: android.graphics.pdf.PdfDocument is a phantom class!

Warning: android.media.RemoteControlClient$OnGetPlaybackPositionListener is a phantom class!

Warning: android.media.RemoteControlClient$OnPlaybackPositionUpdateListener is a phantom class!

Warning: android.print.PageRange is a phantom class!

Warning: android.print.PrintAttributes$Builder is a phantom class!

Warning: android.print.PrintAttributes$MediaSize is a phantom class!

Warning: android.print.PrintAttributes is a phantom class!

Warning: android.print.PrintDocumentAdapter$LayoutResultCallback is a phantom class!

Warning: android.print.PrintDocumentAdapter$WriteResultCallback is a phantom class!

Warning: android.print.PrintDocumentAdapter is a phantom class!

Warning: android.print.PrintDocumentInfo$Builder is a phantom class!

Warning: android.print.PrintDocumentInfo is a phantom class!

Warning: android.print.PrintJob is a phantom class!

Warning: android.print.PrintManager is a phantom class!

Warning: android.print.pdf.PrintedPdfDocument is a phantom class!

Warning: android.view.ViewTreeObserver$OnWindowAttachListener is a phantom class!

Warning: android.view.ViewTreeObserver$OnWindowFocusChangeListener is a phantom class!

May I know the reason, why I am getting these warnings ?

The command that I am using to run the program is :

monika at monika-OptiPlex-990:~$ java -cp .:soot-trunk.jar:soot-infoflow.jar:soot-infoflow-android.jar:slf4j-api-1.7.5.jar:slf4j-simple-1.7.5.jar:axml-2.0.jar:/usr/lib/jvm/java-7-openjdk-i386/jre/lib/rt.jar CFG

I am attaching the output file with this mail. Please, tell me I am getting it correct or not.

Thanks and regards,

Monika

On Wed, Feb 25, 2015 at 3:51 PM, Steven Arzt <Steven.Arzt at cased.de> wrote:

Hi Monika,

The CFG class is not part of Soot. It was just written as an example. You need to compile it on your own.

Best regards,

  Steven

M.Sc. M.Sc. Steven Arzt

Secure Software Engineering Group (SSE)

European Center for Security and Privacy by Design (EC SPRIDE) 

Rheinstraße 75

D-64293 Darmstadt

Phone: +49 61 51 869-336

Fax: +49 61 51 16-72118

eMail:  <mailto:steven.arzt at ec-spride.de> steven.arzt at ec-spride.de

Web: http://sse.ec-spride.de <http://sse.ec-spride.de/> 

Von: Monika Mashalkar [mailto:monikamashalkar at gmail.com] 
Gesendet: Mittwoch, 25. Februar 2015 05:30
An: soot-list at googlegroups.com
Cc: soot-list at sable.mcgill.ca; soot-list at cs.mcgill.ca; Steven.Arzt at cased.de
Betreff: Re: [Soot-list] Generating Android APK CallFlowGraph

Hi Lokesh,

I am very new to the SOOT Framework and Flow-droid. I want to find the flow graph from android apk in my project and I am trying to run the code posted in this thread to do the same. I am trying to run the following command on Ubuntu 

Command : java -cp soot-trunk.jar:soot-infoflow.jar:soot-infoflow-android.jar:slf4j-api-1.7.5.jar:slf4j-simple-1.7.5.jar:axml-2.0.jar -cp .:/usr/lib/jvm/java-7-openjdk-i386/jre/lib/rt.jar CFG

but I am getting Error as : Error: Could not find or load main class CFG

Could you please tell me the command to execute the CFG program.

Thank you,

Monika

On Monday, 21 July 2014 13:46:37 UTC+5:30, LOKESH JAIN wrote:

Hi all,

I resolved the issue and finally i am getting the call graph. Thank you all of you for your time and help. :)

1. But Stevan I am still curious to know how to work with QueueReader Object for generating dot format.

2. And why the size of call graph for RV2013.apk is 54. And is there any way to remove unnecessary size.

Thanks&Regards

Lokesh

On Mon, Jul 21, 2014 at 11:38 AM, LOKESH JAIN <lokesh... at gmail.com> wrote:

Hi all,

Steven I still couldn't figure it out how to work with queuereader object for generating dot format.

I have used dot graph class manually as suggested to me by Stefan but,

I am getting NullPointerException. I don't know why. I have pasted the complete code.

Please help me out.

Exception in thread "main" java.lang.NullPointerException

at DotGraph.getNode(DotGraph.java:53)

at DotGraph.drawNode(DotGraph.java:61)

at CFG.visit(CFG.java:114)

at CFG.main(CFG.java:94)

CFG.java

import java.io.BufferedOutputStream;

import java.io.FileOutputStream;

import java.io.IOException;

import java.io.OutputStream;

import java.util.Collections;

import java.util.HashMap;

import java.util.HashSet;

import java.util.Iterator;

import java.util.LinkedList;

import java.util.List;

import org.xmlpull.v1.XmlPullParserException;

import soot.MethodOrMethodContext;

import soot.PackManager;

import soot.Scene;

import soot.SootMethod;

import soot.jimple.infoflow.android.SetupApplication;

import soot.jimple.toolkits.callgraph.CallGraph;

import soot.jimple.toolkits.callgraph.Targets;

import soot.options.Options;

import soot.util.dot.DotGraphUtility;

import soot.util.dot.Renderable;

public class CFG {

private static DotGraph dot = new DotGraph("callgraph");

private static HashMap <String,Boolean> visited = new HashMap<String,Boolean>(); 

public CFG() {

}

public static void main(String[] args) {

// TODO Auto-generated method stub

SetupApplication app = new SetupApplication("/home/lokesh/Desktop/android-sdk-linux/platforms/android-19/android.jar","/home/lokesh/Desktop/android-instrumentation-tutorial-master/app-example/RV2013/bin/RV2013.apk");

try {

app.calculateSourcesSinksEntrypoints("/home/lokesh/Downloads/soot-infoflow-android-develop/SourcesAndSinks.txt");

} catch (IOException e) {

// TODO Auto-generated catch block

e.printStackTrace();

} catch (XmlPullParserException e) {

// TODO Auto-generated catch block

e.printStackTrace();

}

soot.G.reset();

Options.v().set_src_prec(Options.src_prec_apk);

Options.v().set_process_dir(Collections.singletonList("/home/lokesh/Desktop/android-instrumentation-tutorial-master/app-example/RV2013/bin/RV2013.apk"));

Options.v().set_force_android_jar("/home/lokesh/Desktop/android-sdk-linux/platforms/android-19/android.jar");

Options.v().set_whole_program(true);

Options.v().set_allow_phantom_refs(true);

Options.v().set_output_format(Options.output_format_none);

Options.v().setPhaseOption("cg.spark verbose:true", "on");

Scene.v().loadNecessaryClasses();

SootMethod entryPoint = app.getEntryPointCreator().createDummyMain();

Options.v().set_main_class(entryPoint.getSignature());

Scene.v().setEntryPoints(Collections.singletonList(entryPoint));

System.out.println("............"+entryPoint.getActiveBody());

PackManager.v().runPacks();

System.out.println(Scene.v().getCallGraph().size());

CallGraph cg = Scene.v().getCallGraph();

visit(cg, entryPoint);

dot.plot("/home/lokesh/Desktop/soot1"+ dot.DOT_EXTENSION);

}

private static void visit(CallGraph cg, SootMethod k)

{

String identifier = k.getName();

visited.put(k.getSignature(),true);

dot.drawNode(identifier);

//iterate over unvisited parents

Iterator<MethodOrMethodContext> ptargets = new Targets(cg.edgesInto(k));

if(ptargets != null){

while(ptargets.hasNext())

{

SootMethod p = (SootMethod) ptargets.next();

if(p == null) System.out.println("p is null");

if(!visited.containsKey(p.getSignature()))

visit(cg,p);

}

}

//iterate over unvisited children

Iterator<MethodOrMethodContext> ctargets = new Targets(cg.edgesOutOf(k));

if(ctargets != null){

while(ctargets.hasNext())

{

SootMethod c = (SootMethod) ctargets.next();

if(c == null) System.out.println("c is null");

dot.drawEdge(identifier, c.getName());

if(!visited.containsKey(c.getSignature()))

visit(cg,c);

}

}

}

}

DotGraph.java

import java.io.BufferedOutputStream;

import java.io.FileOutputStream;

import java.io.IOException;

import java.io.OutputStream;

import java.util.HashMap;

import java.util.LinkedList;

import java.util.List;

import soot.util.dot.DotGraphEdge;

import soot.util.dot.DotGraphNode;

import soot.util.dot.DotGraphUtility;

import soot.util.dot.Renderable;

public class DotGraph implements Renderable

{

public final static String DOT_EXTENSION = ".dot";

private HashMap<String, DotGraphNode> nodes;

private boolean isSubGraph;

private List<Renderable> drawElements;

private String graphname;

public DotGraph(String graphname)

{

this.drawElements = new LinkedList<Renderable>();

}

public DotGraph createSubGraph(String label)

{

DotGraph subgraph = new DotGraph(label);

subgraph.isSubGraph = true;

this.drawElements.add(subgraph);

System.out.println(subgraph);

return subgraph;

}

@Override

public void render(OutputStream out, int indent) throws IOException {

String graphname = this.graphname;

   if (!isSubGraph) {

     DotGraphUtility.renderLine(out, "digraph \""+graphname+"\" {", indent);

   } else {

 DotGraphUtility.renderLine(out, "subgraph \""+graphname+"\" {", indent);

   }

}

public void plot(String filename) {

    try {

      BufferedOutputStream out =new BufferedOutputStream(new FileOutputStream(filename));

      render(out, 0);

      out.close();

    } catch (IOException ioe) {

    }

  }

public DotGraphNode getNode(String name){

DotGraphNode node = nodes.get(name);

if (node == null) {

node = new DotGraphNode(name);

nodes.put(name, node);

}

return node;

}

public DotGraphNode drawNode(String name){

DotGraphNode node = getNode(name);

if(node == null)

throw new RuntimeException("Assertion failed.");

if(!this.drawElements.contains(node))

this.drawElements.add(node);

return node;

  }

public DotGraphEdge drawEdge(String from, String to) {

DotGraphNode src = drawNode(from);

DotGraphNode dst = drawNode(to);

DotGraphEdge edge = new DotGraphEdge(src, dst);

this.drawElements.add(edge);

return edge;

  }

} 

Thanks & Regards

Lokesh

On Sun, Jul 20, 2014 at 5:10 PM, Stefan Gommer <gom... at informatik.uni-bremen.de> wrote:

Hi Lokesh,

sorry I misspelled it. Originally it is of type HashMap<String,Boolean>.

You can also use HashSet<String>. In this case you have to use visited.add(k.getSignature());

Regards,

Stefan

Am 20.07.2014 um 13:35 schrieb LOKESH JAIN <lokesh... at gmail.com>:

Hi Stefan,

HashSet<String,Boolean> type is giving error whereas HashSet<String> is correct form. But

then

visited.put(k.getSignature(),true) is giving error.

How do i resolve this?

Thanks & Regards

Lokesh

On Sun, Jul 20, 2014 at 4:52 PM, LOKESH JAIN <lokesh... at gmail.com> wrote:

Hi Stefan,

HashSet<String,Boolean> type is giving error whereas HashSet<String> is correct form. But

then

visited.put(k.getSignature(),true) is giving error.

Thanks & Regards

Lokesh

On Sun, Jul 20, 2014 at 4:16 PM, Stefan Gommer <gom... at informatik.uni-bremen.de> wrote:

Hi Lokesh,

visited is of type HashSet<String,Boolean> but can also be any kind of List like HashSet<String>

Regards

Stefan 

Am 20.07.2014 um 12:44 schrieb LOKESH JAIN <lokesh... at gmail.com>:

Hey Stefan!

What is the type of "visited" ?

Thanks & Regards

Lokesh

On Sun, Jul 20, 2014 at 2:57 PM, Stefan Gommer <gom... at informatik.uni-bremen.de> wrote:

Hi Lokesh,

I did it like this (I skipped the standard soot settings:

private static DotGraph dot = new DotGraph("CallGraph“);

...

SootMethod entryPoint = app.getEntryPointCreator().createDummyMain();

…

PackManager.v().runPacks();

CallGraph cg = Scene.v().getCallGraph();

visit(cg, entryPoint);

…

dot.plot("/Users/neji/Desktop/graph"+ dot.DOT_EXTENSION);

…

private static void visit(CallGraph cg, SootMethod k)

{

String identifier = k.getName();

visited.put(k.getSignature(), true);

dot.drawNode(identifier);

//iterate over unvisited parents

Iterator<MethodOrMethodContext> ptargets = new Targets(cg.edgesInto(k));

if(ptargets != null){

while(ptargets.hasNext())

{

SootMethod p = (SootMethod) ptargets.next();

if(p == null) System.out.println("p is null");

if(!visited.containsKey(p.getSignature()))

visit(cg,p);

}

}

//iterate over unvisited children

Iterator<MethodOrMethodContext> ctargets = new Targets(cg.edgesOutOf(k));

if(ctargets != null){

while(ctargets.hasNext())

{

SootMethod c = (SootMethod) ctargets.next();

if(c == null) System.out.println("c is null");

dot.drawEdge(identifier, c.getName());

if(!visited.containsKey(c.getSignature()))

visit(cg,c);

}

}

}

Am 20.07.2014 um 09:44 schrieb LOKESH JAIN <lokesh... at gmail.com>:

I have tried DotGraph class from the soot.util.dot package but it's giving me null pointer exception. Following is the tried code.

I have converted QueueReader object into string as DotGraph takes only string argument. It's getting complicated. Please suggest some simple solution.

import java.io.IOException;
import java.io.OutputStream;
import java.util.Collections;
import java.util.LinkedList;
import java.util.List;

import org.xmlpull.v1.XmlPullParserException;

import soot.PackManager;
import soot.Scene;
import soot.SootMethod;
import soot.jimple.infoflow.android.SetupApplication;
import soot.options.Options;
import soot.util.dot.DotGraphUtility;
import soot.util.dot.Renderable;
public class CFG {
     //private static Map options;
    static DotGraph dg;
public CFG() {
//this.options=options;
// TODO Auto-generated constructor stub

}
/*public DotGraph createSubGraph(String label)
{

DotGraph createSubGraph = DotGraph.createSubGraph(Scene.v().getCallGraph().listener().toString());    
}*/
public static void main(String[] args) {

// TODO Auto-generated method stub

SetupApplication app = new SetupApplication("/home/lokesh/Desktop/android-sdk-linux/platforms/android-19/android.jar","/home/lokesh/Desktop/android-instrumentation-tutorial-master/app-example/RV2013/bin/RV2013.apk");

try {

app.calculateSourcesSinksEntrypoints("/home/lokesh/Downloads/soot-infoflow-android-develop/SourcesAndSinks.txt");

} catch (IOException e) {

// TODO Auto-generated catch block

e.printStackTrace();

} catch (XmlPullParserException e) {

// TODO Auto-generated catch block

e.printStackTrace();

}

soot.G.reset();

Options.v().set_src_prec(Options.src_prec_apk);

Options.v().set_process_dir(Collections.singletonList("/home/lokesh/Desktop/android-instrumentation-tutorial-master/app-example/RV2013/bin/RV2013.apk"));
Options.v().set_force_android_jar("/home/lokesh/Desktop/android-sdk-linux/platforms/android-19/android.jar");

Options.v().set_whole_program(true);

Options.v().set_allow_phantom_refs(true);

Options.v().set_output_format(Options.output_format_none);

Options.v().setPhaseOption("cg.spark verbose:true", "on");

Scene.v().loadNecessaryClasses();

SootMethod entryPoint = app.getEntryPointCreator().createDummyMain();

Options.v().set_main_class(entryPoint.getSignature());

Scene.v().setEntryPoints(Collections.singletonList(entryPoint));

System.out.println("............"+entryPoint.getActiveBody());

PackManager.v().runPacks();

System.out.println(Scene.v().getCallGraph().size());
String label = Scene.v().getCallGraph().listener().toString();
dg.createSubGraph(label);
}
}

import java.io.IOException;
import java.io.OutputStream;
import java.util.LinkedList;
import java.util.List;

import soot.util.dot.DotGraphUtility;
import soot.util.dot.Renderable;

public class DotGraph implements Renderable
{
private boolean isSubGraph;
private List<Renderable> drawElements;
private String graphname;
public DotGraph(String graphname)
{
this.drawElements = new LinkedList<Renderable>();
}
public DotGraph createSubGraph(String label)
{
DotGraph subgraph = new DotGraph(label);
subgraph.isSubGraph = true;
this.drawElements.add(subgraph);
System.out.println(subgraph);
return subgraph;
}
@Override
public void render(OutputStream out, int indent) throws IOException {
    String graphname = this.graphname;
        if (!isSubGraph) {
          DotGraphUtility.renderLine(out, "digraph \""+graphname+"\" {", indent);
        } else {
 DotGraphUtility.renderLine(out, "subgraph \""+graphname+"\" {", indent);

        }

}
} 

Thanks & Regards

Lokesh

On Sat, Jul 19, 2014 at 10:57 PM, Stefan Gommer <gommer... at googlemail.com> wrote:

Hi Lokesh,

you could use the DotGraph class from the soot.util.dot package and build a Graph on your own while traversing through the cfg. I don’t know if there is a method to do this automatically.

Stefan

Am 19.07.2014 um 15:16 schrieb LOKESH JAIN <lokesh... at gmail.com>:

Hi Steven,

this Scene.v().getCallgraph().listener() is giving me QueueReader object. But I am not able to create a dot file from it. I have tried dump_cfg and various others options but all are of no help. 

1. Please can you tell me how do i print a dot file out of QueueReader object?

Also I am getting the call graph size as 54 for the general example apk "RV2013.apk".

2. How come call graph size is 54?

On Fri, Jul 18, 2014 at 9:37 PM, Steven Arzt <Steve... at cased.de> wrote:

Hi Lokesh,

You can manually iterate over the callgraph using Scene.v().getCallgraph().listener() and print it out in dot format  which should be fairly simple as it is just a QueueReader over the edges. Still, I think Soot can also directly do that, but I have never used  the dot visualizer for callgraphs.

Afterwards, you can visualize the dot file using GraphViz.

Best regards,

  Steven

Von: LOKESH JAIN [mailto:lokesh... at gmail.com] 
Gesendet: Freitag, 18. Juli 2014 18:04
An: Steven Arzt
Cc: Modhi Alsobiehy; soot... at cs.mcgill.ca; soot... at sable.mcgill.ca; soot... at googlegroups.com

Betreff: Re: [Soot-list] Generating Android APK CallFlowGraph

Hi,

A gentle reminder. Please reply fast. I need it for my research project.

Regards

Lokesh

On Thu, Jul 17, 2014 at 9:21 PM, LOKESH JAIN <lokesh... at gmail.com> wrote:

Thanks a lot Steven and Modhi. It worked.

It is showing call graph in words. How can I graphically see the call graph for apk files as we see for java files??

Regards

Lokesh Jain

On Thu, Jul 17, 2014 at 6:11 PM, Steven Arzt <Steve... at cased.de> wrote:

Hi Lokesh,

It seems that you are missing the axml-2.0.jar library on your Java classpath.

Best regards,

  Steven

Von: soot-lis... at CS.McGill.CA [mailto:soot-lis... at CS.McGill.CA] Im Auftrag von Modhi Alsobiehy
Gesendet: Donnerstag, 17. Juli 2014 14:39
An: LOKESH JAIN
Cc: soot... at cs.mcgill.ca; soot... at sable.mcgill.ca; soot... at googlegroups.com
Betreff: Re: [Soot-list] Generating Android APK CallFlowGraph

Hi Lokesh,

Open the folder of your project and add the call back file into it.

-Best,

Modhi

On Jul 17, 2014, at 12:03 PM, "LOKESH JAIN" <lokesh... at gmail.com> wrote:

Hi all,

Modhi, I am using your code for generating call graph for apk file. But, I am getting following error

Exception in thread "main" java.lang.NoClassDefFoundError: pxb/android/axml/NodeVisitor
    at soot.jimple.infoflow.android.manifest.ProcessManifest.handle(Unknown Source)
    at soot.jimple.infoflow.android.manifest.ProcessManifest.<init>(Unknown Source)
    at soot.jimple.infoflow.android.manifest.ProcessManifest.<init>(Unknown Source)
    at soot.jimple.infoflow.android.SetupApplication.calculateSourcesSinksEntrypoints(Unknown Source)
    at soot.jimple.infoflow.android.SetupApplication.calculateSourcesSinksEntrypoints(Unknown Source)
    at CFG.main(CFG.java:33)
Caused by: java.lang.ClassNotFoundException: pxb.android.axml.NodeVisitor
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    ... 6 more

It's pretty much the same error that you were getting. Though I had checked , there is no duplicate library added. Also there is already AndroidCallbacks.txt in soot-infoflow-android, i haven't understood how to add it into my project.
Any help would be appreciated. Thank You
Regards
Lokesh

On Saturday, 5 July 2014 23:04:49 UTC+5:30, Modhi Alsobiehy wrote:

Hi all,

I need to traverse the call flow graph of android apps( apk files).

I understand that I must set an entry point so to start, I tried the following code :

---------------------

...

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailman.CS.McGill.CA/pipermail/soot-list/attachments/20150304/43bde60f/attachment-0003.html 