[Soot-list] Missing callgraph edges with Spark

Fri May 13 17:55:03 EDT 2022

 Dear Prof. Dr. Arzt
Regarding the solutions you mentioned:
b) Implement an imprecise mode that picks up all fragments, regardless of whether they are used or not. This would, however, increase the size and complexity of the dummy main method, because you would need to associate each fragment with each activity.> Yes, I agree this might be too imprecise. I was thinking of possibly treating all fragments as independent components (e.g like activities) for which  lifecycles would be modelled resolved by default, then only when used by some activity, they would be integrated within that activity lifecycle. But this might defeat the purpose of having the dummy main as a single entrypoint, so I'll focus on option a) instead.

> a) Find the code or XML that references the fragment and check why FlowDroid doesn't recognize it. Extend FlowDroid to detect this reference.
I started looking into the fragment identification to figure out what was going wrong on my example. It seems the issue occurs when resolving the type of the fragment in AbstractCallbackAnalyzer.analyzeMethodForFragmentTransaction
Value br = stmt.getInvokeExpr().getArg(i);

// Is this a fragment?
if (br.getType() instanceof RefType) {
   RefType rt = (RefType) br.getType();Here, only the static type of the fragment is retained, independently from the context, which will default to parameter types for method invocations. For e.g:
void createFragment(android.app.Fragment fragment){    fragmentManager.beginTransaction().add(fragment)
}
might be invoked with different types of fragments, but only android.app.Fragment will be returned here. 
I thought of using PointsToAnalysis.reachingObjects, then add all possible types as fragments of that activity. However, the pointer analysis is only set after the callgraph is constructed and is still a DumbPointerAnalysis in this stage.
Thus, I think a solution would either be to set the pointer analysis earlier than when applying the "cg" pack or to perform a backward data-flow analysis for definitions of the fragment object. Would you recommend one over the other?

Best regards,Faridah Akinotcho
    Le vendredi 13 mai 2022, 14:14:15 UTC−7, Arzt, Steven <steven.arzt at sit.fraunhofer.de> a écrit :  

 Dear Faridah,

FlowDroid integrates fragments into its dummy main method if it detects that the fragment is actually used in some activity. This allows FlowDroid to precisely associate fragments with their respective parent activity. Therefore, if the fragment is there, but FlowDroid cannot find code that instantiates it, or XML files that reference it, the fragment is missed. I see two possible solutions:

a) Find the code or XML that references the fragment and check why FlowDroid doesn't recognize it. Extend FlowDroid to detect this reference.

b) Implement an imprecise mode that picks up all fragments, regardless of whether they are used or not. This would, however, increase the size and complexity of the dummy main method, because you would need to associate each fragment with each activity.

Best regards,
  Steven

-----Original Message-----
From: Faridah Akinotcho <faridath.akinotcho at yahoo.fr> 
Sent: Freitag, 13. Mai 2022 21:10
To: Eric Bodden <eric.bodden at uni-paderborn.de>; Arzt, Steven <steven.arzt at sit.fraunhofer.de>
Cc: Soot-list <soot-list at cs.mcgill.ca>
Subject: Re: [Soot-list] Missing callgraph edges with Spark

Hello Prof. Dr. Arzt, Prof.Dr.Bodden,

I reproduced the issue of edges missing for fragment lifecycles using Flowdroid 2.11-SNAPSHOT as well. 
Looking into AndroidEntryPointCreator, as you recommended, it seems method active bodies (and thus outgoing edges), will only be resolved for added fragments (i.e for which transactions and the correct type were identified by the CallbackAnalyzer ) but not for any other fragment class i.e classes within the app's namespace which extend android.app.Fragment or Support Library fragments.

Is this assumption correct? 

If that is the case, I guess one would either need to refine the fragment identifcation for those missing fragments or somehow modify the analysis to resolve active bodies for any fragment lifecycle as long as the fragment is within the app package.

Best regards,
Faridah Akinotcho

Le mardi 10 mai 2022, 12:31:51 UTC−7, Arzt, Steven <steven.arzt at sit.fraunhofer.de> a écrit : 

Dear Faridah,

FlowDroid applies a precise model for app components including fragments. You may want to look into the "AndroidEntryPointCreator" to identify any missing parts. Maybe there is a bug. Feel free to open merge request if you find something.

Best regards,
  Steven

-----Original Message-----
From: Faridah Akinotcho <faridath.akinotcho at yahoo.fr <mailto:faridath.akinotcho at yahoo.fr> > 
Sent: Dienstag, 10. Mai 2022 21:28
To: Eric Bodden <eric.bodden at uni-paderborn.de <mailto:eric.bodden at uni-paderborn.de> >; Arzt, Steven <steven.arzt at sit.fraunhofer.de <mailto:steven.arzt at sit.fraunhofer.de> >
Cc: Soot-list <soot-list at cs.mcgill.ca <mailto:soot-list at cs.mcgill.ca> >
Subject: Re: [Soot-list] Missing callgraph edges with Spark

Hi Prof. Dr. Arzt, Prof. Dr. Bodden,

I followed your recommendations of using Flowdroid and preprocessors, instead of Soot, for which I thank you once again.

While it did provide me with a much more precise callgraph on the example I showed before, some of the edges that were found previously by Soot are now missing, and from a first look, it seems those are all within fragment lifecycles (e.g onViewCreated). Is there anything regarding how fragment classes are handled by Flowdroid which you believe could explain this behavior?

Best regards,
Faridah Akinotcho 

Le lundi 9 mai 2022, 19:27:11 UTC−7, Faridah Akinotcho <faridath.akinotcho at yahoo.fr <mailto:faridath.akinotcho at yahoo.fr> > a écrit : 

Thank you for clarifying this! I'll try setting up my analysis within this Flowdroid callback instead.

Best regards,
Faridah Akinotcho

Le lundi 9 mai 2022, 12:12:00 UTC−7, Arzt, Steven <steven.arzt at sit.fraunhofer.de <mailto:steven.arzt at sit.fraunhofer.de> > a écrit : 

Dear Faridah,

You can call SetupApplication.addPreprocessor() to add a callback that gets invoked once the callgraph construction is done in FlowDroid. The interface has two methods, one before and one after callgraph construction. You just need to use the FlowDroid API, i.e., the SetupApplication class. That should be fairly simple.

Best regards,
  Steven

-----Original Message-----
From: Soot-list <soot-list-bounces at CS.McGill.CA <mailto:soot-list-bounces at CS.McGill.CA>  <mailto:soot-list-bounces at CS.McGill.CA <mailto:soot-list-bounces at CS.McGill.CA> > > On Behalf Of Faridah Akinotcho
Sent: Montag, 9. Mai 2022 21:11
To: Eric Bodden <eric.bodden at uni-paderborn.de <mailto:eric.bodden at uni-paderborn.de>  <mailto:eric.bodden at uni-paderborn.de <mailto:eric.bodden at uni-paderborn.de> > >
Cc: Soot-list <soot-list at cs.mcgill.ca <mailto:soot-list at cs.mcgill.ca>  <mailto:soot-list at cs.mcgill.ca <mailto:soot-list at cs.mcgill.ca> > >
Subject: Re: [Soot-list] Missing callgraph edges with Spark

Hi Prof. Dr. Bodden,

Thank you for the reply! I am using bare Soot because I am trying to add a wjtp subphase and I couldn't figure out how to do with Flowdroid without having to extend it and override constructCallgraph. Would there be a way for me to this with Flowdroid?

Best regards,
Faridah Akinotcho

Le lundi 9 mai 2022, 05:20:22 UTC−7, Eric Bodden <eric.bodden at uni-paderborn.de <mailto:eric.bodden at uni-paderborn.de>  <mailto:eric.bodden at uni-paderborn.de <mailto:eric.bodden at uni-paderborn.de> > > a écrit : 

Hi Faridah. 

When creating callgraphs for Android you should be using our Soot-extension FlowDroid because it models Android’s lifecycle. When you use bare Soot then the callgraph will be very incomplete. Are you using FlowDroid already?

Cheers
Eric

    On 9. May 2022, at 00:11, Faridah Akinotcho <faridath.akinotcho at yahoo.fr <mailto:faridath.akinotcho at yahoo.fr>  <mailto:faridath.akinotcho at yahoo.fr <mailto:faridath.akinotcho at yahoo.fr> >  <mailto:faridath.akinotcho at yahoo.fr <mailto:faridath.akinotcho at yahoo.fr>  <mailto:faridath.akinotcho at yahoo.fr <mailto:faridath.akinotcho at yahoo.fr> > > > wrote:

    Hi,

    I am using Soot (org.soot.oss 4.2.1) to perform an interprocedural analysis on Android methods (in the wjtp phase). To this end, I am extracting reachable methods from Android lifecycles using Scene.v().getCallgraph().edgesOutOf(method). However, I noticed that some virtual and special edges seem to always be missing from the returned set. For e.g, given:

    class B extends A{
        void onCreate(...){
            setContentView(C0027R.layout.merge);
            super.onCreate(bundle);
            setNextActivity(C.class);
        }
    }

    class A{
        public void setNextActivity(){
            ...
        }
    }

    When computing the reachable methods for B.onCreate(), neither setContentView, super.onCreate or setNextActivity are included as targets for the outgoing edges (only clinit is returned). However, when using CHA instead of Spark, these are part of the returned set (along other overapproximations), so I am assuming there might be some issue when performing the points-to analysis. Here are the options I am using:

    Options.v().set_src_prec(Options.src_prec_apk);
    Options.v().set_output_format(Options.output_format_none);

    Options.v().set_no_bodies_for_excluded(true);
    Options.v().set_allow_phantom_refs(true);
    Options.v().set_android_jars(androidJar);

    Options.v().set_process_dir(Collections.singletonList(apkPath));
    Options.v().set_soot_classpath(androidJar);
    Options.v().set_process_multiple_dex(true);

    Options.v().set_whole_program(true);
    Options.v().setPhaseOption("cg", "all-reachable:true");
    Options.v().setPhaseOption("cg.spark", "on");
    Options.v().setPhaseOption("cg.spark", "verbose:true");
    Options.v().setPhaseOption("cg.spark", "string-constants:true");

    Options.v().setPhaseOption("jb.ulp", "off");

    Main.v().autoSetOptions();
    Scene.v().loadNecessaryClasses();

    I would greatly appreciate any pointers about what I might be missing (maybe an option that wasn't set properly) or what the problem could be.

    Best regards,
    Faridah Akinotcho

    _______________________________________________
    Soot-list mailing list
    Soot-list at CS.McGill.CA <mailto:Soot-list at CS.McGill.CA>  <mailto:Soot-list at CS.McGill.CA <mailto:Soot-list at CS.McGill.CA> >  <mailto:Soot-list at CS.McGill.CA <mailto:Soot-list at CS.McGill.CA>  <mailto:Soot-list at CS.McGill.CA <mailto:Soot-list at CS.McGill.CA> > > 

    https://mailman.CS.McGill.CA/mailman/listinfo/soot-list

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.CS.McGill.CA/pipermail/soot-list/attachments/20220513/4df68b82/attachment-0001.html>