[Soot-list] Class Loading - Bug or Feature?

[Marc-André Laverdière]

So that answers my question - 'feature'

One of the things I noticed in the implementation is that Soot still
loads a ton of stuff in memory even if you want to load classes at a
level lower than BODIES. In the ASM branch, is there some kind of
optimization for that?

I know that Karim Ali's Averroes is able to go through a huge library
like the JDK super fast because it goes only through the constant pool.

(And Phil told me he has a good implementation too - maybe using the
same trick)

I think that this is worth investigating. Nobody wants to load 1000
classes that aren't reacheable from the main and jimplify them,
obviously. At the same time, I think we need to be able to answer those
questions.

Marc-André Laverdière-Papineau
Doctorant - PhD Candidate

On 03/18/2014 05:09 PM, Steven Arzt wrote:
> Hi all,
> 
> This is expected behavior if you ask me. Soot only loads the classes it really really needs to load because class loading is expensive both in terms of memory and performance.
> 
> In your example, you are suggesting to load each and every class on the classpath to at least Hierarchy level because it might be a transitive descendant / implementer of List. We have experimented with such eager loading in FlowDroid because we experimented with algorithms that ask very similar questions as what you have in the example. After loading the basic classes, you can just iterate over the rest (dangling classes) and force-load them, so there is no need to have such a feature directly in Soot. Having said that, we dropped the idea altogether in our experiments because loading a compete hierarchy of the jdk is only feasible on big machines with lots of memory and takes forever. With Soot's current class loading and representation in memory, you want to avoid this at all costs.
> 
> I know that this is not the answer you're looking for and it was a pity since we had a really cool idea with FlowDroid (on-demand icfg), but apparently that's how it is.
> 
> Best regards,
> Steven
> 
> Marc-André Laverdière	 <marc-andre.laverdiere-papineau at polymtl.ca> wrote:
> 
>> Hello everybody,
>>
>> There are things about Soot's bug loading that I don't know if they are
>> bugs, or features meant to save memory. This is one of them.
>>
>> The problem I have is that I often need to ask the question "Which
>> classes implement interface X?" and "which classes are subclasses of Y?"
>> And the current class loading is leaving me high and dry.
>>
>> Here is a simple test case that shows how the current class loading can
>> break:
>>
>> G.reset();
>> Options.v().set_prepend_classpath(true);
>> Options.v().set_time(false);
>> Scene.v().addBasicClass("java.util.List");
>> Scene.v().loadNecessaryClasses();
>> SootClass arrayList = Scene.v().getSootClass("java.util.ArrayList");
>> Hierarchy h = new Hierarchy();
>>
>> //This test works only if java.util.ArrayList is added as a basic class
>> //Or if ArrayList is loaded with forceResolve
>> assertTrue("ArrayList should be an implementer of List",
>> h.getImplementersOf(list).contains(arrayList));
>>
>> The test is false. There are no classes implementing List as far as Soot
>> can see.
>>
>> Is this behavior something wanted (a feature)? Or is it a bug?
>>
>> This can be avoided easily by using Scene.forceResolve on all dandling
>> classes. If this is a bug, I'll open a bug in Github and add that
>> resolution step. If it is a feature, should we add a command-line flag
>> for Soot to perform the additional resolution for people like me who
>> need it?
>>
>> Regards,
>>
>> -- 
>> Marc-André Laverdière-Papineau
>> Doctorant - PhD Candidate
>> _______________________________________________
>> Soot-list mailing list
>> Soot-list at sable.mcgill.ca
>> http://mailman.cs.mcgill.ca/mailman/listinfo/soot-list
>