[Soot-list] Lazy loading of .class files

[Niklas Therning]

Hi,

I'm working on an ahead of time (AOT) compiler for java bytecode. What 
I'm doing is to compile java bytecode into LLVM bitcode and then use the 
LLVM tools and GCC to get an x86/arm/whatever object file that can be 
linked into a native executable. I'm basing the standard classes 
(java.*, javax.*, etc) on the Apache Harmony project and I've written a 
small runtime in C. As of now I'm able to produce x86/x86-64 Linux 
executables that work for simple Java programs. I'm also working on an 
Eclipse plugin which extends the Eclipse JDT with incremental 
compilation of Java to native code using my AOT compiler.

I'm using Soot in the first stage (JVM byte code -> LLVM bitcode) since 
Soot's register based Jimple representation is closer to LLVM's bitcode 
representation than the byte code of the JVM. And I get Soot's 
optimization for free which is very nice! Here's how I'm initializing Soot:

         soot.G.reset();
         Options.v().set_output_format(Options.output_format_jimple);
         Options.v().set_include_all(true);
         Options.v().setPhaseOption("jap.npc", "enabled:true");
         Options.v().setPhaseOption("jap.abc", "enabled:true");
         Options.v().set_print_tags_in_output(true);
         Options.v().set_allow_phantom_refs(true);
         Options.v().set_soot_classpath(getSootClasspath(clazzes));

         for (Clazz clazz : clazzes.list()) {
             SootClass c = 
Scene.v().loadClassAndSupport(clazz.getClassName());
             c.setApplicationClass();
         }
         Scene.v().loadNecessaryClasses();

(clazzes handles the classpath entries and Clazz is a wrapper around a 
.class entry in a JAR file or a .class File in a directory)

My problem is that the code above takes about 5 seconds on my machine. 
The JRE classes from Harmony are always loaded by the code above so even 
if I'm compiling a single .class file more than 3000 .class files from 
Harmony have to be loaded by Soot. I would like Soot to load the 
required classes lazily when they are needed. This is especially 
important when I run the compiler from within my Eclipse plugin.

I've tried to replace the loop and call to loadNecessaryClasses() above 
with the following code:

         for (Clazz clazz : clazzes.list()) {
             SootClass c = Scene.v().loadClass(clazz.getClassName(), 
SootClass.DANGLING);
             c.setApplicationClass();
         }

That doesn't work out of the box so I've patched Soot 2.4 in a few 
places. I had to change the checkLevel() method in SootClass to

     public void checkLevel( int level ) {
         if( resolvingLevel < level ) {
             SootResolver.v().resolveClass(this, level);
//            String hint = "\nIf you are extending Soot, try to add the 
following call before calling soot.Main.main(..):\n" +
//                          
"scene().addBasicClass("+getName()+","+levelToString(level)+");\n" +
//                          "Otherwise, try whole-program mode (-w).";
//            throw new RuntimeException(
//                "This operation requires resolving level "+
//                levelToString(level)+" but "+name+
//                " is at resolving level 
"+levelToString(resolvingLevel) + hint);
         }
     }

The problem is that even though Soot doesn't complain any longer and I 
get Jimple code out of Soot the Jimple looks different and behaves 
differently when run than what I get when I use the slower initialization.

So, how can I patch Soot to make lazy loading of required classes 
possible without changing the produced Jimple? I know too little yet 
about the Soot code to work this out on my own. Any help would be much 
appreciated.

Regards,
Niklas Therning