[Soot-list] Possible issue with dex files

Tue Sep 9 16:04:35 EDT 2014

Hi, the email is quite old, unfortunately only today it get accepted into the mailing list.
I have a "tons" of new issues, that i hope you can help me to troubleshot.
I'm trying to instrument the entire/part of the android framework and getting it into a running emulator.
I understand that this is a quite hard task for a tool that convert everything to jimple and back, but eventually, I hope to be able to instrument at least part of it.

I tried different strategies, starting with an empty instrumentation (loading and writing back only)

First strategy: instrument class files.
As far as I have seen there are three kinds of problem for doing this:

- First: the bb.lp phase is broken.
You can find a """"minimal"""" counterexample in the PropertyValuesHolder.java class of the android framework, that has the following shape

private Method getPropertyFunction(Class targetClass, String prefix, Class valueType) {
        // TODO: faster implementation...
        Method returnVal = null;
        String methodName = "some name";
        Class args[] = null;

        args = new Class[3];
            while(true) {
                args[0] = null;
                try {
                    returnVal = targetClass.getMethod(methodName, args);
                    // change the value type to suit
                    mValueType = null;
                    return returnVal;
                } catch (NoSuchMethodException e) {
                    // Swallow the error and keep trying other variants
                }
            }
    }

the bb.lp phase remove a necessary register storing the return of getMethod(methodName, args); probably not getting right the flow with the exception.
I will provide you more detail opening an issue on github.

- Second: annotations
There is something different in the format of the field descriptors that dalvik accepts.
I will give you more detail opening an issue on github.
This is the problem I was referring to in the previous e-mail

- Third: other not investigated issues
- Right now soot crash on MathUtils class
- provide a bad bytecode on other few classes.

All the bugs that i have listed here was found by the dx tool, because it contains a nice value aware simulator.
It would be nice, to include in the soot tests the framework itself, and use the dx compiler  in the framework as "oracle" to check that everything is fine.

After 2 days literally drained by the investigation of these issues i decided to try with the second strategy

Second strategy: instrument dex files

I'm having bad days with dex too.
It is working fine with applications, but when I come to the framework....aaarhg.

- First problem:
The dexes of the framework are splitted in two.
I managed to instrument them and getting them right...but when they are moved inside the emulator the DexOpt complains about the fact that one of the two reference classes that are in the other.
I checked, and I have found that even the non-instrumented dexes are referencing each-other....so I didn't understand what is the difference here.
I have investigated, without success some possible causes:
- maybe DexOpt is not ran on the original dx-ed framework because dx already optimize them: false
Because Running dx with --disable-optimize doesn't cause any problem when not instrumenting
There is not such flag in the dex header (right)
do you have any hints ?

Then i said to me, ok...let's try something simpler, let's try the "single dexed" services.
I had no luck with that, the device is not booting, even though DexOpt seems happy.
Do you know how does Jni works with dex-es....? It is possible that soot interferee with ...something about jni.

I'm really out of ideas right now....
I think that eventually I could try to solve some of these issues, but right now they are too much, and i don't know the soot code-base enough to attach them.
Moreover, experimenting with dex...really drain the time, because you have to compile the entire framework (4 minute), start the emulator, cross the finger, read the logcat......

Do you have any ideas, hints ?
Is there any effort (people) in getting this to work ?
Thanks.

Il giorno 03/set/2014, alle ore 13:56, Gianluca Mezzetti <mezzetti at cs.au.dk<mailto:mezzetti at cs.au.dk>> ha scritto:

Dear sooters,
i'm new to this community, and, first of all, i would like to say thanks to you for developing this cool tool.
However i have some problems that I would like to discuss with you.

I have a setting where i need to instrument single dex files, because of this I modified  src/soot/SourceLocator.java

switch( Options.v().src_prec() ) {
            case Options.src_prec_class:
                classProviders.add(classFileClassProvider);
+                classProviders.add(new DexClassProvider());
                classProviders.add(new JimpleClassProvider());
                classProviders.add(new JavaClassProvider());
                break;

With this setting i'm able to read and write dex files without the need of an android.jar, relaying on the phantom-ref feature.
However, when I try to install the dex i have problems with dexopt, that seems to be originated exactly from the missing jar.
It is the android.jar really needed to compile to dex files ?

The detail of the dexopt error are the following:
Invalid name : 'Ljava/lang/annotation/ElementType;'

raised in DexSwapVerify.cpp
 s = dexStringById(state->pDexFile, item->nameIdx);
   if (!dexIsValidMemberName(s)) {
       ALOGE("Invalid name: '%s'", s);
       return NULL;
   }

to me it seems really not a valid member name, example of member names are the following:

E/dalvikvm( 2067): Valid name example: 'plugin'
E/dalvikvm( 2067): Valid name example: 'toLink'
E/dalvikvm( 2067): Valid name example: 'this$1'
E/dalvikvm( 2067): Valid name example: 'val$classLoader'
E/dalvikvm( 2067): Valid name example: 'this$0'
E/dalvikvm( 2067): Valid name example: 'caches'
E/dalvikvm( 2067): Valid name example: 'map'

where is that spurious name coming from ?
It is an annotation....is it that relevant ?
If you give me some hints about where I could take a look, i could investigate the issue deeper.

Thanks a lot.

Gianluca Mezzetti
Post-doc at Aarhus University, Aarhus, Denmark
_______________________________________________
Soot-list mailing list
Soot-list at CS.McGill.CA<mailto:Soot-list at CS.McGill.CA>
https://mailman.CS.McGill.CA/mailman/listinfo/soot-list

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.CS.McGill.CA/pipermail/soot-list/attachments/20140909/96824ee9/attachment.html 