[Soot-list] Possible issue with dex files

Wed Sep 10 03:25:39 EDT 2014

Hi Gianluca,

Thank you for your really long issue report. I would be great if you could
open issue reports on Github for the individual problems you have
encountered. We are currently pushing for a new Soot release and are looking
into the bugs we currently have in the tracker. Please provide the exact
dex/apk files on which you are experiencing the issues so that we can
reproduce them.

If have to admit that we never tried to instrument the Android framework as
such, though. We only went for applications so far. I'm curious whether you
were running Soot on "normal" dex files or "odexed" ones which might be a
totally different story. As far as I know, much of the Android framework is
odexed for performance reasons.

For instrumenting the class files: This should not be necessary since we
support the dex format (maybe you run into issues due to odex-ing if you go
for the platform itself, I'm not sure about that). Nevertheless, it should
work as all Android-specific part is taken care of by "dx" later, so all
that Soot needs to produce is valid Java class files.

Best regards,

  Steven

Von: soot-list-bounces at CS.McGill.CA [mailto:soot-list-bounces at CS.McGill.CA]
Im Auftrag von Gianluca Mezzetti
Gesendet: Dienstag, 9. September 2014 22:05
An: soot-list at cs.mcgill.ca
Betreff: Re: [Soot-list] Possible issue with dex files

Hi, the email is quite old, unfortunately only today it get accepted into
the mailing list.

I have a "tons" of new issues, that i hope you can help me to troubleshot.

I'm trying to instrument the entire/part of the android framework and
getting it into a running emulator.

I understand that this is a quite hard task for a tool that convert
everything to jimple and back, but eventually, I hope to be able to
instrument at least part of it.

I tried different strategies, starting with an empty instrumentation
(loading and writing back only)

First strategy: instrument class files.

As far as I have seen there are three kinds of problem for doing this:

- First: the bb.lp phase is broken.

You can find a """"minimal"""" counterexample in the
PropertyValuesHolder.java class of the android framework, that has the
following shape

private Method getPropertyFunction(Class targetClass, String prefix, Class
valueType) {

        // TODO: faster implementation...

        Method returnVal = null;

        String methodName = "some name";

        Class args[] = null;

        args = new Class[3];

            while(true) {

                args[0] = null;

                try {

                    returnVal = targetClass.getMethod(methodName, args);

                    // change the value type to suit

                    mValueType = null;

                    return returnVal;

                } catch (NoSuchMethodException e) {

                    // Swallow the error and keep trying other variants

                }

            }

    }

the bb.lp phase remove a necessary register storing the return of
getMethod(methodName, args); probably not getting right the flow with the
exception.

I will provide you more detail opening an issue on github.

- Second: annotations

There is something different in the format of the field descriptors that
dalvik accepts.

I will give you more detail opening an issue on github.

This is the problem I was referring to in the previous e-mail

- Third: other not investigated issues

- Right now soot crash on MathUtils class

- provide a bad bytecode on other few classes.

All the bugs that i have listed here was found by the dx tool, because it
contains a nice value aware simulator.

It would be nice, to include in the soot tests the framework itself, and use
the dx compiler  in the framework as "oracle" to check that everything is
fine.

After 2 days literally drained by the investigation of these issues i
decided to try with the second strategy

Second strategy: instrument dex files

I'm having bad days with dex too.

It is working fine with applications, but when I come to the
framework....aaarhg.

- First problem:

The dexes of the framework are splitted in two.

I managed to instrument them and getting them right...but when they are
moved inside the emulator the DexOpt complains about the fact that one of
the two reference classes that are in the other.

I checked, and I have found that even the non-instrumented dexes are
referencing each-other....so I didn't understand what is the difference
here.

I have investigated, without success some possible causes:

- maybe DexOpt is not ran on the original dx-ed framework because dx already
optimize them: false

Because Running dx with --disable-optimize doesn't cause any problem when
not instrumenting

There is not such flag in the dex header (right)

do you have any hints ?

Then i said to me, ok...let's try something simpler, let's try the "single
dexed" services.

I had no luck with that, the device is not booting, even though DexOpt seems
happy.

Do you know how does Jni works with dex-es....? It is possible that soot
interferee with ...something about jni.

I'm really out of ideas right now....

I think that eventually I could try to solve some of these issues, but right
now they are too much, and i don't know the soot code-base enough to attach
them.

Moreover, experimenting with dex...really drain the time, because you have
to compile the entire framework (4 minute), start the emulator, cross the
finger, read the logcat......

Do you have any ideas, hints ?

Is there any effort (people) in getting this to work ?

Thanks.

Il giorno 03/set/2014, alle ore 13:56, Gianluca Mezzetti <mezzetti at cs.au.dk>
ha scritto:

Dear sooters,
i'm new to this community, and, first of all, i would like to say thanks to
you for developing this cool tool.
However i have some problems that I would like to discuss with you.

I have a setting where i need to instrument single dex files, because of
this I modified  src/soot/SourceLocator.java 

switch( Options.v().src_prec() ) {
            case Options.src_prec_class:
                classProviders.add(classFileClassProvider);
+                classProviders.add(new DexClassProvider());
                classProviders.add(new JimpleClassProvider());
                classProviders.add(new JavaClassProvider());
                break;

With this setting i'm able to read and write dex files without the need of
an android.jar, relaying on the phantom-ref feature.
However, when I try to install the dex i have problems with dexopt, that
seems to be originated exactly from the missing jar.
It is the android.jar really needed to compile to dex files ?

The detail of the dexopt error are the following:
Invalid name : 'Ljava/lang/annotation/ElementType;'

raised in DexSwapVerify.cpp 
 s = dexStringById(state->pDexFile, item->nameIdx);
   if (!dexIsValidMemberName(s)) {
       ALOGE("Invalid name: '%s'", s);
       return NULL;
   }

to me it seems really not a valid member name, example of member names are
the following:

E/dalvikvm( 2067): Valid name example: 'plugin'
E/dalvikvm( 2067): Valid name example: 'toLink'
E/dalvikvm( 2067): Valid name example: 'this$1'
E/dalvikvm( 2067): Valid name example: 'val$classLoader'
E/dalvikvm( 2067): Valid name example: 'this$0'
E/dalvikvm( 2067): Valid name example: 'caches'
E/dalvikvm( 2067): Valid name example: 'map'

where is that spurious name coming from ? 
It is an annotation....is it that relevant ?
If you give me some hints about where I could take a look, i could
investigate the issue deeper.

Thanks a lot.

Gianluca Mezzetti
Post-doc at Aarhus University, Aarhus, Denmark
_______________________________________________
Soot-list mailing list
 <mailto:Soot-list at CS.McGill.CA> Soot-list at CS.McGill.CA
 <https://mailman.CS.McGill.CA/mailman/listinfo/soot-list>
https://mailman.CS.McGill.CA/mailman/listinfo/soot-list

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.CS.McGill.CA/pipermail/soot-list/attachments/20140910/a707bb4f/attachment-0001.html 