[Soot-list] Possible issue with dex files
Gianluca Mezzetti
mezzetti at cs.au.dk
Wed Sep 10 05:32:36 EDT 2014
Hi Steven,
I made a github repository to work on android framework without too much pain
https://github.com/algobardo/sootReproduce
Just type "gradle run", android classes are there
Il giorno 10/set/2014, alle ore 09:53, Gianluca Mezzetti <mezzetti at cs.au.dk<mailto:mezzetti at cs.au.dk>> ha scritto:
Hi Steven,
I'm opening the issues regarding classes now...
For dex files it is harder to reproduce the problem, because you have to setup your instrumented machine, i will delay that issues, so that I can investigate them a bit further.
About dexes: according to
http://www.netmite.com/android/mydroid/dalvik/docs/dexopt.html
and also looking at the makefile, the framework is not odexed before shipped into the machine.
The odexing is also a "linking" phase, so it seems that they are starting a minimal dalvik, that run DexOpt and then dump the file (i may be wrong) at runtime.
Therefore, i have no idea where is the problem... do you know if the dex jni rely on some ordering or something in the dex ?
Il giorno 10/set/2014, alle ore 09:25, Steven Arzt <Steven.Arzt at cased.de<mailto:Steven.Arzt at cased.de>> ha scritto:
Hi Gianluca,
Thank you for your really long issue report. I would be great if you could open issue reports on Github for the individual problems you have encountered. We are currently pushing for a new Soot release and are looking into the bugs we currently have in the tracker. Please provide the exact dex/apk files on which you are experiencing the issues so that we can reproduce them.
If have to admit that we never tried to instrument the Android framework as such, though. We only went for applications so far. I’m curious whether you were running Soot on “normal” dex files or “odexed” ones which might be a totally different story. As far as I know, much of the Android framework is odexed for performance reasons.
For instrumenting the class files: This should not be necessary since we support the dex format (maybe you run into issues due to odex-ing if you go for the platform itself, I’m not sure about that). Nevertheless, it should work as all Android-specific part is taken care of by “dx” later, so all that Soot needs to produce is valid Java class files.
Best regards,
Steven
Von: soot-list-bounces at CS.McGill.CA<mailto:soot-list-bounces at CS.McGill.CA> [mailto:soot-list-bounces at CS.McGill.CA] Im Auftrag von Gianluca Mezzetti
Gesendet: Dienstag, 9. September 2014 22:05
An: soot-list at cs.mcgill.ca<mailto:soot-list at cs.mcgill.ca>
Betreff: Re: [Soot-list] Possible issue with dex files
Hi, the email is quite old, unfortunately only today it get accepted into the mailing list.
I have a "tons" of new issues, that i hope you can help me to troubleshot.
I'm trying to instrument the entire/part of the android framework and getting it into a running emulator.
I understand that this is a quite hard task for a tool that convert everything to jimple and back, but eventually, I hope to be able to instrument at least part of it.
I tried different strategies, starting with an empty instrumentation (loading and writing back only)
First strategy: instrument class files.
As far as I have seen there are three kinds of problem for doing this:
- First: the bb.lp phase is broken.
You can find a """"minimal"""" counterexample in the PropertyValuesHolder.java class of the android framework, that has the following shape
private Method getPropertyFunction(Class targetClass, String prefix, Class valueType) {
// TODO: faster implementation...
Method returnVal = null;
String methodName = "some name";
Class args[] = null;
args = new Class[3];
while(true) {
args[0] = null;
try {
returnVal = targetClass.getMethod(methodName, args);
// change the value type to suit
mValueType = null;
return returnVal;
} catch (NoSuchMethodException e) {
// Swallow the error and keep trying other variants
}
}
}
the bb.lp phase remove a necessary register storing the return of getMethod(methodName, args); probably not getting right the flow with the exception.
I will provide you more detail opening an issue on github.
- Second: annotations
There is something different in the format of the field descriptors that dalvik accepts.
I will give you more detail opening an issue on github.
This is the problem I was referring to in the previous e-mail
- Third: other not investigated issues
- Right now soot crash on MathUtils class
- provide a bad bytecode on other few classes.
All the bugs that i have listed here was found by the dx tool, because it contains a nice value aware simulator.
It would be nice, to include in the soot tests the framework itself, and use the dx compiler in the framework as "oracle" to check that everything is fine.
After 2 days literally drained by the investigation of these issues i decided to try with the second strategy
Second strategy: instrument dex files
I'm having bad days with dex too.
It is working fine with applications, but when I come to the framework....aaarhg.
- First problem:
The dexes of the framework are splitted in two.
I managed to instrument them and getting them right...but when they are moved inside the emulator the DexOpt complains about the fact that one of the two reference classes that are in the other.
I checked, and I have found that even the non-instrumented dexes are referencing each-other....so I didn't understand what is the difference here.
I have investigated, without success some possible causes:
- maybe DexOpt is not ran on the original dx-ed framework because dx already optimize them: false
Because Running dx with --disable-optimize doesn't cause any problem when not instrumenting
There is not such flag in the dex header (right)
do you have any hints ?
Then i said to me, ok...let's try something simpler, let's try the "single dexed" services.
I had no luck with that, the device is not booting, even though DexOpt seems happy.
Do you know how does Jni works with dex-es....? It is possible that soot interferee with ...something about jni.
I'm really out of ideas right now....
I think that eventually I could try to solve some of these issues, but right now they are too much, and i don't know the soot code-base enough to attach them.
Moreover, experimenting with dex...really drain the time, because you have to compile the entire framework (4 minute), start the emulator, cross the finger, read the logcat......
Do you have any ideas, hints ?
Is there any effort (people) in getting this to work ?
Thanks.
Il giorno 03/set/2014, alle ore 13:56, Gianluca Mezzetti <mezzetti at cs.au.dk<mailto:mezzetti at cs.au.dk>> ha scritto:
Dear sooters,
i'm new to this community, and, first of all, i would like to say thanks to you for developing this cool tool.
However i have some problems that I would like to discuss with you.
I have a setting where i need to instrument single dex files, because of this I modified src/soot/SourceLocator.java
switch( Options.v().src_prec() ) {
case Options.src_prec_class:
classProviders.add(classFileClassProvider);
+ classProviders.add(new DexClassProvider());
classProviders.add(new JimpleClassProvider());
classProviders.add(new JavaClassProvider());
break;
With this setting i'm able to read and write dex files without the need of an android.jar, relaying on the phantom-ref feature.
However, when I try to install the dex i have problems with dexopt, that seems to be originated exactly from the missing jar.
It is the android.jar really needed to compile to dex files ?
The detail of the dexopt error are the following:
Invalid name : 'Ljava/lang/annotation/ElementType;'
raised in DexSwapVerify.cpp
s = dexStringById(state->pDexFile, item->nameIdx);
if (!dexIsValidMemberName(s)) {
ALOGE("Invalid name: '%s'", s);
return NULL;
}
to me it seems really not a valid member name, example of member names are the following:
E/dalvikvm( 2067): Valid name example: 'plugin'
E/dalvikvm( 2067): Valid name example: 'toLink'
E/dalvikvm( 2067): Valid name example: 'this$1'
E/dalvikvm( 2067): Valid name example: 'val$classLoader'
E/dalvikvm( 2067): Valid name example: 'this$0'
E/dalvikvm( 2067): Valid name example: 'caches'
E/dalvikvm( 2067): Valid name example: 'map'
where is that spurious name coming from ?
It is an annotation....is it that relevant ?
If you give me some hints about where I could take a look, i could investigate the issue deeper.
Thanks a lot.
Gianluca Mezzetti
Post-doc at Aarhus University, Aarhus, Denmark
_______________________________________________
Soot-list mailing list
Soot-list at CS.McGill.CA<mailto:Soot-list at CS.McGill.CA>
https://mailman.CS.McGill.CA/mailman/listinfo/soot-list<https://mailman.cs.mcgill.ca/mailman/listinfo/soot-list>
_______________________________________________
Soot-list mailing list
Soot-list at CS.McGill.CA<mailto:Soot-list at CS.McGill.CA>
https://mailman.CS.McGill.CA/mailman/listinfo/soot-list
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.CS.McGill.CA/pipermail/soot-list/attachments/20140910/0c0db069/attachment-0001.html
More information about the Soot-list
mailing list