Using AdaptJ

Installation and Tutorial

Bruno Dufour

1  Overview

AdaptJ is a toolkit which serves two primary purposes:
These two main goals are accomplished through the use of two separate tools, the union of which composes the AdaptJ toolkit:

2  Installation

2.1  Installation Requirements

AdaptJ is distributed in various distributions:
Full distribution:
This distribution is the easiest way to install AdaptJ, and is strongly recommended. It contains the entire source code, binary distribution of the AdaptJ analysis tool and the full documentation.
Source distribution:
This distribution only contains the source code for both tools, and is the minimal distribution of AdaptJ.
AdaptJ requires Java 1.4 or later if you want support for Instruction Start events. Earlier versions of the JDK do also work, but will not send such events to the AdaptJ agent (Instruction Start events have been introduced by Sun in their Java 1.3 implementation, but it only seems to work properly for the Microsoft Windows version of the Java 1.3 SDK). AdaptJ has only been tested with Sun's Java implementation, although support for the upcoming IBM Java2 1.4 is planned.
To compile the AdaptJ Agent, the typical GNU tools (gcc and GNU make) are strongly recommended. You must also have a copy of 'zlib' installed on your system. Visit http://www.gzip.org/zlib/ if you don't. Most Linux systems have it installed by default (you can type 'locate libz' at your shell prompt to check).
Jakarta Ant is also recommended if you ever need to (re-)compile AdaptJ. It is not necessary for the full distribution of AdaptJ, but is strongly recommended for the source distribution. You can get a copy of the latest version of at http://jakarta.apache.org/ant/.
AdaptJ makes use of the following Java libraries (all of which are included in the full-distribution):
The following installation instructions assume that a UN*X system is used. There is currently no support for AdaptJ under Microsoft Windows, although only minor modifications should be required in order to get AdaptJ to run under a Windows OS.

2.2  Obtaining a copy of the latest AdaptJ distribution

If you have not already done so, get the latest copy of the AdaptJ distribution. AdaptJ is available from:
http://www.sable.mcgill.ca/~bdufou1/AdaptJ/
The full distribution is strongly recommended.
Unjar the archive file into a directory of your choice (which we will refer to as ${ADAPTJ_HOME} in the present instructions). This can be done using the following command (which should be executed from the directory in which you saved the AdaptJ distribution file). Note that in all shell commands presented in this document, '$' represents your shell prompt and is not part of the command:
$ jar -xvf <your-downloaded-archive.jar>

e.g.
$ jar -xvf AdaptJ-0.1beta2.jar

Then make the created subdirectory the current directory:
e.g.
$ cd AdaptJ-0.1beta2

Now, in order to allow typing the commands exactly as shown here, set the value of the ADAPTJ_HOME environment variable to point to your current directory by typing the following:
$ export ADAPTJ_HOME=`pwd`

If you do not want to perform the last step, remember to replace every occurence of the expression ${ADAPTJ_HOME} by the appropriate directory name throughout the installation process.

2.3  Installing the AdaptJ JVMPI Agent

The installation of the AdaptJ JVMPI Agent requires the compilation of the Agent itself, and is automated using GNU make.
First, make the AdaptJAgent source directory the current one by typing:
$ cd src/AdaptJAgent

Then, edit the file named 'Makefile' in the following way:
Locate the line which reads:
JAVA_HOME=/opt/j2sdk-1.4.1

and adjust the right-hand part of it to point to the root of your local java installation. To tell whether you have found to correct directory, try to locate a subdirectory named 'include'.
If you do not want to use the default C compiler (gcc), change the line which reads 'CC=gcc' to suit your needs.
Then, type 'make' at the prompt to compile the Agent and install it.
In order to get Java to find the agent you just compiled, you must add ${ADAPTJ_HOME}/ldlib to your LD_LIBRARY_PATH.
If you are using the 'bash' shell, this can be done by typing:
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${ADAPTJ_HOME}/ldlib

You may want to put the above line in your .bash_profile or .bashrc script to make the required change each time you login to your account.
Finally, because the command line that is needed to execute both tools can get tedious to type, tiny convenience scripts are provided in the ${ADAPTJ_HOME}/bin directory. You may want to add this directory to your PATH if you want to use them, like so:
$ export PATH=$PATH:${ADAPTJ_HOME}/bin

You should also make both scripts executable by typing:
$ chmod a+x ${ADAPTJ_HOME}/bin/*

since the jar tool does not preserve the executable bit.

2.4  Installation of the AdaptJ Trace Analyzer Tool

If you have the full distribution of AdaptJ, then you only need to add every .jar file in the ${ADAPTJ_HOME}/lib directory to your CLASSPATH.
Using the 'bash' shell, this is achieved as follows:
$ export CLASSPATH=$CLASSPATH:${ADAPTJ_HOME}/lib/AdaptJ.jar:\
${ADAPTJ_HOME}/lib/bcel.jar:${ADAPTJ_HOME}/lib/jakarta-regexp-1.2.jar:\
${ADAPTJ_HOME}/lib/fastUtil-2.11.jar

If you have the source distribution of AdaptJ, you need to compile it first. This can be automated using Jakarta Ant. From the ${ADAPTJ_HOME} directory, type:
$ ant makejar

This will create the ${ADAPTJ_HOME}/lib directory containing the file 'AdaptJ.jar'. Add it to your CLASSPATH in a way the same way as for the full distribution. The remaining libraries (see section 2.1) must be obtained and installed too.

3  Collecting Trace Data using the AdaptJ Agent

The AdaptJ Agent is a program that "hooks" into the Java Virtual Machine via the Java Virtual Machine Profiler Interface (JVMPI). It has the ability to record various types of events that occur within the JVM while a program is executing. Examples of events are the Method Entry event, which is triggered when a method starts executing in the running java program, or the Object Allocation event, which is triggered when the JVM creates a new instance of an object at runtime. Another program which uses the JVMPI is Sun's hprof profiler, which is distributed as part of Sun's Java SDK.
The AdaptJ Agent is designed to record such events and pass them to another program in a very simple and flexible way. The agent accepts a number of command line parameters that determine exactly which information gets recorded into the trace files.

3.1  Recording a Trace

To get a feel for the agent, let's try invoking it's help function. At your shell prompt, type the following command:
$ java -XrunAdaptJ:help

If you are using the AdaptJAgent script, then you can also type:
$ AdaptJAgent help

to achieve the same effect.
The AdaptJ agent should then respond with the following message:
AdaptJ Agent Options
===================

 file=<output_file>       Specifies the name of the output file.
                          Defaults to "AdaptJ.dat"
 events=<event_char>+     Specifies the events to be recorded
 counters=<event_char>+   Specifies the events to be counted
                          Usage "help=events" to get a list of event chars
 split=<size>             Specifies the threshold at which a new file is to be started
 gc=<size>                Specifies at which interval to force garbage collection
                          <size> is a number, optionally followed by one of 'k', 'm'
                          or 'g', which stand for 'kilobytes', 'megabytes' and
                          'gigabytes', respectively. THIS FEATURE IS CURRENTLY BROKEN.
 pipe                     Output data to a named piped instead of a regular file
 quiet                    Turns off agent messages
 opt                      Attempts to reduce the size of the trace file by using
                          compression techniques
 cp=<path1>+              Specifies the classpath to use for resolving classes
 cp+=<path1>+             Specifies the classpath to use for resolving classes.
                          The value of the environment variable CLASSPATH is append to
                          the specified list
 help[=events]            Prints this help message. If 'events' is specified, a list
                          of event chars is output

Now, let's try creating a simple trace file. First, let's create a small "Hello World"-like java program:
$ cat > AdaptJTest.java
public class AdaptJTest {
    public static void main(String[] args) {
        System.out.println("AdaptJ rocks!");
    }
}
<CTRL-D>
$ javac AdaptJTest.java

We now have to choose the events we want to record. Type:
$ java -XrunAdaptJ:help=events

AdaptJ should display the following message:
AdaptJ Event Mappings
====================

JVMPI_EVENT_ARENA_DELETE                    A
JVMPI_EVENT_ARENA_NEW                       a

JVMPI_EVENT_CLASS_LOAD                      c
JVMPI_EVENT_CLASS_LOAD_HOOK                 k
JVMPI_EVENT_CLASS_UNLOAD                    C

JVMPI_EVENT_COMPILED_METHOD_LOAD            l
JVMPI_EVENT_COMPILED_METHOD_UNLOAD          L

JVMPI_EVENT_DATA_DUMP_REQUEST               q
JVMPI_EVENT_DATA_RESET_REQUEST              Q

JVMPI_EVENT_GC_FINISH                       G
JVMPI_EVENT_GC_START                        g

JVMPI_EVENT_HEAP_DUMP                       h

JVMPI_EVENT_JNI_GLOBALREF_ALLOC             j
JVMPI_EVENT_JNI_GLOBALREF_FREE              J

JVMPI_EVENT_JNI_WEAK_GLOBALREF_ALLOC        w
JVMPI_EVENT_JNI_WEAK_GLOBALREF_FREE         W

JVMPI_EVENT_JVM_INIT_DONE                   v
JVMPI_EVENT_JVM_SHUT_DOWN                   V

JVMPI_EVENT_METHOD_ENTRY                    b
JVMPI_EVENT_METHOD_ENTRY2                   m
JVMPI_EVENT_METHOD_EXIT                     M

JVMPI_EVENT_MONITOR_CONTENDED_ENTER         d
JVMPI_EVENT_MONITOR_CONTENDED_ENTERED       e
JVMPI_EVENT_MONITOR_CONTENDED_EXIT          D

JVMPI_EVENT_MONITOR_DUMP                    x
JVMPI_EVENT_MONITOR_WAIT                    y
JVMPI_EVENT_MONITOR_WAITED                  Y

JVMPI_EVENT_OBJECT_ALLOC                    o
JVMPI_EVENT_OBJECT_DUMP                     p
JVMPI_EVENT_OBJECT_FREE                     O
JVMPI_EVENT_OBJECT_MOVE                     P

JVMPI_EVENT_RAW_MONITOR_CONTENDED_ENTER     r
JVMPI_EVENT_RAW_MONITOR_CONTENDED_ENTERED   E
JVMPI_EVENT_RAW_MONITOR_CONTENDED_EXIT      R

JVMPI_EVENT_THREAD_END                      T
JVMPI_EVENT_THREAD_START                    t

JVMPI_EVENT_INSTRUCTION_START               i

This is a list of all JVMPI Events, along with the mapping to a character which we can use to tell the agent to record the corresponding event. Say we want to record the Class Load events and the Method Entry Events. These correspond to 'c' and 'b', respectively. Therefore, we would type the following to invoke the agent:
$ java -XrunAdaptJ:events=cb AdaptJTest

This should be relatively fast. The output of the AdaptJ agent is reproduced below:
AdaptJ Agent Warning> No file name specified, using default: "AdaptJ.dat"
AdaptJ Agent> JVMPI_EVENT_CLASS_LOAD enabled
AdaptJ Agent> JVMPI_EVENT_METHOD_ENTRY enabled
AdaptJ Agent> Initialization completed
AdaptJ rocks!
AdaptJ Agent> Done

First, the agent tells you that you have not specified a file name, so it will output the event trace to 'AdaptJ.dat'. We are OK with this, so it does not matter for now. Then, it tells you that it has asked the JVM to send it the events you requested, and that it encountered no problem will doing so. Finally, you see the normal output of the program, followed by the last agent message, telling you that the agent has terminated normally.
To learn more about the information that is available for each event, the JVMPI documentation is a very valuable reference (see http://java.sun.com/j2se/1.4.1/docs/guide/jvmpi/jvmpi.html). A quick look at it will reveal that while the Method Entry event is a very simple event (it has only a single field: the jvmpi ID of the method being entered), the Class Load event is much more complex. It comprises 9 different fields, not all of which may be of interest to a given user:
class_name
The name of class being loaded.
source_name
The name of source file that defines the class.
num_interfaces
The number of interfaces implemented by this class.
methods
The methods defined in the class.
num_static_fields
The number of static fields defined in this class.
statics
The static fields defined in the class.
num_instance_fields
The number of instance fields defined in this class.
instances
The instance fields defined in the class.
class_id
The JVMPI class object ID.
The quick way of selecting events that we have seen so far will record all of the information for each of the selected events. While this coarse-grained selection of the recorded information may be useful, it is only rarely necessary. The major disadvantage of the technique has to do with the size of the resulting trace files. They can grow very quickly and end up in the range of the gigabytes for relatively small programs, depending on which events are selected (especially the Instruction Start event).
The AdaptJ Agent allows a more fine-grained control over the selection of the recorded information by allowing a selection of the individual fields which make up an event. This is achieved via the use of an AdaptJ Event Specification file, an example of which is reproduced below:
default: no;

event ClassLoad {
    recorded: yes;
    
    class_name: yes;
    methods: yes;
    class_id: yes;
}

event MethodEntry {
    recorded: yes;
    
    method_id: yes;
}

This file specifies that only the Class Load and Method Entry events should be recorded, as in the previous case. However, this time we explicitly state that we are only interested in the class_name, methods and class_id fields of the Class Load event. In order to be usable with the AdaptJ agent, this file has to be compiled to a compact binary representation of it, which is then included in the trace file so that the program reading it will know which information is available (In the previous case, the trace file also contained this binary specification, but it was generated automatically by the agent based on the selected events). Compiling the specification file is done with the adaptj.spec.Compiler program. Say we saved this file under the name AgentDemo.adaptj, then we would invoke the compiler by typing:
$ java adaptj.spec.Compiler AgentDemo.adaptj AgentDemo.spec

The resulting compiled file is output to AgentDemo.spec. It can then be used with the agent by specifying in the following way:
$ java -XrunAdaptJ:specFile=AgentDemo.spec AdaptJTest

which should execute the same way as before.
In the next section, we will look at how to analyze the trace files that the agent generates.

4  Analyzing Trace Data using AdaptJ Trace Analyzer

The AdaptJ Trace Analyzer is able to read the trace file generated by the agent and apply a sequence of operations to it. It is designed to minimize the amount of time and effort required to implement new analyses.

4.1  Overview

The mechanism that occurs behind the scenes in the analyzer is rather simple: as events are read from the trace, they are successively passed to a sequence of chained operations. In the simplest cases, these operations will only want to monitor the occurrence of the event and perform some computation. However, AdaptJ also allows each operation to modify the contents of an event or to replace it completely a different event. It also allows an operation to stop the processing chain, forcing the next event to be read from the trace and make its way through the chain starting from the beginning. In effect, this allows an operation to act as a filter for the subsequent operations in the chain. This level of flexibility is required in order to let operations provide services to other operations in the chain.
For example, the InstructionResolver operation adds information about the bytecode corresponding to an Instruction Start event, and thus modifies the content of the actual event object. The EventDistiller operation is able to filter out some events based on their associated class name by stopping the processing chain once it determines that an event should be excluded. The IDResolver operation is a very important operation that executes very early in the processing chain, which has as primary function to keep track of internal identifiers used by the JVMPI, and provides some utility functions for other operations to access this information.
In order to help organize the various operations and to facilitate their configuration, AdaptJ allows arbitrary groupings of operations under Packs. Packs are simply containers that can contain both other packs and operations. This allows the creation of a pack/operation hierarchy within the framework. The order in which operations are executed is determined from this hierarchical organization of the operations, and corresponds to a order in which the operations appear during a standard traversal of the tree.
Each pack/operation has a unique fully-qualified name, which is simply the concatenation of all parent packs of this pack/operation, followed by the name of the pack/operation, all separated by dots. So operation init.prn contained in the pack called init (which is itself a direct dependent of the root pack), and is named prn.

4.2  The Command-Line Interface

AdaptJ does not (yet) possess a graphical interface, so all of the interaction between users and the tool occurs from the command-line. Invoking the analyzer is simply done like so:
$ java adaptj.Main --help

or
$ AdaptJAnalyzer --help

if you are using the AdaptJAnalyzer script.
AdaptJ should then respond with the following message:
AdaptJ version 0.1 Beta

Written by Bruno Dufour
Contact: bruno.dufour@mail.mcgill.ca
AdaptJ homepage: http://www.sable.mcgill.ca/~bdufou1/AdaptJ/

Usage:
    java adaptj.Main [options] <trace file>

Option                                            Description
-----------------------------------------------   ------------------------------
-q <quiet mode>, --quiet <quiet mode>             Sets the quiet mode:
                                                    Any combination of:
                                                      m or M: Hide messages
                                                      w or W: Hide warnings
                                                      e or E: Hide errors
                                                      0: Show everything
                                                      1: same as M
                                                      2: same as MW
                                                      3: same as MWE)
-v, --verbose                                     Enables verbose mode
--version                                         Prints the version and exits
--no-gui                                          Disables the graphical
                                                  progress indicator
-x, --all-off                                     Turns off all operations
-f <file>, --file <file>                          Reads arguments from a file
                                                  (Other arguments can be given)
-r <rate>, --refresh-rate <rate>                  Sets the amount of time
                                                  between GUI updates (in msec)
--pipe                                            Forces to read input as if
                                                  from a pipe (no preparsing)
--show-hierarchy                                  Displays the package/operation
                                                  hierarchy
-t, --time                                        Prints time statistics
--showver                                         Prints the version and
                                                  continues
-p <pack/operation> <option[:value]>              Sets option for a
                                                  pack/operation
-cp <operation> <paths>, --classpath              Manipulates the classpath used
<operation> <paths>                               by AdaptJ
-h [package/option], --help [package/option]      Prints help and exits

Copyright (C) 2002 Sable Research Group
(http://www.sable.mcgill.ca/)

AdaptJ comes with ABSOLUTELY NO WARRANTY.  AdaptJ is free software,
and you are welcome to redistribute it under certain conditions.
See the accompanying file 'license.html' for details.

You get a list of command-line options that you can use to control AdaptJ. The same list can be obtained by invoking AdaptJ with the -h or --help options. Now, let's query AdaptJ and determine what is available in the standard AdaptJ operation library. The the following at your prompt:
$ java adaptj.Main --show-hierarchy

You should then see the following response:
RootPack
 \_ init
    \_ prn
 \_ tp
    \_ IDResolver
    \_ ClassNameResolver
    \_ InstructionResolver
    \_ ED
 \_ mp
    \_ BM
    \_ MM
    \_ PM
    \_ PSM
    \_ SM
    \_ FPM
    \_ IMM
 \_ ap
 \_ pp
    \_ prn
    \_ eprn
    \_ mprn

From this ASCII art representation of the hierarchy, you can note that there are 5 predefined packs in this hierarchy: init, tp (for Transformation Pack), mp (for Metrics Pack), ap (for Analysis Pack) and pp (for Printers Pack). In order to find out information about a particular pack or operation, invoke AdaptJ with the optional parameter to the --help option. This argument must the fully-qualified name of the desired pack/operation. For example, typing:
$ java adaptj.Main --help init.prn

will display:
Pack/Operation help for "init.prn"

Description: Prints events as they enter the processing chain

Option                      Description
-------------------------   ----------------------------------------------------
enabled[:<boolean>]         Controls whether this pack/operation is active or
                            not
verbose[:<boolean>]         Controls whether this pack/operation displays
                            extended information or not
timed[:boolean]             Controls whether this operation is timed
file:<file>                 Specifies the name of a file to write the output to

This represents a list of the options that this operation supports. Options can be passed to packs or operations using the -p command-line option. For example:
$ java adaptj.Main -p init.prn enabled -p init.prn file:MyFile.txt AdaptJ.dat

will turn on the default printer in the init pack and instruct it to dump its output to the file Myfile.txt. Try it out and look at the output to see what you have recorded in the previous section.
Another very important option in AdaptJ is the -cp option, which is used to manipulate the class path that the tool uses to locate the classes. By default, AdaptJ will use the class path from the java.class.path property. If you want to add a an entry to the class path, you can use the -cp add <entry>[:<entry>]* command to do so. For example:
$ java adaptj.Main -cp add classes AdaptJ.dat

If AdaptJ cannot find a class that it absolutely requires, it will throw an exception and ask you to update its class path.

4.3  Dependencies

AdaptJ requires every operation to explicitly state two kinds of dependencies: event dependencies and operation dependencies. Event dependencies are used to state which events (and their respective fields) the operation absolutely requires in order to be able to perform its task, and which ones it would like to receive if they are available. These are called optional event dependencies. This is easy to check since every trace file that the AdaptJ agent includes a description of the events (and their respective fields) which have been enabled when producing the trace. For example, the InstructionResolver cannot execute its task if it does not receive Instruction Start events, which is thus an event dependency for it. On the other hand, a printer can perform its task even if an event is not available, and thus only has optional event dependencies. Event dependencies allows to improve the processing speed of the tool by avoiding costly event dispatching to operations that do not care about certain types of events.
Operation dependencies specifies which operations in the AdaptJ toolkit provide services for a given application. If one of these operations is disabled for any reason, then all operations that depend on it must also be disabled, since it would be impossible for them to perform their task. The InstructionResolver and IDResolver operations are the two most frequent operation dependencies.

4.4  Computing Metrics

One primary goal of the AdaptJ toolkit (and the main reason it exists) is to compute dynamic software metrics (see http://www.sable.mcgill.ca/metrics/ for more information). Various software metrics are already implemented within AdaptJ, and more are on their way. Computing metrics is easily achieved using two basic steps:
  1. Generate an event trace using the AdaptJ.spec event spec file that came with your AdaptJ distribution (or a superset of it).

  2. Invoke AdaptJ in the following way:
        $ java adaptj.Main -p mp enabled,xmlfile:Metrics.xml AdaptJ.dat
        
    
    where AdaptJ.dat is the name of your previously collected trace file.
    The generated XML file declares its own XSL stylesheet URL so that you can directly view it in browsers that do support this feature (note that the file metrics.xsl should be located in the same directory as the XML file. You can find it under the directory ${ADAPTJ_HOME}/xslt or online at http://www.sable.mcgill.ca/metrics/xsl/metrics.xsl).
    To convert the XML file to its HTML representation, you can also use a tool like Apache Xalan-J (http://xml.apache.org/xalan-j/index.html) to convert it statically and display it in any browser.

5  Tips and Tricks

5.1  Using UN*X Named Pipes

Because the size of the trace files can get large very quickly, it is very common to run out of disk space using AdaptJ. In that case, add the pipe option to the ones you pass to the agent. It will create a named pipe (essentially a FIFO special file, so that the information is not written to the disk but sent via a pipe) instead of a regular file. Note that the AdaptJ Analyzer tool needs to be executed simultaneously with the agent in order to read from the pipe as information becomes available.
The AdaptJ tool automatically detects that a file corresponds to a named, and will skip the preparsing phase. Tip: If you have a large file that you would like to process without first preparsing it, you can force AdaptJ to pretend that the file is a FIFO file by giving it the --pipe option.


File translated from TEX by TTH, version 3.30.
On 16 Jan 2003, 00:13.