October 26th, 2010 by Dominique Gallot
First, why do you want to force the garbage collection? According to me, you probably never need to do it !
So we should never do it, but then, why do I implement it ? Well, in my case , it makes sense 
Ok, the title of the article is misleading you. I am not really able to force the garbage collection, but I am able to detect when the garbage collection works.
At work, we are developing an application server. It allows to dynamically add applications, and of course also allows to uninstall it, more or less as any other application server.
But even for this, you do not need to force the garbage collection; yes and no.
The problems come when you are installing code requiring some JNI libraries.
JNI implementation has some limitations that makes the loaded DLL highly linked with the current Class Loader.
It means that if you do the following steps :
- Deploy an application which uses a dll
- Undeploy this application
- And deploy it again
You will get a UnsatisfiedLinkError with the message : Native Library already loaded in another classloader.
This is weird (IMHO) ! Once a DLL has been loaded, you cannot load it until the ClassLoader object has been reclaimed by the garbage collection.
Let’s check the JVM code to see how it is working.
First when you load a library, the loadLibrary code checks if the dll is not already loaded.
Let’s check the code of java.lang.ClassLoader.loadLibrary code.
private static boolean loadLibrary0(Class fromClass, final File file) {
…
synchronized (loadedLibraryNames) {
if (loadedLibraryNames.contains(name)) {
throw new UnsatisfiedLinkError ("Native Library " +
name + " already loaded in another classloader");
};
…
}
As you can see, it is checking the list of loaded libraries, if the library is already associated with an otherclassloader. If it is, the load will fail.
Only when the ClassLoader.finalize is called, you will be able to load the library again.
protected void finalize() {
...
for (int i = 0; i < size; i++) {
if (name.equals(loadedLibraryNames.elementAt(i))) {
loadedLibraryNames.removeElementAt(i);
break;
}
...
}
Then, how can we force the garbage collection to allow the finalized to be run, or at least how do we know when the class loader has been completely finalized.
During my experiments, I try just to call the method “System.gc”
It does work, but it is unpredictable. I sometimes need to call it twice, or even more. It is really not easy to be sure that the garbage collection has been executed.
I therefore developed this small piece of code:
public static void waitGc( long timeout ) {
final boolean[] wait = new boolean[1];
@SuppressWarnings("unused")
// create a small graph of object
Object graph = new Object() {
private Object leaf = new Object() {
protected void finalize() throws Throwable {
synchronized( wait ) {
logger.finest("leaf finalize called");
wait[0] = true;
wait.notify();
}
}
};
protected void finalize() throws Throwable {
logger.finest("graph finalize called");
}
};
long start = System.currentTimeMillis();
graph = null;
while ( System.currentTimeMillis() < start + timeout ) {
// defer a little bit the gc
try {
if ( wait[0] ) {
logger.finest("finalize Called on object graph, gc has been executed");
break;
}
System.gc();
System.runFinalization();
logger.finest("finalize not (yet) called");
synchronized( wait ) {
wait.wait(timeout / 10);
}
} catch (InterruptedException e) {
// do not care
}
}
}
The code is not really easy to understand. Here is how it works.
First I create an object graph, two objects which hold a reference to another object.
These objects have been singly modified to notify when the object are finalized.
The rest of the code is a simple loop, which runs System.gc and System.runFinization until it receives the notification that the object graph has been finalized.
For more security, the code does not wait indefinitely, but only for a predefined time.
Inside the container we do two things:
- When a application is undeployed, we check if it has loaded some dynamic library
- If not, we just undeploy it. If yes, we wait until the ClassLoader has been finalized using the same technique than above.
Even if the System.gc does nothing, we will be notified that the ClassLoader is not yet finalized, and return to the user a message stating that the component has been undeployed. But for some reasons, which will probably a memory leak on the application, the loaded dynamic library is not released yet.
Do not forget that there are some jvm options which disable the System.gc() calls.
-XX:-DisableExplicitGCDisable calls to System.gc(), JVM still performs garbage collection when necessary.
In a next article
I will explain how to dig into the native library info, which is not available by any public way. In that purpose, I will use a class that you probably do not know, sun.misc.Unsafe the modern java peek and poke library!
April 6th, 2010 by Dominique Gallot
I was getting a blank page while trying to reach my vmware server web ui, with some really strange java script error, since few days.
I was also getting some error page in the log file of vmare-mui ( /var/log/vmware-mui )
[error] Failed to connect. Reason = (11) 501 Global command GLOBAL server-vmdb to non-host agent targets not supported.\n
After upgrading a new version of vmware server (2.0.2), the problem was still occurring. Really frustrating !
I finally made it working.
Basically it is an incompatibility between the web ui of vmware server during the ssl 3.0 handshake, which is crashing the server.
The javascript errors was coming from partially loaded files.And due to the watchdog which is protecting the webserver, I never really notice that the server was crashing really often !
The solution is to activate the ssl 2.0 on your firefox. To do so, put about:config in the url, pass the warning page.
And activate the ssl 2.0 by changing the flags
security.enable_ssl2 : true
security.enable_ssl3 : false
about:config
March 23rd, 2010 by Dominique Gallot
While driving back home, I was thinking about a use for the Instrumentation of java, and I thought of implementing the __LINE__ macro like in c.
Instrumentation what is that ? An instrumentation service is a way of modifying the loaded java bytes code on the fly, before it really exists in the jvm. The main usages of services are to implements memory profiling, code profiling and most ( all ? ) AOP frameworks.
__LINE__ __FILE__ for java
As you know, as a java developer you do not have access to a preprocessor. Most of the time, you can live with it. But when it comes to logging, it is another thing. If you are like me, most of the time you want to print with your log statement, with the classname/file name which issued the log statement and also want to see the line number.
Good logging framework provides this, but how it is fetching the information is amazingly slow.
When the logging framework needs to locate the location of the log statement, it needs to get a snapshoot of the calling stack, then needs to guess who is the caller, by carefully examining it.
Let’s examine the code which does this in the logging framework provided with the jdk ( java.util.logging.* )
// Private method to infer the caller's class and method names
private void inferCaller() {
needToInferCaller = false;
// Get the stack trace.
StackTraceElement stack[] = (new Throwable()).getStackTrace();
// First, search back to a method in the Logger class.
int ix = 0;
while (ix < stack.length) {
StackTraceElement frame = stack[ix];
String cname = frame.getClassName();
if (cname.equals("java.util.logging.Logger")) {
break;
}
ix++;
}
// Now search for the first frame before the "Logger" class.
while (ix < stack.length) {
StackTraceElement frame = stack[ix];
String cname = frame.getClassName();
if (!cname.equals("java.util.logging.Logger")) {
// We've found the relevant frame.
setSourceClassName(cname);
setSourceMethodName(frame.getMethodName());
// I do not understand why they do not try to load the getLineNumber
return;
}
ix++;
}
// We haven't found a suitable frame, so just punt. This is
// OK as we are only committed to making a "best effort" here.
}
A lot of code just to detect the caller, isn'it ? And even more it does not work all the time! In the following example, the line number will be improperly guessed!
public class Test {
Logger logger = Logger.getLogger( Test.class.getName() );
public void main() {
error( “Error in main” ); <-- I want this line
}
private void error( String message ) {
logger.sever( message ) <-- I wll be getting this one
}
}
You are basically not allowed to pass by any other method before logging!
My solution
Is to define some static fields which will be returning the __FILE__ ...
And behind the scene I use the instrumentation, byte code manipulation, and the available debug information to dynamically add the correct file/line number.
Here is how to use it
package com.dg.test;
import static com.dg.ab.use.instument.CodeLocation.__FILE__;
import static com.dg.ab.use.instument.CodeLocation.__LINE__;
import static com.dg.ab.use.instument.CodeLocation.__FILE_LINE__;
public class CodeLocationTest {
public void run() {
System.err.println( __FILE_LINE__ );
System.err.println( __FILE_LINE__ + ": Entering in run method" );
System.err.println( __FILE_LINE__ + ": Entering in run method" );
for ( int i = 0; i < 10; i++ ) {
System.err.println( __FILE__ + ":" + __LINE__ + " value of i is '"+i+"'");
}
}
public static void main( String args[] ) {
new Test().run();
}
}
Thanks to the static import, it really looks like c code isn’t it ? And when running it with the appropriate agent ( adding -javaagent:CodeLocationAgent.jar to you java parameter), the __LINE__, __FILE__, __FILE_LINE__ will be properly replaced, and give this result !
CodeLocationTest.java:9
CodeLocationTest.java:10: Entering in run method
CodeLocationTest.java:11: Entering in run method
CodeLocationTest.java:13 value of i is '0'
CodeLocationTest.java:13 value of i is '1'
CodeLocationTest.java:13 value of i is '2'
CodeLocationTest.java:13 value of i is '3'
CodeLocationTest.java:13 value of i is '4'
CodeLocationTest.java:13 value of i is '5'
CodeLocationTest.java:13 value of i is '6'
CodeLocationTest.java:13 value of i is '7'
CodeLocationTest.java:13 value of i is '8'
CodeLocationTest.java:13 value of i is '9'
Want to try ? Download the code from my svn repository, and run the Test.launch run configuration.
If you are sceptic about the performance boost, I even made a comparison of both systems.
Just run the testcase Speedtest, which is running about 1OOO times the same test, but disable the output stream to remove the lag of logging to the console.
The result ?
By inferring the location : 400 ms
Using the CodeLocationAgent : 50 ms
But of course without counting time took to process the class bytecode.
Next week I will explain you how I did it easily, with the help of the ASM library!
But in the meantime, you can check the source code on the svn.
Sources :
The agent code svn repository is http://svn.gallot.be/blog/Instument-abUse/Instrument-abUse/
The test code svn repository is http://svn.gallot.be/blog/Instument-abUse/Instrument-abUse-tets/
