Java XPCOM connector - Event memory leak

[Max D]

There seems to be a memory leak when retrieving events from Virtualbox when using the XPCOM Java connector. This does not appear when using the WebServices connector.

OutOfMemory attached is a standalone class for both WS & XPCOM. The class was compiled within a jar called oom-test.jar, and both WS & XPCOM jars were put in the same directory.

Classpath was adapated to switch between the bindings and vbox.ws property was use to switch at the test code level, and vbox.ws.host is used to specifiy the host for WS.

Commands to run the test on XPCOM & WS were as follow, ran from the directory were all jars are put.

XPCOM:

java -cp oom-test.jar:vboxjxpcom.jar -Xmx2M -Dvbox.home=/usr/lib/virtualbox OutOfMemory

WebServices

java -cp oom-test.jar:vboxjws.jar -Xms10M -Xmx40M -Dvbox.ws=1 -Dvbox.ws.host=localhost OutOfMemory

All code tested against 4.3.20 and similar behaviour is seen on 4.3.16

[Max D]

Test code

[Max D]

Java Heap usage monitoring while using Web Services

[Max D]

Java Heap usage monitoring while using XPCOM

[Max D]

Java Heap memory usage screenshots using VisualVM profiler attached for both WS & XPCOM

[Max D]

Compiled source

[Frank Mehnert]

I think I see a memory leak in the java process and this probably comes from our Java bindings. This needs extensive debugging, valgrind didn't show anything suspicious.

[Frank Mehnert]

Spent some time debugging this problem but didn't find a fix. I know that the session object (from ISession session = vboxManager.getSessionObject();) leaks and that some objects which are created in the following line (IProgress pOn = machine.launchVMProcess(session, "headless", null);) leak as well. The garbage collector should take care of deleting the unreferneced objects but it does not. Even setting session = null after calling session.unlockMachine() does not help.

This is some fundamental problem of the VBox JNI interface. At the moment I don't know what prevents the garbage collector from deleting unused objects.

[Frank Mehnert]

The recent updates (r56603, r56604, r56605) introduce some fixes which make the Java garbage collector close some open XPCOM resources. There are more issues. It looks like the major issue is running the XPCOM event queue which isn't done for Java clients. This needs to be fixed.

[Klaus Espenlaub]

The remaining part of the problem can be solved mostly by better documentation (will be in 5.0.0 and soon in 4.3, too). The sample code already has the secret sauce for solving the remaining XPCOM event pileup (as the queue isn't serviced), but doesn't say a word about the why and what: mgr.waitForEvents(0); processes what has accumulated (MUST be done in the main thread!), and it's enough to call it every few 100 msec to every few seconds. The call can be made with all Java VirtualBox API variants, it automatically turns into an (always local!) noop.

[Klaus Espenlaub]

Hmmm... after double checking I'm not so sure any more if the requirement "MUST be done in the main thread" is 100% accurate. The C++ code is extremely convoluted. I guess it's worth an experiment with doing these calls in the thread where the first VirtualBoxManager.createInstance(...) call is done. No time to test this.

[Max D]

After checking 5.0 RC2, I can confirm that I can't see any memory leak at first glance.

This is with calling VirtualBoxManager::waitForEvent() outside of the main thread, in a dedicated event thread (already present in the initial code).
The good news is that you don't need to do it in the main thread or keep some thread active just to perform this action. Seems like any thread does the job.
Finally, I've slightly updated the code to run several concurent threads to speed up the test.

You'll find three new files attached:

• Java code source
• Compiled java
• Screenshot of heap usage after 1h of running

Code can be run using the following command, after putting the 5.0 RC2 xpcom binding jar in the current directory:

java -cp oom-5_0_RC2.jar:vboxjxpcom.jar -Xmx2M -Dvbox.home=/usr/lib/virtualbox OutOfMemory_5_0_RC2

My final test is going to be in Hyperbox directly and see if I can release the XPCOM binding at runtime or if it is still getting stuck due to some other memory leak.

[Max D]

Test Source code for 5.0 RC2

[Max D]

Compiled test case for 5.0 RC2

[Max D]

Heap Monitoring for 5.0 RC2