[vbox-dev] Access violation when exiting VBoxHeadless.exe

[David P. Reese, Jr.]

We ran into an access violation in VBoxHeadless.exe just after a VM was successfully powered off, but before the VMM process exited.  Here is the stack trace:

0:005> t
(5e4.adc): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
ole32!CStdMarshal::ReleaseAllIPIDEntries+0x3a:
000007fe`fe1995c2 488b4b20        mov     rcx,qword ptr [rbx+20h] ds:00000000`0038c020=????????????????
0:000> kbn
 # RetAddr           : Args to Child                                                           : Call Site
00 000007fe`fe199550 : 000007fe`00000000 00000000`003be768 00000000`00000000 00000000`01ea69c0 : ole32!CStdMarshal::ReleaseAllIPIDEntries+0x3a
01 000007fe`fe199428 : 00000000`00000000 00000000`00000001 00000000`00000000 00000000`01ea69c0 : ole32!CStdMarshal::~CStdMarshal+0x18
02 000007fe`fe199b49 : 00000000`003be760 00000000`01ea69c0 00000001`00000000 00000000`00000000 : ole32!CStdIdentity::`scalar deleting destructor'+0x14
03 00000001`4000832f : 00000000`80000000 00000000`0036f8a0 00000000`00000000 00000000`0000015a : ole32!CStdIdentity::CInternalUnk::Release+0xdc
04 00000001`40008543 : 00000000`00000000 00000000`00000005 00000000`01ea8560 00000000`00000001 : VBoxHeadless!TrustedMain+0x29bf [c:\code\bst\contrib\vbox64\src\vbox\frontends\vboxheadless\vboxheadless.cpp @ 1294]
05 00000001`400096f0 : 00000000`00130000 00000000`00000000 00000000`00130000 00000000`00000000 : VBoxHeadless!main+0x63 [c:\code\bst\contrib\vbox64\src\vbox\frontends\vboxheadless\vboxheadless.cpp @ 1323]
06 00000000`76eb652d : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : VBoxHeadless!__tmainCRTStartup+0x120 [f:\sp\vctools\crt_bld\self_64_amd64\crt\src\crtexe.c @ 597]
07 00000000`770ec521 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : kernel32!BaseThreadInitThunk+0xd
08 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x1d

The access violation occurred during the return statement at the end of TrustedMain().  It looks like the local variable `machine' may still contain a COM pointer when TrustedMain() returns.  At this line, the ComPtr<> destructor attempts to release the COM object, but this is after com::Shutdown(), which may very well cause an access violation.  Adding `machine.setNull()' before com::Shutdown() seems to make our problem go away.

We discovered this issue in 4.0.4 and it is still present in 4.0.6.  To my surprise, the fix seems to have been snuck into the tree via changeset 35741 on January 27th 2011, which was almost three months before the 4.0.6 release.  Our issue is that without being able to see which revisions were merged into the VBox-4.0 branch, it's a little hard for us to predict which changes will make it into the next release build.

For a given changeset, is there any way for us to know if/when it will make it into the next release?

Thank you.

--
David P. Reese, Jr.
BlueStacks, Inc.
daver at bluestacks.com