VBoxDrv!rtR0MemObjNativeFree assertion

[Mihai Hanor]

The occasional assertion occurs on my Windows XP 32 bit machine. I can't reproduce it at will, but it always triggers at the same point: restarting the Windbg's debuggee (ctrl+shift+f5), while the current running process is suspended.

I'm playing around with the OSE win32 debug build (svn 30690, build with VS2010 and gcc 4.5.1). Excepting hardware virtualization, which is not functional with my build, the OSE seems to work without other major problems, at least no BSODs. I can provide the full kernel memory dump (+binaries, if it's necessary).

1: kd> .cxr 0xffffffffba4eb5fc ; kP
eax=8a893801 ebx=87e11970 ecx=8052b720 edx=00000001 esi=8973b030 edi=86a6bb48
eip=8a883ea8 esp=ba4eb9c8 ebp=ba4eb9d8 iopl=0         nv up ei pl nz na po nc
cs=0008  ss=0010  ds=0023  es=0023  fs=0030  gs=0000             efl=00000202
VBoxDrv!rtR0MemObjNativeFree+0x508:
8a883ea8 cc              int     3
  *** Stack trace for last set context - .thread/.cxr resets it
ChildEBP RetAddr  
ba4eb9d8 8a871fb8 VBoxDrv!rtR0MemObjNativeFree(
			struct RTR0MEMOBJINTERNAL * pMem = 0x886a2088)+0x508 [d:\vbox\src\vbox\runtime\r0drv\nt\memobj-r0drv-nt.cpp @ 184]
ba4eba24 8abcc3e0 VBoxDrv!RTR0MemObjFree(
			struct RTR0MEMOBJINTERNAL * MemObj = 0x886a2088, 
			bool fFreeMappings = false)+0x378 [d:\vbox\src\vbox\runtime\r0drv\memobj-r0drv.cpp @ 355]
ba4eba70 8acd0c90 VMMR0!gmmR0CleanupVMScanChunk(
			struct _AVLU32NodeCore * pNode = 0x891de018, 
			void * pvGVM = 0x88626890)+0x2e0 [d:\vbox\src\vbox\vmm\vmmr0\gmmr0.cpp @ 1019]
ba4ebb18 8abcbdb8 VMMR0!RTAvlU32DoWithAll(
			struct _AVLU32NodeCore ** ppTree = 0x8631b018, 
			int fFromLeft = 0n1, 
			<function> * pfnCallBack = 0x8abcc100, 
			void * pvParam = 0x88626890)+0xd0 [d:\vbox\src\vbox\runtime\common\table\avl_dowithall.cpp.h @ 74]
ba4ebb84 8abd7830 VMMR0!GMMR0CleanupVM(
			struct GVM * pGVM = 0x88626890)+0x268 [d:\vbox\src\vbox\vmm\vmmr0\gmmr0.cpp @ 870]
ba4ebb9c 8abd6fbb VMMR0!gvmmR0CleanupVM(
			struct GVM * pGVM = 0x88626890)+0x120 [d:\vbox\src\vbox\vmm\vmmr0\gvmmr0.cpp @ 921]
ba4ebc08 8a8608b4 VMMR0!gvmmR0HandleObjDestructor(
			void * pvObj = 0x88067e28, 
			void * pvGVMM = 0x867c1018, 
			void * pvHandle = 0x867c104c)+0x46b [d:\vbox\src\vbox\vmm\vmmr0\gvmmr0.cpp @ 1016]
ba4ebcc0 8a8604b3 VBoxDrv!supdrvCleanupSession(
			struct SUPDRVDEVEXT * pDevExt = 0x8901eb98, 
			struct SUPDRVSESSION * pSession = 0x886dd648)+0x374 [d:\vbox\src\vbox\hostdrivers\support\supdrv.c @ 745]
ba4ebcd8 8a85eac8 VBoxDrv!supdrvCloseSession(
			struct SUPDRVDEVEXT * pDevExt = 0x8901eb98, 
			struct SUPDRVSESSION * pSession = 0x886dd648)+0x13 [d:\vbox\src\vbox\hostdrivers\support\supdrv.c @ 661]
ba4ebd00 804ef19f VBoxDrv!VBoxDrvNtClose(
			struct _DEVICE_OBJECT * pDevObj = 0x8901eae0, 
			struct _IRP * pIrp = 0x86a6bb38)+0xa8 [d:\vbox\src\vbox\hostdrivers\support\win\supdrv-win.cpp @ 286]
ba4ebd10 80583af8 nt!IopfCallDriver+0x31
ba4ebd48 805bb456 nt!IopDeleteFile+0x132
ba4ebd64 805bb8a8 nt!ObpRemoveObjectRoutine+0xe0
ba4ebd7c 80538789 nt!ObpProcessRemoveObjectQueue+0x36
ba4ebdac 805cff62 nt!ExpWorkerThread+0xef
ba4ebddc 8054611e nt!PspSystemThreadStartup+0x34
00000000 00000000 nt!KiThreadStartup+0x16

[Mihai Hanor]

small dump

[Mihai Hanor]

The assert is also triggered by svn 35407 win32 debug build. pMemNt->Core.u.Mapping.R0Process is not equal to 0 (NIL_RTR0PROCESS), the second condition must be triggering the assert

VBoxDrv!rtR0MemObjNativeFree+0x558 [f:\vbox\src\vbox\runtime\r0drv\nt\memobj-r0drv-nt.cpp @ 195]
VBoxDrv!RTR0MemObjFree+0x378 [f:\vbox\src\vbox\runtime\r0drv\memobj-r0drv.cpp @ 355]
VMMR0!gmmR0CleanupVMScanChunk+0x2e0 [f:\vbox\src\vbox\vmm\vmmr0\gmmr0.cpp @ 1021]
VMMR0!RTAvlU32DoWithAll+0xd0 [f:\vbox\src\vbox\runtime\common\table\avl_dowithall.cpp.h @ 74]
VMMR0!GMMR0CleanupVM+0x268 [f:\vbox\src\vbox\vmm\vmmr0\gmmr0.cpp @ 872]
VMMR0!gvmmR0CleanupVM+0x120 [f:\vbox\src\vbox\vmm\vmmr0\gvmmr0.cpp @ 1143]
VMMR0!gvmmR0HandleObjDestructor+0x46b [f:\vbox\src\vbox\vmm\vmmr0\gvmmr0.cpp @ 1238]
VBoxDrv!supdrvCleanupSession+0x374 [f:\vbox\src\vbox\hostdrivers\support\supdrv.c @ 781]
VBoxDrv!supdrvCloseSession+0x17 [f:\vbox\src\vbox\hostdrivers\support\supdrv.c @ 697]
VBoxDrv!VBoxDrvNtClose+0xa8 [f:\vbox\src\vbox\hostdrivers\support\win\supdrv-win.cpp @ 286]
nt!IopfCallDriver+0x31
nt!IopDeleteFile+0x132
nt!ObpRemoveObjectRoutine+0xe0
nt!ObpProcessRemoveObjectQueue+0x36
nt!ExpWorkerThread+0xef
nt!PspSystemThreadStartup+0x34
nt!KiThreadStartup+0x16

Assert(     pMemNt->Core.u.Mapping.R0Process == NIL_RTR0PROCESS
       ||   pMemNt->Core.u.Mapping.R0Process == RTR0ProcHandleSelf());

Ignore the first mentions about not working VT-x, it was caused by buggy BIOS. To reproduce, have the debug build running some VM (even without an OS), using the debugger, break inside virtualbox.exe user code, then issue an debuggee restart (ctrl+shift+f5 for Windbg). If you try doing that several times, you could hit the assert.

[Mihai Hanor]

​http://www.sendspace.com/file/pz2zys
It contains the full kernel dump, VMMR0 and VBoxDrv (bin & obj files), to be put next to svn35407 sources

[aeichner]

Please reopen if still relevant with a recent VirtualBox release.