[vbox-dev] "An invalid or unaligned stack was encountered during an unwind operation" in VBoxREM.dll

[Klaus Espenlaub]

Rūdolfs,

On 27.10.2014 22:39, Rūdolfs Bundulis wrote:
> Hi,
>
> my custom frontend for VirtualBox seemed to work fine and stable but
> then I started to play around with number of CPU's and memory and I
> started getting "An invalid or unaligned stack was encountered during an
> unwind operation" from VBoxREM.dll. I downloaded the sources for
> VirtualBox 4.3.18, built it for x64 debug and started debugging.
>
> * The first weird thing was that I could not find a .pdb file for
> VBoxREM.dll, is that some kind of special .dll? I have pdb files for
> VBoxC.dll and friends under "out\win.amd64\debug\stage\debug\bin" but I
> can't find one for VBoxREM.dll.

That's expected... VBoxREM.dll is very special, it contains the (qemu) 
recompiler, which must be built using gcc, and that's the reason why 
there are no pdb files for it. It should contain dwarf debug info 
straight in the dll, which gdb understands. Yes, it's annoying because 
one has to pick the right debugger straight from the beginning.

> * The stack trace is (I know it won't give much help):
>
> ntdll.dll!00007ffe679ac0b4()Unknown
> ntdll.dll!00007ffe67913356()Unknown
> msvcrt.dll!__longjmp_internal
()Unknown
>  >VBoxREM.dll!000000006fb0f3c4()Unknown
>
> The frontend does not have any output from VirtualBox on stdout or
> stderr and the logfile also does not show any errors.
>
> Could anyone provide any pointers how should I approach debugging this?

Hope gdb is helpful.

On modern hardware which supports VT-x/AMD-V it's somewhat unexpected 
that you end up in the recompiler, but there are some corner cases which 
it is still used for.

Klaus

>
> Best Regards,
> Rudolfs Bundulis