VM does not stop if client is connected via VNC during shutdown/poweroff/stop

[Derek Schrock]

On FreeBSD 11.1 amd64 with VNC support enabled if the VM is shutdown while a client is connected to the VM console via VNC the VM will become stuck in a Stopping state and does not power off and VNC server remains up.

If no VNC sessions are open during power off the VM will shutdown.

This appears to when started via virtualbox and vboxheadless (from the command line and phpvirtualbox).

The only way to get the VM to a stat where it can be interacted with is to kill the virtualbox or vboxheadless process. Once killed the VM is given an "Aborted" status.

[Derek Schrock]

VBox.log the VM stops logging after an attempt to resize the console

[Derek Schrock]

vboxheadless output containing some libvncserver messages

[Quentin BUATHIER]

The issue is also in the version 5.2.10_Debianr121806 on Debian Stretch (Kernel 4.9.0-6). It isn't on Jessie with the exact same version (compile by hand). The oldest version I did test is the 5.1.6 and it already has the issue too on Stretch and not Jessie.

To reproduce the issue, I'm just configuring a VM to enable VNC, start it, launch my VNC client and using vboxmanage to poweroff the VM. 100% reproducible.

I will attach the log of my VBox.log and vboxheadless.log too, if it could be useful.

Could it be due to the newest version of libvncserver available on Stretch?

Derek: Can you change the "Host type" of the ticket?

[Quentin BUATHIER]

VBox.log (on Debian Stretch) the VM stops logging after an attempt to resize the console

[Quentin BUATHIER]

vboxheadless (on Debian Stretch) output containing some libvncserver messages

[Quentin BUATHIER]

I did use GDB to see the backtrace of VBoxHeadless when the VM is stuck and it looks like one thread is stuck in the libvncserver itself.

Thread 27 (Thread 0x7fd4f41b7700 (LWP 19607)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00007fd524ddd11d in rfbClientConnectionGone () from /usr/lib/x86_64-linux-gnu/libvncserver.so.1
#2  0x00007fd524ddc65a in rfbShutdownServer () from /usr/lib/x86_64-linux-gnu/libvncserver.so.1
#3  0x00007fd52503315a in ?? () from /usr/lib/virtualbox/ExtensionPacks/VNC/linux.amd64/VBoxVNC.so

I will attach the full output of GDB

[Quentin BUATHIER]

gdb backtrace output (on Debian Stretch) of a stuck vboxheadless process

[Quentin BUATHIER]

I opened a pull request on libvncserver to fix the issue and successfully test it on a hand-compiled version of Virtualbox. ​https://github.com/LibVNC/libvncserver/pull/238

[Valery Ushakov]

Thanks for the update. Since VBox uses host's libvncserver library I'm closing this as not a bug.