SegFault in _XGetRequest () from /usr/lib/libX11.so.6 during "Import Appliance"

[uh]

Clicking on the File Selector in "Import Appliance" causes a segfault in VirtualBox 5.1

kernel: VirtualBox[32689]: segfault at 0 ip 00007fc02e1edf81 sp 00007fff44cf73b0 error 6 in libX11.so.6.3.0[7fc02e1a9000+13b000]

gdb backtrace (let me know if you want me to recompile with proper symbols):

Thread 1 "VirtualBox" received signal SIGSEGV, Segmentation fault.
0x00007fac3ecf7f81 in _XGetRequest () from /usr/lib/libX11.so.6
(gdb) bt
#0  0x00007fac3ecf7f81 in _XGetRequest () from /usr/lib/libX11.so.6
#1  0x00007fac3ecd2579 in XChangeProperty () from /usr/lib/libX11.so.6
#2  0x00007fac3ecef919 in XSetTransientForHint () from /usr/lib/libX11.so.6
#3  0x00007fac2f79d48e in ?? () from /usr/lib/qt/plugins/platformthemes/libqgtk3.so
#4  0x00007fac2f7a0121 in ?? () from /usr/lib/qt/plugins/platformthemes/libqgtk3.so
#5  0x00007fac3d47a780 in QDialogPrivate::setNativeDialogVisible(bool) () from /usr/lib/libQt5Widgets.so.5
#6  0x00007fac3d48cdc2 in QFileDialog::setVisible(bool) () from /usr/lib/libQt5Widgets.so.5
#7  0x00007fac3d47b631 in QDialog::exec() () from /usr/lib/libQt5Widgets.so.5
#8  0x00007fac3d48d635 in QFileDialog::getOpenFileUrl(QWidget*, QString const&, QUrl const&, QString const&, QString*, QFlags<QFileDialog::Option>, QStringList const&) () from /usr/lib/libQt5Widgets.so.5
#9  0x00007fac3d48d6fe in QFileDialog::getOpenFileName(QWidget*, QString const&, QString const&, QString const&, QString*, QFlags<QFileDialog::Option>) () from /usr/lib/libQt5Widgets.so.5
#10 0x00007fac3fbef8f5 in ?? () from /usr/lib/virtualbox/VirtualBox.so
#11 0x00007fac3fbefaac in ?? () from /usr/lib/virtualbox/VirtualBox.so
#12 0x00007fac3fdea168 in ?? () from /usr/lib/virtualbox/VirtualBox.so
#13 0x00007fac3ff48795 in ?? () from /usr/lib/virtualbox/VirtualBox.so
#14 0x00007fac3e3824e9 in QMetaObject::activate(QObject*, int, int, void**) () from /usr/lib/libQt5Core.so.5
#15 0x00007fac3d376142 in QAbstractButton::clicked(bool) () from /usr/lib/libQt5Widgets.so.5
#16 0x00007fac3d376394 in ?? () from /usr/lib/libQt5Widgets.so.5
#17 0x00007fac3d377787 in ?? () from /usr/lib/libQt5Widgets.so.5
#18 0x00007fac3d377964 in QAbstractButton::mouseReleaseEvent(QMouseEvent*) () from /usr/lib/libQt5Widgets.so.5
#19 0x00007fac3d44330a in QToolButton::mouseReleaseEvent(QMouseEvent*) () from /usr/lib/libQt5Widgets.so.5
#20 0x00007fac3d2d5f18 in QWidget::event(QEvent*) () from /usr/lib/libQt5Widgets.so.5
#21 0x00007fac3d4433e9 in QToolButton::event(QEvent*) () from /usr/lib/libQt5Widgets.so.5
#22 0x00007fac3d28ee3c in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from /usr/lib/libQt5Widgets.so.5
#23 0x00007fac3d29746d in QApplication::notify(QObject*, QEvent*) () from /usr/lib/libQt5Widgets.so.5
#24 0x00007fac3e356c80 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from /usr/lib/libQt5Core.so.5
#25 0x00007fac3d295b1d in QApplicationPrivate::sendMouseEvent(QWidget*, QMouseEvent*, QWidget*, QWidget*, QWidget**, QPointer<QWidget>&, bool) () from /usr/lib/libQt5Widgets.so.5

[uh]

Forgot to mention that I'm running Wayland. OS is Arch Linux.

(also, terribly sorry about the formatting)

[Michael Thayer]

Just a warning for starters: this sounds like something that we are not likely to be able to look at due to the rather interesting reproduction set-up. That said, is there any way you can see if you can reproduce this using classic X.Org? And did you build VirtualBox yourself (or get it from someone who did)? If so, would you be able to test with one of our binaries?

Needless to say, any additional investigation you can do is welcome.

[Michael Thayer]

And of course attaching a log file is always good as it contains lots of useful information.

[Frank Mehnert]

Hi uh, if you could recompile VBox yourself and provide proper symbols that might help.

[uh]

The issue doesn't occur with X.org. Looks like it's more of a qt bug then?

[Phylu]

I can confirm this bug with VirtualBox 5.1.2-1 on Arch Linux with Gnome 3.20.2. This only happens, if Gnome is started with Wayland. When the Chooser is clicked (not only in Import appliance, but also when trying to load an optical device), VirtualBox crashes.

In a X.Org Session, this does not happen.

[Frank Mehnert]

Please reopen if still relevant with VBox 5.1.22.