VRDEAuth seqfault

[Stephen Rondeau]

Here is the message I get in dmesg on a Fedora 28 updated system:

VRDEAuth[40662]: segfault at 7fabb53016c0 ip 00007fabb53016c0 sp 00007fabb5506a08 error 15

Fedora 28 "uname -a":

Linux cn4-vcl4 4.16.14-300.fc28.x86_64 #1 SMP Tue Jun 5 16:23:44 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

It happens regardless of the guest OS (Fedora, Windows 2012 R2, or Windows 10).

The VRDP session freezes or drops after some amount of time, and the only thing we can do is power-cycle the VM to get back the session for a short time.

VBox.log attached

[Stephen Rondeau]

VBox.log file

[Stephen Rondeau]

Could someone please change the subject line to:

VRDEAuth segfault

for easier searching? Thanks!

[Stephen Rondeau]

This is still happening with 5.2.16 on a Fedora 28 host:

Linux cn1-vcl1 4.17.7-200.fc28.x86_64 #1 SMP Tue Jul 17 16:28:31 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

The first attempt to connect to a newly-started VM works, but any subsequent attempts fail because the VRDEAuth command from VBoxHeadless segfaults. I have tried running pamtester to make sure the vrdpauth pam module allows repeated authentications, and it does so successfully. I've attached my vrdpauth PAM file.

[Stephen Rondeau]

vrdpauth file from /etc/pam.d

[Stephen Rondeau]

Here is the latest dmesg:

[ 1922.126307] VRDEAuth[6875]: segfault at 7f48cedbf6c0 ip 00007f48cedbf6c0 sp 00007f49e2a719e8 error 14 in libpam.so.0.84.2[7f48cec78000+1ff000]

and this is the corresponding message in /var/log/messages:

Jul 25 07:42:36 cn1-vcl1 audit[6875]: ANOM_ABEND auid=1000 uid=1000 gid=1000 ses=3 pid=6875 comm="VRDEAuth" exe="/usr/lib/virtualbox/VBoxHeadless" sig=11 res=1

I really need a fix for this or an idea of where in the source code I can find VRDE-related code so I can attempt to fix it myself. I have tried (perhaps too quickly, or it is licensed code), but can't find anything.

Hundreds of computer science and information technology students depend on this method for remotely accessing the graphical desktops of VMs throughout our academic year. If it is as unreliable as it became in the past few months, VirtualBox can no longer be our preferred hypervisor, as it has been since 2009. On guest Windows, we can use Remote Desktop once the guest OS is up, but it gets more confusing for the students, who are already confused by the virtual "computer in the cloud" concept. And we would need a different solution for Linux guest OSes.

[pentagonik]

Can you please try to supply a core dump of when this has happened? See here for more information: https://www.virtualbox.org/wiki/Core_dump

[sunlover]

srondeau, the source code is at: https://www.virtualbox.org/browser/vbox/trunk/src/VBox/HostServices/auth/pam/VBoxAuthPAM.c

It would be great if you could find the problem.

[Stephen Rondeau]

Replying to pentagonik:

Can you please try to supply a core dump of when this has happened? See here for more information: https://www.virtualbox.org/wiki/Core_dump

I generated one, but it is 11MB compressed. The file upload limit on this site is 512KB.

Here it is:

​https://drive.google.com/open?id=1RxOjOedqmREzj-c-dIuLRIcDGBrTAucu

[Stephen Rondeau]

Replying to sunlover:

srondeau, the source code is at: https://www.virtualbox.org/browser/vbox/trunk/src/VBox/HostServices/auth/pam/VBoxAuthPAM.c

It would be great if you could find the problem.

Thanks for the pointer. It took me hours to completely compile (without hardening) the source, but it was difficult to get everything working after that. So I reverted to trying to get the core file created for pentagonik.

I did find out that the problem is not 100% reproducible. I could not get VRDEAuth to segfault a couple of times. That behavior may be a sign of uninitialized storage somewhere -- most of the time it causes a crash, but once in a while it is initialized properly and continues to work.

[AlexIz]

I have exactly the same issue

Host: the latest FC28 with the next rpms:

VirtualBox-5.2.14-1.fc28.x86_64
VirtualBox-kmodsrc-5.2.14-1.fc28.noarch
VirtualBox-server-5.2.14-1.fc28.x86_64

Guest: win7 64x Remote access is enabled and authorization method set to "external" After guest machine is started:

1. start remote session, finish remote session
2. try to start remote session --- it doesn't connect
3. suspend / unsuspend guest VM
4. now remote access works again, go to step #1

Dmesg is full of messages like this:

[82358.640919] VRDEAuth[9420]: segfault at 7f041d6ff6c0 ip 00007f041d6ff6c0 sp 00007f067bc73a48 error 15
[82358.758500] VRDEAuth[9421]: segfault at 7f041d6ff6c0 ip 00007f041d6ff6c0 sp 00007f067bc73a08 error 15
[855796.782003] VRDEAuth[22461]: segfault at 7f22b39bf6c0 ip 00007f22b39bf6c0 sp 00007f23e21e9a48 error 15 in zero (deleted)[7f22b37fd000+200000]
[855796.896764] VRDEAuth[22462]: segfault at 7f22b39bf6c0 ip 00007f22b39bf6c0 sp 00007f23e21e9a08 error 15 in zero (deleted)[7f22b37fd000+200000]
[939731.960953] VRDEAuth[30964]: segfault at 7f041973f6c0 ip 00007f041973f6c0 sp 00007f067e1bfa48 error 15 in zero (deleted)[7f04196bd000+200000]
[939732.079756] VRDEAuth[30965]: segfault at 7f041973f6c0 ip 00007f041973f6c0 sp 00007f067e1bfa08 error 15 in zero (deleted)[7f04196bd000+200000]
[944088.973046] VRDEAuth[31385]: segfault at 7f041973f6c0 ip 00007f041973f6c0 sp 00007f067e1bfa28 error 15 in zero (deleted)[7f04196bd000+200000]
[944089.088910] VRDEAuth[31386]: segfault at 7f041973f6c0 ip 00007f041973f6c0 sp 00007f067e1bf9e8 error 15 in zero (deleted)[7f04196bd000+200000]
[945273.489183] VRDEAuth[31634]: segfault at 7f8a7cbbf6c0 ip 00007f8a7cbbf6c0 sp 00007f8ce5209a48 error 15
[945273.606072] VRDEAuth[31635]: segfault at 7f8a7cbbf6c0 ip 00007f8a7cbbf6c0 sp 00007f8ce5209a08 error 15
[945303.710701] VRDEAuth[31637]: segfault at 7f8a7cbbf6c0 ip 00007f8a7cbbf6c0 sp 00007f8ce5209a28 error 15
[945303.828855] VRDEAuth[31638]: segfault at 7f8a7cbbf6c0 ip 00007f8a7cbbf6c0 sp 00007f8ce52099e8 error 15
[1080452.168684] VRDEAuth[13066]: segfault at 7fba082bf6c0 ip 00007fba082bf6c0 sp 00007fbc718d8a48 error 14 in libcap-ng.so.0.0.0[7fba080c2000+200000]
[1080452.292717] VRDEAuth[13067]: segfault at 7fba082bf6c0 ip 00007fba082bf6c0 sp 00007fbc718d8a08 error 14 in libcap-ng.so.0.0.0[7fba080c2000+200000]
[1232286.193352] VRDEAuth[28668]: segfault at 7f70ea25f6c0 ip 00007f70ea25f6c0 sp 00007f734f83aa48 error 14 in libcap-ng.so.0.0.0[7f70ea062000+200000]
[1232286.326457] VRDEAuth[28669]: segfault at 7f70ea25f6c0 ip 00007f70ea25f6c0 sp 00007f734f83aa08 error 14 in libcap-ng.so.0.0.0[7f70ea062000+200000]
[1287329.872030] VRDEAuth[1952]: segfault at 7f26b0e5f6c0 ip 00007f26b0e5f6c0 sp 00007f291a47ca48 error 14 in libcap-ng.so.0.0.0[7f26b0ca2000+200000]
[1287330.008307] VRDEAuth[1953]: segfault at 7f26b0e5f6c0 ip 00007f26b0e5f6c0 sp 00007f291a47ca08 error 14 in libcap-ng.so.0.0.0[7f26b0ca2000+200000]
[1290840.068559] VRDEAuth[2323]: segfault at 7f26b0e5f6c0 ip 00007f26b0e5f6c0 sp 00007f291a47ca28 error 14 in libcap-ng.so.0.0.0[7f26b0cc2000+200000]
[1290840.206597] VRDEAuth[2324]: segfault at 7f26b0e5f6c0 ip 00007f26b0e5f6c0 sp 00007f291a47c9e8 error 14 in libcap-ng.so.0.0.0[7f26b0cc2000+200000]
[1291093.535414] VRDEAuth[2364]: segfault at 7f26b0e5f6c0 ip 00007f26b0e5f6c0 sp 00007f291a47ca08 error 14 in libcap-ng.so.0.0.0[7f26b0cc2000+200000]
[1291093.674687] VRDEAuth[2365]: segfault at 7f26b0e5f6c0 ip 00007f26b0e5f6c0 sp 00007f291a47c9c8 error 14 in libcap-ng.so.0.0.0[7f26b0cc2000+200000]
[1292701.034574] VRDEAuth[2525]: segfault at 7f26b0e5f6c0 ip 00007f26b0e5f6c0 sp 00007f291a47c9e8 error 14 in libcap-ng.so.0.0.0[7f26b0cc2000+200000]
[1292701.175022] VRDEAuth[2526]: segfault at 7f26b0e5f6c0 ip 00007f26b0e5f6c0 sp 00007f291a47c9a8 error 14 in libcap-ng.so.0.0.0[7f26b0cc2000+200000]
[1309883.872710] VRDEAuth[13262]: segfault at 7f1f3467f6c0 ip 00007f1f3467f6c0 sp 00007f2190917a28 error 14 in pam_permit.so[7f1f344b4000+1ff000]
[1309884.001642] VRDEAuth[13263]: segfault at 7f1f3467f6c0 ip 00007f1f3467f6c0 sp 00007f21909179e8 error 14 in pam_permit.so[7f1f344b4000+1ff000]
[1309899.587311] VRDEAuth[13266]: segfault at 7f1f3467f6c0 ip 00007f1f3467f6c0 sp 00007f2190917a08 error 14 in pam_permit.so[7f1f344b4000+1ff000]
[1309899.716757] VRDEAuth[13267]: segfault at 7f1f3467f6c0 ip 00007f1f3467f6c0 sp 00007f21909179c8 error 14 in pam_permit.so[7f1f344b4000+1ff000]
[1312507.416112] VRDEAuth[13639]: segfault at 7f5e41dbf6c0 ip 00007f5e41dbf6c0 sp 00007f6021ca1a48 error 15
[1312507.542911] VRDEAuth[13640]: segfault at 7f5e41dbf6c0 ip 00007f5e41dbf6c0 sp 00007f6021ca1a08 error 15
[1313353.880419] VRDEAuth[13845]: segfault at 7f2e499bf6c0 ip 00007f2e499bf6c0 sp 00007f307a21ba48 error 15 in zero (deleted)[7f2e497fd000+200000]
[1313353.998672] VRDEAuth[13846]: segfault at 7f2e499bf6c0 ip 00007f2e499bf6c0 sp 00007f307a21ba08 error 15 in zero (deleted)[7f2e497fd000+200000]
[1819259.055794] VRDEAuth[952]: segfault at 7f2e499bf6c0 ip 00007f2e499bf6c0 sp 00007f307a21ba28 error 15 in zero (deleted)[7f2e497fd000+200000]
[1819259.186395] VRDEAuth[953]: segfault at 7f2e499bf6c0 ip 00007f2e499bf6c0 sp 00007f307a21b9e8 error 15 in zero (deleted)[7f2e497fd000+200000]
[2148325.477115] VRDEAuth[2599]: segfault at 7f7aacfbf6c0 ip 00007f7aacfbf6c0 sp 00007f7d0e6d9a48 error 15 in zero (deleted)[7f7aacdfd000+200000]
[2148325.607611] VRDEAuth[2600]: segfault at 7f7aacfbf6c0 ip 00007f7aacfbf6c0 sp 00007f7d0e6d9a08 error 15 in zero (deleted)[7f7aacdfd000+200000]
[2171225.593269] VRDEAuth[5369]: segfault at 7fc6ae9d56c0 ip 00007fc6ae9d56c0 sp 00007fc8b3afea48 error 14 in pam_nologin.so[7fc6ae7d9000+200000]
[2171225.718225] VRDEAuth[5370]: segfault at 7fc6ae9d56c0 ip 00007fc6ae9d56c0 sp 00007fc8b3afea08 error 14 in pam_nologin.so[7fc6ae7d9000+200000]
[2171284.365611] VRDEAuth[5496]: segfault at 7fc6ae9d56c0 ip 00007fc6ae9d56c0 sp 00007fc8b3afea28 error 14 in pam_nologin.so[7fc6ae7d9000+200000]
[2171284.488520] VRDEAuth[5497]: segfault at 7fc6ae9d56c0 ip 00007fc6ae9d56c0 sp 00007fc8b3afe9e8 error 14 in pam_nologin.so[7fc6ae7d9000+200000]

[Stephen Rondeau]

It may be that this problem is in Linux somewhere. I recently upgraded to Fedora 30's latest updates, and VRDP doesn't seem to segfault nearly as much or at all. This is with VirtualBox 5.2.34 and 6.1.0