rdesktop connecting to VRDP crashes guest VM

[jbrown10]

After upgrading to 4.2.4 from 4.2.2 I have begun to experience periodic events where a guest vm crashes as soon as I connect to it via rdesktop.

syslog contains the following record:

kernel: [61012.245046] VRDP-OUT[9395]: segfault at 1400000004 ip 00007f8f622a4f92 sp 00007f8f2a847b10 error 6 in libc-2.15.so[7f8f62222000+1b5000]

Host system: Ubuntu 12.10 64-bit
Linux explorer 3.5.0-18-generic #29-Ubuntu SMP Fri Oct 19 10:26:51 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

Guest OS: Windows 7 64-bit

rdesktop: Version 1.7.1.

Log snippet from the vm beginning where the connection comes in:

03:59:16.128063 VRDP: New connection: 
03:59:16.172195 VRDP: Methods 0x00000003
03:59:16.172234 VRDP: Channel: [cliprdr] [1004]. Accepted.
03:59:16.172256 VRDP: Channel: [rdpdr] [1005]. Accepted.
03:59:16.323294 VRDP: Client seems to be rdesktop.
03:59:16.323337 VRDP: Logon: explorer (127.0.0.1) build 2600. User: [jpb] Domain: [] Screen: 0
03:59:16.324743 AUTH: User: [jpb]. Domain: []. Authentication type: [External]
03:59:16.325248 AUTH: ConsoleVRDPServer::Authenticate: loading external authentication library 'VBoxAuth'
03:59:16.325908 AUTH: Using entry point 'AuthEntry'.
03:59:16.563754 AUTH: external authentication module returned 'access granted'
03:59:16.563783 AUTH: Access granted.
03:59:16.568895 VBVA: VRDP acceleration has been requested.
03:59:17.620661 VRDP: SunFlsh disabled.

The log terminated with the last line above and the entry in syslog appeared.

To recreate:

1. Activate RDP on a Windows 7 64-bit VM.
   1. port 33881
   2. authentication method External
   3. authentication timeout 5000
   4. allow multiple connections checked
2. Start the VM
3. Open a linux command line on the host
4. Attempt to connect to the guest:

   /var/log$ rdesktop -u <userid> -p <password> localhost:33881
   

Output in the command window:

Autoselected keyboard map en-us
ERROR: recv: Connection reset by peer
ERROR: send: Broken pipe

Syslog:

Oct 31 19:45:08 explorer VirtualBox: pam_ecryptfs: seteuid error
Oct 31 19:45:08 explorer kernel: [85957.901303] VRDP-OUT[22947]: segfault at 1400000004 ip 00007fe1f82baf92 sp 00007fe1c09319a0 error 6 in libc-2.15.so[7fe1f8238000+1b5000]

I was able to repeat this with a Ubuntu 12.10 64-bit guest vm as well.

It appears to be a problem with the reported pam error and then an unhandled exception coming from the VRDPAUTH.

Changing the authentication method from External to Null for RDP and all works fine since it doesn't access pam via VRDPAUTH.

[jbrown10]

Additional information: 4.2.0 and 4.2.2 worked under Ubuntu 12.10. I ran 4.2.0 for several days under 12.10 and then upgraded to 4.2.2. Since 4.2.4 RDP has crashed the Vbox guest when using external authentication.

[Frank Mehnert]

Can you provide a core dump?

[jbrown10]

Replying to frank:

Can you provide a core dump?

Dump sent to Frank M per the core dump instructions.