VirtualBox

Opened 13 years ago

Closed 13 years ago

#7981 closed defect (fixed)

rdesktop-vrdp crashes with memory corruption - regression from 3.2.12-68302 -> fixed as of 14 Jan 2011

Reported by: Paul Kronenwetter Owned by:
Component: RDP Version: VirtualBox 4.0.0
Keywords: regression Cc:
Guest type: Linux Host type: Linux

Description

rdesktop-vrdp -r usb {host}

Autoselected keyboard map en-us
WARNING: Remote desktop changed from 800x600 to 1020x550.
*** glibc detected *** rdesktop-vrdp: free(): invalid next size (fast): 0x000000000236a930 ***
*** glibc detected *** rdesktop-vrdp: malloc(): memory corruption: 0x000000000236a950 ***

From here the window doesn't update and sometimes causes the rdesktop client (Fedora 14) to freeze / crash.

Problematic rdesktop-vrdp is from VirtualBox-4.0-4.0.0_69151_fedora14-1.x86_64.rpm. Previous rdesktop-vrdp from VirtualBox-3.2-3.2.12_68302_fedora14-1.x86_64.rpm works perfectly.

Attachments (3)

backtrace.txt (14.3 KB ) - added by Paul Kronenwetter 13 years ago.
Backtrace from failed session.
VBox.log (46.7 KB ) - added by Paul Kronenwetter 13 years ago.
VBox log file from last comment.
valgrind.log (37.0 KB ) - added by Paul Kronenwetter 13 years ago.
Valgrind output with debug-enabled binary - updated 7-Jan-2011

Download all attachments as: .zip

Change History (20)

comment:1 by Frank Mehnert, 13 years ago

Any easy way to reproduce this crash?

comment:2 by Paul Kronenwetter, 13 years ago

The glibc memory corruption occurs almost immediately, but only when redirecting USB devices. I have only seen the system freeze symptom a few times and not recently.

comment:3 by Paul Kronenwetter, 13 years ago

Ok, there may be another factor involved. Either it's due to my USB scanner (from NeatWorks) or the fact that I'm using IPv6 at home and not from work... Let me investigate further and report back.

by Paul Kronenwetter, 13 years ago

Attachment: backtrace.txt added

Backtrace from failed session.

comment:4 by Paul Kronenwetter, 13 years ago

I've tested a few scenarios and it's not IPv6. I've attached a backtrace though I don't get one every time. The command that generated it was: $ rdesktop-vrdp -r usb -z -P -a 16 host:3407
Though the options don't seem to matter much but the USB device connection / disconnection is important.
Let me stop rambling...

Here's a sequence of steps with outcomes:

Start the VM (WinXP SP3) and attach rdesktop-vrdp -r usb host:3407 while the VBox splash screen was still visible. XP saw the USB devices.
Unplug the scanner and rdesktop crashes.
Re-connected rdesktop, crashed almost immediately.
Re-plugged in the scanner and reconnected rdesktop, it was fine but no USB devices appeared.
Rebooted Win XP, still no USB devices.
Completely shut down VM, started VM and reconnected with scanner attached, devices are seen.
Detached rdesktop, unplugged scanner, re-connected rdesktop. Crash again.
Detached rdesktop, re-plug scanner, connect with rdesktop-vrdp from 3.2_68302.
Do all of the above stuff, never crashes, all works as expected.
Detached rdesktop, re-connect with 4.0's rdesktop-vrdp and with scanner connected, seems happy.

Hopefully this is something you can use. I can test with a debug version (64-bit) or try other combinations.

by Paul Kronenwetter, 13 years ago

Attachment: VBox.log added

VBox log file from last comment.

comment:5 by Michael Thayer, 13 years ago

If you can reproduce this using

http://www.virtualbox.org/download/testcase/rdesktop-vrdp

which contains debug symbols, could you try running it in valgrind to see what output is produced? Thanks.

comment:6 by Michael Thayer, 13 years ago

Did the run under Valgrind also abort with the error you reported initially? I am slightly confused, as I can't see any trace of it in the output.

comment:7 by Michael Thayer, 13 years ago

And could you also provide the output of "VBoxManage list usbhost" with and without the scanner plugged in, in a situation in which you can reproduce the crash? Thanks again.

comment:8 by Paul Kronenwetter, 13 years ago

I honestly don't remember... Sorry. I did get it to crash this time, so I'll replace the log file. Looks much more interesting. I'll try to get the list usbhost, but it doesn't give me a lot of time before it bombs...

by Paul Kronenwetter, 13 years ago

Attachment: valgrind.log added

Valgrind output with debug-enabled binary - updated 7-Jan-2011

comment:9 by Paul Kronenwetter, 13 years ago

Unless I'm mistaken, list usbhost only shows the USB devices actually attached to the host. I have two of them, but they're attached to a different VM and they're not the scanner involved. That's why I'm using RDP with USB extensions. :)

comment:10 by Michael Thayer, 13 years ago

You are right of course - is it possible to install VirtualBox on the system you are attaching the scanner to, in order to do that test?

comment:11 by Michael Thayer, 13 years ago

Although hopefully the Valgrind output will be enough to fix this - this time it does look like what I need.

comment:12 by Paul Kronenwetter, 13 years ago

Um yes, though it'll be *much* slower. I'll see what I can do and get back to you.

comment:13 by Michael Thayer, 13 years ago

Don't worry, I think I have fixed this locally. More to follow.

comment:14 by Michael Thayer, 13 years ago

I have uploaded a version of VirtualBox containing a hopefully fixed copy of rdesktop-vrdp. This is a test build from the stable branch of VirtualBox (disclaimer here). It is a distribution-agnostic build with a shell script installer. To unpack it without installing, do

$ ./VirtualBox-*.run --keep --noexec

which will create a subdirectory called "install" in the current directory with the installer files, including a tar.bz2 containing the actual executables. The URL for the test build is

http://www.virtualbox.org/download/testcase/VirtualBox-2011-01-13-16-01-33-lin64-rel-4.0.1-r69417.run

comment:15 by Paul Kronenwetter, 13 years ago

My testing shows that this looks like it fixed it. I've pulled and re-inserted the scanner a few times, no crashes. Thanks!

comment:16 by Michael Thayer, 13 years ago

Summary: rdesktop-vrdp crashes with memory corruption - regression from 3.2.12-68302rdesktop-vrdp crashes with memory corruption - regression from 3.2.12-68302 -> fixed as of 14 Jan 2011

Thanks for testing.

comment:17 by Frank Mehnert, 13 years ago

Resolution: fixed
Status: newclosed
Note: See TracTickets for help on using tickets.

© 2023 Oracle
ContactPrivacy policyTerms of Use