VirtualBox

Ticket #6554 (new defect)

Opened 4 years ago

Last modified 4 years ago

rdesktop-vrdp segfaults in XPutImage on Debian Squeeze/i386

Reported by: mga Owned by:
Priority: major Component: other
Version: VirtualBox 3.1.6 Keywords:
Cc: noel.koethe@… Guest type: other
Host type: Linux

Description

I created a Debian package for rdesktop-vrdp from VirtualBox 3.1.6. It works fine on a Debian Squeeze amd64 workstation, but segfaults after login on a Debian Squeeze i386 live system I am creating.

I have localized the problem in ui_desktop_restore in xwin.c. The call to XCreateImage returns NULL sometimes and then XPutImage triggers a segmentation fault.

(gdb) bt
#0  0xb75f7d92 in XPutImage (dpy=0x9cad588, d=4194309, gc=0x9cc4688, image=0x0, req_xoffset=0, req_yoffset=0, x=184, 
    y=89, req_width=432, req_height=234) at ../../src/PutImage.c:967
#1  0x080523b8 in ui_desktop_restore (offset=0, x=184, y=89, cx=432, cy=234) at xwin.c:3595
#2  0x0806726f in process_desksave (s=0x80c0600, num_orders=43) at orders.c:384
#3  process_orders (s=0x80c0600, num_orders=43) at orders.c:1256
#4  0x0806c77e in rdp5_process (s=0x80c0600) at rdp5.c:94
#5  0x08063528 in rdp_recv (type=0xbfbb84ef "") at rdp.c:110
#6  0x08063cb4 in rdp_loop (deactivated=0xbfbb888c, ext_disc_reason=0xbfbb8888) at rdp.c:1420
#7  0x0806489c in rdp_main_loop (deactivated=0xbfbb888c, ext_disc_reason=0xbfbb8888) at rdp.c:1405
#8  0x0804df46 in main (argc=12, argv=0xbfbb8934) at rdesktop.c:994

If you need more details, please let me know.

Change History

comment:1 Changed 4 years ago by mga

Some more infos: The problem also happens with a recent trunk checkout of rdesktop and if the live system is amd64.

comment:2 Changed 4 years ago by frank

We can't reproduce this problem, tried this with Debian/unstable. The function in question (ui_desktop_restore()) is not changed  upstream. Your backtrace shows an image parameter of 0 which is probably the cause of the segfault so adding a simple sanity check might solve this problem. As your backtrace is probably not from rdesktop-vrdp I assume your backtrace was taken with Debians rdesktop.

comment:3 Changed 4 years ago by frank

  • Host type changed from other to Linux

comment:4 Changed 4 years ago by mga

No, I did use rdesktop-vrdp. I built my own package based on the tarball included with VirtualBox. But I *also* tried with upstream's trunk, same problem.

XCreateImage returns NULL if the arguments are invalid. Adding a simple sanity check will create artifacts on the screen because parts are not redrawn properly.

I have been trying to get more details with gdb, but so far it's been pretty fruitless.

comment:5 Changed 4 years ago by frank

Before trying out a fix I would like to reproduce the problem. Any idea how to reproduce the problem? The regular Lenny rdesktop should have the same problem...

comment:6 Changed 4 years ago by mga

Unfortunately, as of yet, I have no idea why the problem happens on the live system, but not on an installed Debian system. live-helper uses vanilla Debian packages, so there are no patches applied to X libraries or anything.

comment:7 Changed 4 years ago by frank

Perhaps a different video mode / video driver.

comment:8 Changed 4 years ago by mga

I now tried to use the live system including rdesktop-vrdp on real hardware (before that I used QEMU/KVM). The problem just does not exist there.

comment:9 Changed 4 years ago by mga

Debian bug describing the same problem in vanilla rdesktop:  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=437664

Note: See TracTickets for help on using tickets.

www.oracle.com
ContactPrivacy policyTerms of Use