[vbox-dev] 3.1.2 GA vboxmouse driver for x11 calculates erroneous mouse coordinates on Linux Lenny (64 bit) with server 1.4.2
carjay at gmx.net
Mon Dec 28 14:22:46 PST 2009
I decided to play a bit with Virtualbox using a 64 bit guest on an
x86_64 Bit host.
Host is running (K)ubuntu Karmic 64 bit and on a whim I used Debian
Lenny 64 bit (net installer) for the guest.
Virtualbox was latest public svn (OSE) with no modifications.
After installing the official ISO guest additions (3.1.2) and starting
gdm/kdm I noticed the mouse wasn't working correctly and seemed stuck in
the bottom right corner.
I couldn't find any hints on what was causing this in the forums and/or
bug tracker so I decided to hunt it down by building the
vboxmouse-driver and stepping through the callbacks. I found out that
the incorrect results came from "xf86ScaleAxis" which is called by
"VBoxReadInput". This function maps a value in an input range to the
according one in a given output range. But here it was always clamped to
the maximum output value even though the input was valid. I then
analysed the xorg-server sources to see how "xf86ScaleAxis" was implemented.
It turns out that the prototype of xf86ScaleAxis is "extern _X_EXPORT
int xf86ScaleAxis(int Cx, int Sxhigh, int Sxlow, int Rxhigh, int Rxlow)"
while Virtualbox uses "xf86ScaleAxis(cx, 0,
screenInfo.screens->width, 0, 65536)". Carefully checking the input
parameter names shows that they are obviously used in reverse regarding
minimum and maximum value.
In theory this should not work but obviously it did so checking the git
commitlog of the xorg server showed that the "xf86ScaleAxis"-function
(implemented in xorg-server/hw/xfree86/common/xf86Xinput.c) was broken
for several years because it got the final clamp check backwards.
Finally this was fixed in xorg-server commit
a3a7c12fcf8e4ac1418f9ea53f76091f309a721b on 2008-06-08, preceding 1.6.0.
Now, Lenny uses server 1.4.2 which predates this change so it shouldn't
be affected. But the Debian folks backported this patch among others to
2:1.4.2-9 so that's the reason why it fails to work now even on a 1.4
Going by the commit log of Virtualbox the newer driver (vboxmouse_15.c)
was only extended to work for 1.3 and 1.4 by the end of November 2009,
so it didn't affect people before (and testing the 3.1.0 GAs confirms
this, they were still working fine). Also, "VBoxReadInput" uses
xf86ScaleAxis in a conditional way (it's #ifdef'ed and the comment
reads: "Bug in the 1.4 X server series - conversion_proc was no longer
called, but the server didn't yet do the conversion itself") so it
obviously won't get used on later versions of the xorg server.
There is another place in the Virtualbox driver where "xf86ScaleAxis" is
used ("VBoxConvert") but from the commit I take it that this callback
will only get used on 1.3 because it obviously is not called on 1.4 (see
comment in last paragraph) and is no longer necessary for >=1.5.
So things would still be fine if not for Debian backporting the 1.6
changes to 1.4 which leads to vboxmouse_15.c using the parameters in a
wrong order for "xf86ScaleAxis" on systems that use this patched version.
Avoiding any further headaches in trying to detect this situation my
solution was to simply replace "xf86ScaleAxis" by a new function
"VBoxScale" in vboxmouse_15.c.
In case anyone is interested I have attached this as a patch. It's
licensed under MIT, in case it's needed.
It also fixes another bug due to missing brackets (the #ifdef lines for
1.4 were inserted without noticing that the preceding if-condition only
covers one line due to missing brackets, this will only show up in case
of an error and even then only on 1.4 servers since for all other cases
there is still only one line after the if)
Note that I didn't test this on any other server than the Lenny one yet
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 3281 bytes
Desc: not available
Url : http://www.virtualbox.org/pipermail/vbox-dev/attachments/20091228/2813d1e4/attachment-0001.bin
More information about the vbox-dev