Guest Additions video and mouse integration fail to funciton on Solaris10u6 64-bit Guest => Fixed in SVN

[Timothy C. Howard]

Configuration: Using Solaris 10 u6 (1008) 64-bit guest (zfs file system) on Windows Vista Ultimate 64-bit Host running on 2.4 GHZ Core 2 Quad core (Q6600) in Shuttle SX38P2 with 8GB RAM. Windows Vista install is fully patched by MS Update. Using xVM VirtualBox v2.1.2 installed over a previous xVM VirtualBox v2.1.0 install. Guest set with 2048MB base memory, 128MB video memory, ACPI enabled, IO ACPI enabled, VT-x Enabled, Nested Paging disabled, PAE/NX disabled, 3D acc disabled, NIC set to Host Interface.

Steps performed and errors noted ----

1) Installed Solaris (full OEM install on zfs file system on 40GB compressible disk) - no problems occured;
2) Updated resolv.conf & nsswitch.conf to allow DNS resolution registered installation - network/internet access functional; 
3) Rebooted - login screen displayed; 
4) Registered system with Sun - successful;
5) Updated install via Sun update service - successful;
6) Rebooted - login screen displayed;
7) Logged in as root and shut down system;
8) Replicated virtual disk with VBoxManage cloneHD;
9) Created new virtual machine with replicated vdi file;
10) Booted new system;
11) Installed Guest additions - no errors indicated in output of package install;
12) Rebooted - Xserver failed to start;
13) Performed maintenance login, examined Xserver log, indicated no modes specified in xorg.conf, examined /etc/X11/xorg.conf and found no modes specified, edited /etc/X11/xorg.conf to add modes for display (based on a fully functional Solaris 10 u6 32-bit VBox guest machine);
14) Reboot -  login screen displayed;
15) Logged in as root - Video and mouse integration not functional;
16) Examined Xorg.0.log file - found multiple occurances of the following errors:

(EE) VBoxVideo(0): Failed to obtain the last resolution requested by the guest, rc= -36

(EE) VBoxVideo(0): Unable to activate VirtualBox graphics acceleration - the request to the virtual machine failed. You may be running an old version of VirtualBox.

(EE) VBoxVideo(0): Unable to determine whether the virtual machine supports mouse pointer integration - request initialization failed with return code -36

Error querying host mouse position! rc = -38

17) Tried various combinations of enabling and disabling of IO APIC, 3D acceleration with same results 18) Based on previous experience with 32bit Solaris, have noted that not all components are actually 64bit, so replaced /opt/VirtualBoxAddtions/amd64 contents with 32bit versions and received virtually identical output in Xorg.0.log file - same results. 19) Replicated original disk again (multiple times) and tried installing previous Guest Additions, each on a new clean replication (v2.1.0, v2.0.6, v2.0.4, v2.0.2) - same result

Note that this appears to be a problem with 64bit Solaris 10 guests only. I have created several 32bit Solaris 10 guests with guest additions on which video and mouse integration works fine. I have tried this multiple times with 64bit Solaris 10 and always had the same failure (hence, why I have started replicating the vdi file after the initial install - I got tired of redoing the install everytime the guest additions trashed the machine).

[Timothy C. Howard]

After reading a few other Bug reports decided to attempt to boot virtual machine in 32bit mode by disabling Vt-x. Solaris booted in 32bit mode and video and mouse integration was functional. Rebooted virtual machine in 64bit mode by enabling Vt-x and video and mouse integration was again non-functional.

[Ramshankar Venkataraman]

Could you please provide the /var/adm/messages file from the guest? (Namely all lines with 'vbox' in it, grep -i vbox)

[Nikolay Botev]

I have the exact same problem with solaris 10 u6 64-bit running on virtualbox 2.1.4, guest additions 1.4 I get the same error messages and behavior.

[Klaus-Dieter Mirsch]

Same problem on Mac OS X (10.5.6) host

[Ramshankar Venkataraman]

Yes this is a problem with Solaris 64-bit loading 32-bit Xorg binaries and VirtualBox 64-bit driver expecting on 64-bit binary interaction, And this has been fixed internally in SVN. Thank you for the report.