XAUTHORITY is not passed along with DISPLAY from Frontends to VM process => Fixed in SVN/3.0.8

[talonso]

Changeset 4064 (also 4127) introduced passing the DISPLAY variable of the starting application to the VM process but neglected to also pass XAUTHORITY, which is needed to open the X window.

If VBoxSVC is already running without XAUTHORITY set, opening a VM via the VirtualBox GUI or VBoxManage fails ("/usr/bin/VirtualBox --startvm " does work).

Only DISPLAY is passed from the Frontends.
E.g. from /src/VBox/Frontends/VBoxManage/VBoxManage.cpp:

const char *display = RTEnvGet ("DISPLAY");
if (display)
env = Utf8StrFmt ("DISPLAY=%s", display);

Sample error messsage:

$ VBoxManage startvm IMARS                                                                                
VirtualBox Command Line Management Interface Version 3.0.4
(C) 2005-2009 Sun Microsystems, Inc.
All rights reserved.

Waiting for the remote session to open...
ERROR: Virtual machine 'IMARS' has terminated unexpectedly during startup
Details: code NS_ERROR_FAILURE (0x80004005), component Machine, interface IMachine, callee <NULL>

Of course if I disable X security if works fine:

$ xhost +                                                                                                 
access control disabled, clients can connect from any host
$ VBoxManage startvm IMARS                                                                                
VirtualBox Command Line Management Interface Version 3.0.4
(C) 2005-2009 Sun Microsystems, Inc.
All rights reserved.

Waiting for the remote session to open...
Remote session has been successfully opened.

[frank]

That result is a bit surprising as the default location of the .Xauthority file is in the user's home directory. Did you set the XAUTHORITY environment variable?

[talonso]

Here's the environment for VBoxSVC (started by an SMF service):

$ ps -ef | fgrep VBoxSVC
 talonso 29822     1   0   Sep 18 ?           6:48 /opt/VirtualBox/amd64/VBoxSVC --automate

$ pfexec pargs -e 29822
29822:  /opt/VirtualBox/amd64/VBoxSVC --automate
envp[0]: VBOX_XPCOM_HOME=/opt/VirtualBox/amd64
envp[1]: _=/opt/VirtualBox/amd64/VBoxManage
envp[2]: LANG=en_US.UTF-8
envp[3]: PATH=/usr/sbin:/usr/bin
envp[4]: PWD=/export/home/talonso
envp[5]: SHLVL=1
envp[6]: SMF_FMRI=svc:/application/virtualbox/autoboot:GSPrint
envp[7]: SMF_METHOD=start
envp[8]: SMF_RESTARTER=svc:/system/svc/restarter:default
envp[9]: SMF_ZONENAME=global
envp[10]: TZ=America/New_York
envp[11]: A__z="*SHLVL

And here is the environment I get when starting VirtualBox within a gnome session:

$ ps -ef | fgrep WorkWin2000d
 talonso  1485 23654   0   Sep 18 pts/6     195:01 /opt/VirtualBox/amd64/VirtualBox --startvm WorkWin2000d

$ pfexec pargs -e 1485
1485:   /opt/VirtualBox/amd64/VirtualBox --startvm WorkWin2000d
envp[0]: VBOX_XPCOM_HOME=/opt/VirtualBox/amd64
envp[1]: DESKTOP_STARTUP_ID=
envp[2]: _=/opt/VirtualBox/amd64/VirtualBox
envp[3]: COLORTERM=gnome-terminal
envp[4]: DBUS_SESSION_BUS_ADDRESS=unix:path=/tmp/dbus-k5XY0FZJdR,guid=003aa032840df0b3425f03ea4aac7507
envp[5]: DESKTOP_SESSION=gnome
envp[6]: DISPLAY=:1.0
envp[7]: DTSTARTIMS=False
envp[8]: EDITOR=gedit
envp[9]: GDMSESSION=gnome
envp[10]: GDM_LANG=en_US.UTF-8
envp[11]: GDM_XSERVER_LOCATION=local
envp[12]: GNOME_DESKTOP_SESSION_ID=this-is-deprecated
envp[13]: GNOME_KEYRING_SOCKET=/var/tmp/keyring-QnaJZZ/socket
envp[14]: GTK_IM_MODULE=iiim
envp[15]: GTK_RC_FILES=/etc/gtk/gtkrc:/export/home/talonso/.gtkrc-1.2-gnome2
envp[16]: G_BROKEN_FILENAMES=yes
envp[17]: G_FILENAME_ENCODING=@locale,UTF-8
envp[18]: HOME=/export/home/talonso
envp[19]: LANG=en_US.UTF-8
envp[20]: LOGNAME=talonso
envp[21]: MAIL=/var/mail/talonso
envp[22]: MANPATH=/usr/gnu/share/man:/usr/share/man:/usr/X11/share/man
envp[23]: ORBIT_SOCKETDIR=/var/tmp/orbit-talonso
envp[24]: PAGER=/usr/bin/less -ins
envp[25]: PATH=/usr/gnu/bin:/usr/bin:/usr/X11/bin:/usr/sbin:/sbin
envp[26]: PWD=/export/home/talonso
envp[27]: SESSIONTYPE=1
envp[28]: SESSION_MANAGER=local/R300:/tmp/.ICE-unix/22731
envp[29]: SHELL=/bin/ksh
envp[30]: SHLVL=1
envp[31]: SSH_AUTH_SOCK=/var/tmp/keyring-QnaJZZ/socket.ssh
envp[32]: TERM=xterm
envp[33]: USER=talonso
envp[34]: USERNAME=talonso
envp[35]: WINDOWID=54526003
envp[36]: XAUTHORITY=/tmp/.gdmAQ2RZU
envp[37]: XDG_DATA_DIRS=/usr/local/share/:/usr/share/:/usr/share/gdm/
envp[38]: XMODIFIERS=@im=iiimx
envp[39]: _AST_FEATURES=UNIVERSE - att
envp[40]: A__z="*SHLVL

[frank]

Thanks for the report and the clarification. Fixed in SVN and in the next maintenance release.