Wrong IPC socket being deleted when VirtualBox is run via su on Linux, leading to possible disk corruption in VMs => fixed in svn

[Busan15]

This is related to ticket #17029 which has the same symptoms, but I'm not sure if it is caused by the same issue.

VirtualBox, via src/VBox/Installer/linux/VBox.sh, checks for the presence of a VBoxSVC process owned by the current user and will delete a user-specific directory in /tmp if no process is found. But the process check uses the "whoami" command to get the current username and the deletion uses the environment variables $LOGNAME or $USER. Certain implementations of su, such as the one from util-linux, does not update $LOGNAME or $USER when switching to the root user, but whoami will return "root". Thus, if a user named "vmuser" runs VirtualBox through su, it will search for processes owned by root but delete the socket belonging to vmuser.

The removal of these files will then lead to bad side effects for VMs that were previously running, as future calls to VirtualBox will create a new IPC socket in /tmp. When using this new socket, all VMs will be shown as being powered off when they might still be running. It is also possible to start another instance of a running VM, which leads to disk corruption when both instances attempt to write to the same disk image.

Not all implementations of su seem to behave in this way, and using sudo or "sudo su" can sometimes cause $LOGNAME and $USER to be changed to "root", mitigating the issue. Using su by itself seems to have the highest chance of triggering this bug.

Replication steps:

1. Start any VM using "vboxmanage startvm <VM_NAME> --type headless" (take a snapshot beforehand in case of disk corruption).
2. Check that two files called "ipcd" and "lock" are present using "ls -l /tmp/.vbox-$USER-ipc".
3. Try to start the same VM again, this should fail with a message that the VM is already locked by a session.
4. Switch to the root account using "su" and run "vboxmanage list vms".
5. Exit the root shell and run "ls -l /tmp/.vbox-$USER-ipc" again to see that the files from step #2 are now missing.
6. Try to start the VM with the same command as in step #1, this should now succeed.
7. Run "ps aux | grep -i vboxheadless" and see that there are now two processes started for the same VM.

The expected behavior would be that the files in /tmp/.vbox-$USER-ipc/ remain untouched and that step #6 should refuse to start another VM instance.

While this issue only seems to occur when su does not update $LOGNAME and $USER, I'd argue that either whoami or $LOGNAME/$USER should be used instead of having a mix of both. While running VirtualBox as root is not recommended, it would be nice if accidentally running "vboxmanage list vms" as root did not immediately break existing IPC sessions. I have attached a small patch to replace the use of whoami with $LOGNAME/$USER, and hereby make this modification available under the MIT license.

The VirtualBox log files are unlikely to contain any relevant information about this bug since it occurs in the initial shell wrapper script, but I can attach them if needed. Here are the results of running the replication steps on an unpatched installation (tested on Ubuntu 22.04 using VirtualBox 6.1.34):

$ vboxmanage startvm test_VM --type headless
Waiting for VM "test_VM" to power on...
VM "test_VM" has been successfully started.
$ ls -l /tmp/.vbox-$USER-ipc
total 4
srwx------ 1 vmuser vmuser 0 May  5 11:21 ipcd
-rw------- 1 vmuser vmuser 6 May  5 11:21 lock
$ vboxmanage startvm test_VM --type headless
VBoxManage: error: The machine 'test_VM' is already locked by a session (or being locked or unlocked)
VBoxManage: error: Details: code VBOX_E_INVALID_OBJECT_STATE (0x80bb0007), component MachineWrap, interface IMachine, callee nsISupports
VBoxManage: error: Context: "LaunchVMProcess(a->session, sessionType.raw(), ComSafeArrayAsInParam(aBstrEnv), progress.asOutParam())" at line 726 of file VBoxManageMisc.cpp
$ su
Password: 
# vboxmanage list vms
"<inaccessible>" {7abbc342-2595-4b5d-b13b-4f57b7d4d367}
"<inaccessible>" {26b6e251-e3c9-4004-a23e-30e5dd55f652}
"<inaccessible>" {ed047e7d-83db-46b6-8d09-cd99f05f7362}
"<inaccessible>" {2256db63-2e33-40fd-9182-7631a68a9e64}
"<inaccessible>" {bf2390e8-3978-4363-9792-e370986ce5fc}
# exit
$ ls -l /tmp/.vbox-$USER-ipc
/usr/bin/ls: cannot access '/tmp/.vbox-vmuser-ipc': No such file or directory
$ vboxmanage startvm test_VM --type headless
Waiting for VM "test_VM" to power on...
VM "test_VM" has been successfully started.
$ ps aux | grep -i vboxheadless
vmuser     46799  2.2  0.6 1757468 431480 ?      Sl   11:38   0:01 /opt/VirtualBox/VBoxHeadless --comment test_VM --startvm a179e065-a0b4-4fc2-a55a-d59b599aad1e --vrde config
vmuser     47353 59.8  0.2 1482004 157324 ?      Sl   11:39   0:03 /opt/VirtualBox/VBoxHeadless --comment test_VM --startvm a179e065-a0b4-4fc2-a55a-d59b599aad1e --vrde config

[Busan15]

Patch to fix user check in IPC socket cleanup

[galitsyn]

Hi Busan15,

We have a fix for the issue. Could you please give it a try to on of the "Latest 7.0.x test builds" from test builds page?

[Busan15]

Hello,

Apologies for the delayed response, my email notifications didn't seem to be enabled correctly. I can confirm that the issue is fixed in the latest test build (7.0.x revision 162549) and the 7.0.16 release. Thank you!