VirtualBox

Ticket #5329 (closed defect: fixed)

Opened 4 years ago

Last modified 4 years ago

vbox claims SVME, even when in use by KVM -> fixed in SVN

Reported by: bmwiedemann Owned by:
Priority: minor Component: VMM
Version: VirtualBox 3.0.8 Keywords: SVM SVME KVM
Cc: Guest type: Linux
Host type: Linux

Description

vbox claims SVME, even when it is already in use by KVM

How to reproduce: have a CPU/system AMD VT-x (SVM) start a VM using KVM start VirtualBox ; start a VM that uses hardware virtualisation (SVM)

Actual Results: VBox VM started, but at the same time KVM VM stopped execution and could only be terminated with killall -9 qemu-kvm with some bugs visible in dmesg. A reboot was needed to get it back to a working state.

Expected Results: when a VM using HVM is actually started, availability of SVME should be checked and an error be reported to the user like "hardware virtualisation is already in use - either stop kvm or disable HVM on this VM"

maybe related to the fix of bug 3058  http://www.virtualbox.org/ticket/3058

Change History

comment:1 Changed 4 years ago by sandervl73

  • Status changed from new to closed
  • Resolution set to wontfix

VirtualBox checks if somebody else is using AMD-V. It all depends on what KVM does to initialize AMD-V. Never *ever* start two hypervisors at the same time.

comment:2 Changed 4 years ago by bmwiedemann

  • Status changed from closed to reopened
  • Resolution wontfix deleted

VirtualBox checks if somebody else is using AMD-V

I found this to not be true for version 3.0.8 and 3.0.10.

example test code I used:

# Makefile KERNELVER=uname -r KERNELDIR="/lib/modules/${KERNELVER}/build"

ifndef KERNELRELEASE all:

make -C ${KERNELDIR} M=pwd

clean:

make -C ${KERNELDIR} M=pwd clean #rm -f *.o *.ko .*.cmd rm -f modules.order Module.markers

else obj-m += svmealloc.o endif

svmealloc.c only works right with maxcpus=1 #include <linux/kernel.h> #include <linux/module.h>

#define EFER_SVME (1<<12)

static int init init_svme(void) {

uint64_t efer; int me = raw_smp_processor_id(); printk("loading svmealloc on CPU %i\n", me); rdmsrl(MSR_EFER, efer); printk("EFER=%08llX\n", efer); if (efer & EFER_SVME) {

printk("SVME already allocated by someone else\n"); return -EBUSY;

} wrmsrl(MSR_EFER, efer | EFER_SVME); rdmsrl(MSR_EFER, efer); printk("EFER=%08llX\n", efer); if(!(efer & EFER_SVME)) {

printk("allocating EFER failed\n");

} return 0;

}

static void exit cleanup_svme(void) {

uint64_t efer; int me = raw_smp_processor_id(); printk("unloading svmealloc on CPU %i\n", me); rdmsrl(MSR_EFER, efer); printk("EFER=%08llX\n", efer); if (!(efer & EFER_SVME)) {

printk("error: someone freed SVME while we used it\n");

} wrmsrl(MSR_EFER, efer & ~EFER_SVME); rdmsrl(MSR_EFER, efer); printk("EFER=%08llX\n", efer);

}

module_init(init_svme); module_exit(cleanup_svme);

# example call (with a VirtualBox VM started and stopped during the sleep): insmod ./svmealloc.ko && sleep 20 && rmmod svmealloc ; dmesg |tail # produced: loading svmealloc on CPU 0 EFER=00000D01 EFER=00001D01 unloading svmealloc on CPU 0 EFER=00000D01 error: someone freed SVME while we used it EFER=00000D01

# for comparison, dmesg where openSUSE's kvm-84 properly checks: loading svmealloc on CPU 0 EFER=00000D01 EFER=00001D01 kvm: enabling virtualization on CPU0 failed unloading svmealloc on CPU 0 EFER=00001D01 EFER=00000D01

Never *ever* start two hypervisors at the same time.

It will eventually happen by mistake (as it did to me) and you do not want VirtualBox to crash machines in those cases. Just fix the check.

comment:3 Changed 4 years ago by bmwiedemann

should have used preview...

#Makefile
KERNELVER=`uname -r`
KERNELDIR="/lib/modules/${KERNELVER}/build"

ifndef KERNELRELEASE
all:
	make -C ${KERNELDIR} M=`pwd`
clean:
	make -C ${KERNELDIR} M=`pwd` clean
	#rm -f *.o *.ko .*.cmd
	rm -f modules.order Module.markers
else
obj-m += svmealloc.o
endif
// svmealloc.c
// only works right with maxcpus=1
#include <linux/kernel.h>
#include <linux/module.h>

#define EFER_SVME (1<<12)

static int __init init_svme(void)
{
	uint64_t efer;
	int me = raw_smp_processor_id();
	printk("loading svmealloc on CPU %i\n", me);
	rdmsrl(MSR_EFER, efer);
	printk("EFER=%08llX\n", efer);
	if (efer & EFER_SVME) {
		printk("SVME already allocated by someone else\n");
		return -EBUSY;
	}
	wrmsrl(MSR_EFER, efer | EFER_SVME);
	rdmsrl(MSR_EFER, efer);
	printk("EFER=%08llX\n", efer);
	if(!(efer & EFER_SVME)) {
		printk("allocating EFER failed\n");
	}
	return 0;
}

static void __exit cleanup_svme(void)
{
	uint64_t efer;
	int me = raw_smp_processor_id();
	printk("unloading svmealloc on CPU %i\n", me);
	rdmsrl(MSR_EFER, efer);
	printk("EFER=%08llX\n", efer);
	if (!(efer & EFER_SVME)) {
		printk("error: someone freed SVME while we used it\n");
	} 
	wrmsrl(MSR_EFER, efer & ~EFER_SVME);
	rdmsrl(MSR_EFER, efer);
	printk("EFER=%08llX\n", efer);
}

module_init(init_svme);
module_exit(cleanup_svme);

comment:4 Changed 4 years ago by sandervl73

Check again with 3.0.10. Should fail there.

comment:5 Changed 4 years ago by sandervl73

  • Status changed from reopened to closed
  • Resolution set to fixed

comment:6 Changed 4 years ago by bmwiedemann

  • Status changed from closed to reopened
  • Resolution fixed deleted

retested this today and it is still what I wrote three days ago

VirtualBox checks if somebody else is using AMD-V

I found this to not be true for version 3.0.8 and 3.0.10.

so to summarize:

  • start VBox, insmod svmealloc.ko fails nicely (-EBUSY)
  • start VBox, start kvm fails nicely
  • start kvm, insmod svmealloc.ko fails nicely
  • insmod svmealloc.ko, start kvm fails nicely
  • insmod svmealloc.ko, start VBox, stop VBox shows "error: someone freed SVME while we used it" on rmmod svmealloc
  • insmod svmealloc.ko, start VBox, rmmod svmealloc causes VBox to hang because hardware virtualisation is suddenly unavailable

(* start kvm, start VBox causes kvm to hang because hardware virtualisation is suddenly unavailable)

conclusion: VBox allocates/uses SVME, even when in use by someone else before, thus failing badly.

comment:7 Changed 4 years ago by sandervl73

The code checks for it and the failure paths look fine. Could you provide a VBox.log file for the failed experiment above? I can't repeat your experiment as I don't run Linux.

comment:8 Changed 4 years ago by bmwiedemann

 http://www.zq1.de/~bernhard/temp/Logs/VBox.log.2 this is from just start+stop

 http://www.zq1.de/~bernhard/temp/Logs/VBox.log.1 is from case 5 above and virtually identical to the above

 http://www.zq1.de/~bernhard/temp/Logs/VBox.log is from case 6

 http://www.zq1.de/~bernhard/temp/Logs/dmesg including my .ko output and vboxdrv failure

btw: the case 7 (kvm) fail message also starts with "invalid opcode"

another sidenote: I tried to locate the SVM-check but could not find where/how the ASMRdMsr/ASMWrMsr function of /usr/share/virtualbox/src/vboxdrv are used. Documentation pointers welcome.

comment:9 Changed 4 years ago by sandervl73

  • Summary changed from vbox claims SVME, even when in use by KVM to vbox claims SVME, even when in use by KVM -> fixed in SVN

Ok, found the problem. Fixed in SVN now.

You were looking at the wrong source code. The AMD-V code is not in vboxdrv, but in src/vbox/vmm.

comment:10 Changed 4 years ago by bmwiedemann

I can see the fix now in r24368. And I saw, that this was introduced with r15987:15989 dated 2009-01-16 11:50:47 - exactly the time of bug #3058 that I linked in the top post.

great.

comment:11 Changed 4 years ago by frank

  • Status changed from reopened to closed
  • Resolution set to fixed

The fix is in VBox 3.1.0, please reopen if this problem persists with this version.

Note: See TracTickets for help on using tickets.

www.oracle.com
ContactPrivacy policyTerms of Use