Host crash: vboxdrv crashes, when run in VMX mode, together with coLinux

[Technologov]

Hello,

Host: Windows XP, SP2, 32-bit, Intel Core 2 Quad, VirtualBox 2.2.0

Guest: Ubuntu 8.04, 32-bit

I decided to try Ubuntu 8.04 - in both "coLinux" mode and "virtualized" mode side-by-side.

For "coLinux" mode I used "Portable Ubuntu for Windows" distro. (​http://portableubuntu.demonccc.com.ar/)

For "virtualized" mode I used VirtualBox 2.2.0. with VT-x enabled.

Running those two solutions together at the same time results in Host Crash with BSOD, if VMX is used by VirtualBox. If running software virtualization, no crash occur.

I cannot blame VirtualBox for it, as I don't know who is responsible for crash, but it would be helpful if VirtualBox checked whenever it is safe to enter VMX mode.

Due to crash, no logs were generated.

-Technologov, 14.4.2009.

[Sander van Leeuwen]

We do check if it's safe to enable VT-x. I don't know what this coLinux does though.

[Technologov]

Could we check what else could be of conflict ? which resources ? (assuming coLinux doesn't enters VMX mode)

[Technologov]

Host crash deserves higher priority in my opinion.

Added /var/log/messages from coLinux -0.7.4-rc2 before crash.

-Technologov

[Technologov]

coLinux 0.7.4 rc2 -- /var/log/messages -- before crash

[Technologov]

To summarize:

The Good news:

I have added /var/log/messages from coLinux -0.7.4-rc2 before crash.

The Bad news:

Crash is so hard that no core dumps get generated. Mini dumps generation is set, but there are no mini-dumps from year 2009 exists in "\WINDOWS\Minidump".

..and The ugly:

I have no idea what to do next.

-Technologov

[Technologov]

We have opened a new bug report at coLinux bugzilla: ​coLinux bug report

-Technologov

[Sander van Leeuwen]

Same happens with MS VirtualPC apparently. See ​http://sourceforge.net/tracker/?func=detail&atid=622063&aid=1959846&group_id=98788

So it looks like it's their bug or the coLinux driver does things that are not allowed in VMX root mode. Either way not much we can do about it.

[Sander van Leeuwen]

coLinux can easily see if VMX root is active on the CPU by checking X86_CR4_VMXE (bit 13 in CR4).

[Henry N.]

Replying to sandervl73:

coLinux can easily see if VMX root is active on the CPU by checking X86_CR4_VMXE (bit 13 in CR4).

Thanks sandervl73 for pointing to this bit. Checking X86_CR4_VMXE would help to avoid crashing the host. But this does not help to running coLinux. While switching from Windows host to Linux Guest coLinux needs to clear CR4 and disable paging. Both would GP-fault the host, if VMX is running.

Can we disable VMX for short time while the coLinux machine is running, and can we restore the VMX running state later?

[Technologov]

sandervl73: HenryN is a core developer of coLinux.

HenryN: sandervl73 is a core developer of VirtualBox.

I really hope two Open-Source drivers needs to "co"-exists together, and find status-quo. All-in-all that's where "co" in coLinux comes from in the first place, isn't it?

-Technologov

[Sander van Leeuwen]

You can disable VMX root mode if you want to, but you'll have to restore it exactly as it was. Unfortunately Intel designed VT-x in this way; no such issues exist with AMD-V.

We perform similar checks to see if somebody else has activated VMX root mode and simply refuse to run in that case. VirtualBox only activates VMX root mode when it's running VMs, so refusing to run and issuing a warning might be sufficient.

[Sander van Leeuwen]

VirtualBox 3.1.0 has changed the way VMX and AMD-V are used. (at least for Windows hosts) It only turns on VMX root mode on the cpu that is executing guest code and turns it back off again afterwards.

The above mentioned conflict should no longer exist as of 3.1.2. I strongly suggest you implement the VMX root mode check anyway to prevent crashes.

[Technologov]

coLinux 0.7.5 check if someone uses VMX mode, and refuses to run.

See: ​http://colinux.wikia.com/wiki/Version_0.7.5

Anyway it may make sense to test VBox 3.1.2 together with problematic drivers (VirtualPC 5.1/2004, or coLinux 0.7.3/4)

-Technologov