Ticket #4032 (new enhancement)

Opened 6 years ago

Last modified 5 weeks ago

[feature-request] Nested Virtualization: VT-in-VT

Reported by: Technologov Owned by:
Priority: minor Component: VMM
Version: VirtualBox 2.2.2 Keywords:
Cc: Guest type: Windows
Host type: other

Description (last modified by frank) (diff)

Hi All,

Because more and more virtualization software requires VT (Intel VMX or AMD-V) to run, it makes sense to virtualize it somehow. (at least if host CPU supports hardware VT)

It will allow the following software to run in guests: -Xen (full virtualization mode) -KVM -Hyper-V -Windows Virtual PC -- this is basically a requirement for those who wanna run Windows 7 "XP mode". i.e. It allows for more complete user experience on Win7 guests.

Others can take advantage of it too: -VirtualBox itself :) -VMware Workstation -VMware ESX

-Technologov (yes, I will take advantage of it too :-), yay! )

Change History

comment:1 Changed 6 years ago by Technologov

opened on 17.5.2009.

comment:2 follow-up: ↓ 38 Changed 6 years ago by sandervl73

  • Priority changed from major to minor

A lot of work for questionable usefulness. Definitely very low on our priority list.

comment:3 Changed 5 years ago by Technologov

This is partially related to wish #2988


comment:4 Changed 5 years ago by Technologov

KVM team has developed (untested) patches for both nested VMX and nested SVM. Maybe some ideas can be taken from those patches.

See: (General Info)



comment:5 Changed 4 years ago by Technologov

update: VMware Workstation 8 claims to support this feature !


comment:6 Changed 3 years ago by Technologov

It seems that Linux 3.1 KVM finally achieved nested VMX.

This feature is yet another filler feature that is a "must-have":

Research paper:


comment:7 follow-up: ↓ 8 Changed 3 years ago by frank

What you denote as "must have" needs a lot of implementation time, is very difficult to implement and the benefit of nested virtualization is questionable.

comment:8 in reply to: ↑ 7 Changed 3 years ago by Johan Ryberg

Replying to frank:

What you denote as "must have" needs a lot of implementation time, is very difficult to implement and the benefit of nested virtualization is questionable.

I would very much see this in Virtualbox. I was just about to test a VMware cluster setup and I wanted to use VT-in-VT for this but it was not possible to use hardware virtualization.

Best regards Johan Ryberg

Version 0, edited 3 years ago by Johan Ryberg (next)

comment:9 Changed 3 years ago by vbfun

I would also be very interested to see this in VB too. It'll allow another full fledged hypervisor to run on VB and allow 64bit guests to run in the hypervisor.

comment:10 Changed 3 years ago by DNS

This would be a very useful feature IMHO. In virtualBox, currently I could test almost any software I like, with the exception of other hypervisors. With this capability, the full potential of VirtualBox can be used. Can the devs please reconsider adding this?

Thanks, DNS

comment:11 Changed 3 years ago by Tristan

This feature is essential for development of low-level OS kernel components when your workstation operating system is not suitable - eg, running Linux in a VM, then running your OS component under KVM with gdm attached.

It's also essential for development and testing of VM provisioning systems, or of any software that otherwise interacts with VMMs.

comment:12 Changed 2 years ago by x5560

I know for a fact that VMWare Workstation 9.0 and ESXi 5.0 does infact support this. What it does, is to emulate hardware VT and tricks the guest into thinking it runs on real HW. This is very usefull if you work with hypervisors and need to test certain functionality of a hypervisor but do not actually have thousands of dollars to spend on HW for testing.

I've been using Virtualbox for years, but this is a dealbreaker for me, as i work as a sysadmin and need to be able to test this kind of stuff in my home lab. I am seriously considering to shell out the dough for a VMWare Workstation license and abandon Virualbox.

It's sad, because Virualbox has so much potential. It's the only real open source type 2 hypervisor that is a real competition for VMWare.

comment:13 Changed 2 years ago by frank

  • Description modified (diff)

I know that other virtualization products support nested virtualization. Even with VirtualBox it is possible to execute VirtualBox without VT-x/AMD-V as a guest of VirtualBox with VT-x/AMD-V enabled. But apart from that, implementing full virtualization is too much effort for too little benefit.

comment:14 Changed 2 years ago by Tsso

I also couldn't run Qubes OS in Virtual Box.

comment:15 Changed 2 years ago by itarchitectlev

It seems that this ticket was evaluated a long time ago when the usefulness may have been disputed. In any case, there are real reasons people want to have nested hardware VT.

There is a lot of working happening in the cloud community and evaluating products in this arena often requires hardware VT - which implies two things:

1) You have real hardware available 2) You have nested hardware VT capable virtualization software (e.g. VMware)

Given that we are in a world of cheap memory coupled with multiple core fast CPUs, virtualization is a great boost to doing cloud infrastructure development work. The problem is that the virtualized hardware created under VBox is only one part of the picture - as you're creating virtual hardware to run infrastructure designed to virtualize. Nested Hardware VT is essential here as software emulation in this world is painfully slow.

I have just purchased a VMware product because VirtualBox doesn't support this fundamental feature that I require which will now render my use of VirtualBox redundant.

I have been a big advocate of VirtualBox, but future work now sees this being left behind.

Regards, Kev @itarchitectkev

comment:16 follow-up: ↓ 17 Changed 2 years ago by frank

We didn't discuss against the usefulness of nested virtualization in general. But it remains valid that this is currently too much effort for too few benefit.

comment:17 in reply to: ↑ 16 Changed 2 years ago by denka_b

Replying to frank:

We didn't discuss against the usefulness of nested virtualization in general. But it remains valid that this is currently too much effort for too few benefit.

How is the benefit estimated, and is there anything different in VBox architecture from VMware that makes it feasible there but not here?

I'm in same boat as many, forced to go VMware way because Hyper-V in Windows 8 (guest) is required for Windows Phone 8 emulator to work. I'm not skimping on upgrade, but there is no upgrade path from Windows 7 Pro to Windows 8 :(

Stupid Microsoft - I guess they too have their "too much effort for too little benefit" reasons :)

comment:18 Changed 2 years ago by frank

I don't want to start a discussion here, a public bugtracker is not the right place for doing this. But please be aware that the VirtualBox development is primary driven by paying customer demands. Of course that does not mean that community demands are not being considered. We just have limited resources. Code contribution from the community may speed up the implementation of certain features.

comment:19 Changed 2 years ago by Tristan

How much would this cost? Or what is the right place for doing this - if the community were to save up to buy this feature like they did to buy blender I imagine it would need to be somewhere public.

comment:20 Changed 23 months ago by bluezeak

One of the primary uses of virtualization is for sandbox testing. Nested virtualization allows testing of other hyper visors, as well as ANY application that requires access to processor VT. This type of testing is one of the MAIN reasons people use hypervisors. Virtualbox is my favorite hypervisor, and so I want to see this capability added. If it isn't, I will be required to use other hypervisors that do have it. KVM has it now. And VMware Workstation has it. I will be willing to pay for vmware workstation ... if I have to. I would RATHER get it from VirtualBox.

comment:21 Changed 15 months ago by peterdk

I would love to see this implemented.

My use case: I am a Mobile App developer, and use Ubuntu as my main OS. To be able to develop for Windows Phone I need to run Windows 8 with HyperV enabled (otherwise I can't run WP emulators). Since HyperV requires VT-X, I can't use Virtualbox to run a virtual WP dev environment. Now I need to dualboot.

comment:22 Changed 11 months ago by snowch

This feature would be very useful. Running a OpenStack development environment inside virtualbox falls back to Qemu instead of Kvm making it very slow.

comment:23 Changed 11 months ago by Smackey's Dad

Will nested VM work using "Intel Haswell CPU" which has VMCS feature allowing nested virtualization. If it does not it makes sense (in my opinion) to add support for nested VM with New CPU VMCS feature rather than writing the emulation layer.

comment:24 Changed 11 months ago by ramshankar

No. Haswell CPUs while it has features like VMCS shadowing that help make Nested Virtualization perform better and easier to program, we still have to implement code which makes use of it. At the moment, we still feel when we implement the nested virtualization feature, having a Haswell CPU as the minimum requirement is not reasonable as it's still too new, and will exclude quite a bit of the users even two years down the line.

This feature is most definitely on our radar for a while now, but again, it's a question of managing our development resources which are limited.

Last edited 11 months ago by ramshankar (previous) (diff)

comment:25 Changed 10 months ago by rdx

Very interested in this feature. We're running Android emulators inside virtual machines, and it would be great to see them using KVM acceleration. Would be great if this could be done starting with only certain limited CPUs, and then go up from there.

comment:26 Changed 7 months ago by Bertrualex

I need this feature to develop driver that uses AMD-V. It would be great if you provide this feature

comment:27 Changed 7 months ago by EnesKorukcu

This feature is mandatory for installing openstack, or just making PoC for scalable systems. I'm looking forward to it.

comment:28 Changed 7 months ago by AfUnix

+1 for the feature. RHEV/oVirt requires nested virtualization. OpenStack highly recommends.

comment:29 in reply to: ↑ description Changed 6 months ago by dario

This would be a great addition, as I'm stuck on using Windows as my main workspace OS (enterprise environment) so VM-in-VM is essential for CI development.

comment:30 Changed 5 months ago by anrichter

+1 for this feature. I need it to develop Windows Phone Applications an an virtualized environment. With VMware is it possible. But i like VirtualBox and would use it for all my virtual environments.

comment:31 Changed 4 months ago by Miles Chen

I would really like to see this feature (aka vt-in-vt) to be implemented in the near future. Running kvm inside a Linux VM on VirtualBox will no longer be a dream!

comment:32 Changed 4 months ago by Jimbob

Please, Please, Please implement this feature, it is about the only thing that our org is sticking to its guns over keeping VM-Ware ESXi everywhere. It it had this, we could use the same VBox images everywhere! Our lives would be simple, your brand would be saved, and we would all be happy people!

+1 To implement this feature!!! Please, please, please - with a cherry on the top!!


Last edited 3 months ago by frank (previous) (diff)

comment:33 Changed 3 months ago by frank

I removed the last comment. Please stop spamming the bugtracker!

comment:34 Changed 3 months ago by adam84luong

This feature is needed to run Android virtual device with x86 emulation that require hardware accelerator. This need KVM is installed. I would love to see it ASAP.

comment:35 Changed 2 months ago by Coffee_fan

The only reason I have VMWare workstation is because of this feature. Understand it may be very hard to implement. FWIW, it is useful for all devops that want to run hypervisor on hypervisor.

comment:36 Changed 2 months ago by frank

It's not a question if this feature is useful or not. The feature is definitely useful. The only question is the benefit in relation to the effort and the comparison of this factor to other roadmap items.

comment:37 Changed 2 months ago by everflux

Any estimation how expensive this feature would be? I imagine it could perhaps get crowd funded... (at least initial development, maintenance is another topic, I know.) I would throw in up to 500 euros if that helps any way.

comment:38 in reply to: ↑ 2 Changed 6 weeks ago by purpletoad

Replying to sandervl73:

A lot of work for questionable usefulness. Definitely very low on our priority list.

still looking forward for this feature, been a long time user of virtualbox since version 2.2, really don't want to switch to another software, but some times lack of VT-in-VT feature really killing the job on virtualbox. would like to see this feature in your high priority list.

comment:39 Changed 5 weeks ago by Bernd Vogt


Note: See TracTickets for help on using tickets.
ContactPrivacy policyTerms of Use