Ticket #4032 (new enhancement)

Opened 7 years ago

Last modified 2 weeks ago

[feature-request] Nested Virtualization: VT-in-VT

Reported by: Technologov Owned by:
Priority: minor Component: VMM
Version: VirtualBox 2.2.2 Keywords:
Cc: Guest type: Windows
Host type: other

Description (last modified by frank) (diff)

Hi All,

Because more and more virtualization software requires VT (Intel VMX or AMD-V) to run, it makes sense to virtualize it somehow. (at least if host CPU supports hardware VT)

It will allow the following software to run in guests: -Xen (full virtualization mode) -KVM -Hyper-V -Windows Virtual PC -- this is basically a requirement for those who wanna run Windows 7 "XP mode". i.e. It allows for more complete user experience on Win7 guests.

Others can take advantage of it too: -VirtualBox itself :) -VMware Workstation -VMware ESX

-Technologov (yes, I will take advantage of it too :-), yay! )

Change History

comment:1 Changed 7 years ago by Technologov

opened on 17.5.2009.

comment:2 follow-up: ↓ 38 Changed 7 years ago by sandervl73

  • Priority changed from major to minor

A lot of work for questionable usefulness. Definitely very low on our priority list.

comment:3 Changed 6 years ago by Technologov

This is partially related to wish #2988


comment:4 Changed 6 years ago by Technologov

KVM team has developed (untested) patches for both nested VMX and nested SVM. Maybe some ideas can be taken from those patches.

See: (General Info)



comment:5 Changed 4 years ago by Technologov

update: VMware Workstation 8 claims to support this feature !


comment:6 Changed 4 years ago by Technologov

It seems that Linux 3.1 KVM finally achieved nested VMX.

This feature is yet another filler feature that is a "must-have":

Research paper:


comment:7 follow-up: ↓ 8 Changed 4 years ago by frank

What you denote as "must have" needs a lot of implementation time, is very difficult to implement and the benefit of nested virtualization is questionable.

comment:8 in reply to: ↑ 7 Changed 4 years ago by Johan Ryberg

Replying to frank:

What you denote as "must have" needs a lot of implementation time, is very difficult to implement and the benefit of nested virtualization is questionable.

I would very much see this in Virtualbox. I was just about to test a VMware cluster setup and I wanted to use VT-in-VT for this but it was not possible to use hardware virtualization.

Best regards Johan Ryberg

Version 0, edited 4 years ago by Johan Ryberg (next)

comment:9 Changed 4 years ago by vbfun

I would also be very interested to see this in VB too. It'll allow another full fledged hypervisor to run on VB and allow 64bit guests to run in the hypervisor.

comment:10 Changed 4 years ago by DNS

This would be a very useful feature IMHO. In virtualBox, currently I could test almost any software I like, with the exception of other hypervisors. With this capability, the full potential of VirtualBox can be used. Can the devs please reconsider adding this?

Thanks, DNS

comment:11 Changed 3 years ago by Tristan

This feature is essential for development of low-level OS kernel components when your workstation operating system is not suitable - eg, running Linux in a VM, then running your OS component under KVM with gdm attached.

It's also essential for development and testing of VM provisioning systems, or of any software that otherwise interacts with VMMs.

comment:12 Changed 3 years ago by x5560

I know for a fact that VMWare Workstation 9.0 and ESXi 5.0 does infact support this. What it does, is to emulate hardware VT and tricks the guest into thinking it runs on real HW. This is very usefull if you work with hypervisors and need to test certain functionality of a hypervisor but do not actually have thousands of dollars to spend on HW for testing.

I've been using Virtualbox for years, but this is a dealbreaker for me, as i work as a sysadmin and need to be able to test this kind of stuff in my home lab. I am seriously considering to shell out the dough for a VMWare Workstation license and abandon Virualbox.

It's sad, because Virualbox has so much potential. It's the only real open source type 2 hypervisor that is a real competition for VMWare.

comment:13 Changed 3 years ago by frank

  • Description modified (diff)

I know that other virtualization products support nested virtualization. Even with VirtualBox it is possible to execute VirtualBox without VT-x/AMD-V as a guest of VirtualBox with VT-x/AMD-V enabled. But apart from that, implementing full virtualization is too much effort for too little benefit.

comment:14 Changed 3 years ago by Tsso

I also couldn't run Qubes OS in Virtual Box.

comment:15 Changed 3 years ago by itarchitectlev

It seems that this ticket was evaluated a long time ago when the usefulness may have been disputed. In any case, there are real reasons people want to have nested hardware VT.

There is a lot of working happening in the cloud community and evaluating products in this arena often requires hardware VT - which implies two things:

1) You have real hardware available 2) You have nested hardware VT capable virtualization software (e.g. VMware)

Given that we are in a world of cheap memory coupled with multiple core fast CPUs, virtualization is a great boost to doing cloud infrastructure development work. The problem is that the virtualized hardware created under VBox is only one part of the picture - as you're creating virtual hardware to run infrastructure designed to virtualize. Nested Hardware VT is essential here as software emulation in this world is painfully slow.

I have just purchased a VMware product because VirtualBox doesn't support this fundamental feature that I require which will now render my use of VirtualBox redundant.

I have been a big advocate of VirtualBox, but future work now sees this being left behind.

Regards, Kev @itarchitectkev

comment:16 follow-up: ↓ 17 Changed 3 years ago by frank

We didn't discuss against the usefulness of nested virtualization in general. But it remains valid that this is currently too much effort for too few benefit.

comment:17 in reply to: ↑ 16 Changed 3 years ago by denka_b

Replying to frank:

We didn't discuss against the usefulness of nested virtualization in general. But it remains valid that this is currently too much effort for too few benefit.

How is the benefit estimated, and is there anything different in VBox architecture from VMware that makes it feasible there but not here?

I'm in same boat as many, forced to go VMware way because Hyper-V in Windows 8 (guest) is required for Windows Phone 8 emulator to work. I'm not skimping on upgrade, but there is no upgrade path from Windows 7 Pro to Windows 8 :(

Stupid Microsoft - I guess they too have their "too much effort for too little benefit" reasons :)

comment:18 Changed 3 years ago by frank

I don't want to start a discussion here, a public bugtracker is not the right place for doing this. But please be aware that the VirtualBox development is primary driven by paying customer demands. Of course that does not mean that community demands are not being considered. We just have limited resources. Code contribution from the community may speed up the implementation of certain features.

comment:19 Changed 3 years ago by Tristan

How much would this cost? Or what is the right place for doing this - if the community were to save up to buy this feature like they did to buy blender I imagine it would need to be somewhere public.

comment:20 Changed 3 years ago by bluezeak

One of the primary uses of virtualization is for sandbox testing. Nested virtualization allows testing of other hyper visors, as well as ANY application that requires access to processor VT. This type of testing is one of the MAIN reasons people use hypervisors. Virtualbox is my favorite hypervisor, and so I want to see this capability added. If it isn't, I will be required to use other hypervisors that do have it. KVM has it now. And VMware Workstation has it. I will be willing to pay for vmware workstation ... if I have to. I would RATHER get it from VirtualBox.

comment:21 Changed 23 months ago by peterdk

I would love to see this implemented.

My use case: I am a Mobile App developer, and use Ubuntu as my main OS. To be able to develop for Windows Phone I need to run Windows 8 with HyperV enabled (otherwise I can't run WP emulators). Since HyperV requires VT-X, I can't use Virtualbox to run a virtual WP dev environment. Now I need to dualboot.

comment:22 Changed 19 months ago by snowch

This feature would be very useful. Running a OpenStack development environment inside virtualbox falls back to Qemu instead of Kvm making it very slow.

comment:23 Changed 19 months ago by Smackey's Dad

Will nested VM work using "Intel Haswell CPU" which has VMCS feature allowing nested virtualization. If it does not it makes sense (in my opinion) to add support for nested VM with New CPU VMCS feature rather than writing the emulation layer.

comment:24 Changed 19 months ago by ramshankar

No. Haswell CPUs while it has features like VMCS shadowing that help make Nested Virtualization perform better and easier to program, we still have to implement code which makes use of it. At the moment, we still feel when we implement the nested virtualization feature, having a Haswell CPU as the minimum requirement is not reasonable as it's still too new, and will exclude quite a bit of the users even two years down the line.

This feature is most definitely on our radar for a while now, but again, it's a question of managing our development resources which are limited.

Last edited 19 months ago by ramshankar (previous) (diff)

comment:25 Changed 18 months ago by rdx

Very interested in this feature. We're running Android emulators inside virtual machines, and it would be great to see them using KVM acceleration. Would be great if this could be done starting with only certain limited CPUs, and then go up from there.

comment:26 Changed 16 months ago by Bertrualex

I need this feature to develop driver that uses AMD-V. It would be great if you provide this feature

comment:27 Changed 16 months ago by EnesKorukcu

This feature is mandatory for installing openstack, or just making PoC for scalable systems. I'm looking forward to it.

comment:28 Changed 15 months ago by AfUnix

+1 for the feature. RHEV/oVirt requires nested virtualization. OpenStack highly recommends.

comment:29 in reply to: ↑ description Changed 14 months ago by dario

This would be a great addition, as I'm stuck on using Windows as my main workspace OS (enterprise environment) so VM-in-VM is essential for CI development.

comment:30 Changed 13 months ago by anrichter

+1 for this feature. I need it to develop Windows Phone Applications an an virtualized environment. With VMware is it possible. But i like VirtualBox and would use it for all my virtual environments.

comment:31 Changed 13 months ago by Miles Chen

I would really like to see this feature (aka vt-in-vt) to be implemented in the near future. Running kvm inside a Linux VM on VirtualBox will no longer be a dream!

comment:32 Changed 12 months ago by Jimbob

Please, Please, Please implement this feature, it is about the only thing that our org is sticking to its guns over keeping VM-Ware ESXi everywhere. It it had this, we could use the same VBox images everywhere! Our lives would be simple, your brand would be saved, and we would all be happy people!

+1 To implement this feature!!! Please, please, please - with a cherry on the top!!


Last edited 11 months ago by frank (previous) (diff)

comment:33 Changed 11 months ago by frank

I removed the last comment. Please stop spamming the bugtracker!

comment:34 Changed 11 months ago by adam84luong

This feature is needed to run Android virtual device with x86 emulation that require hardware accelerator. This need KVM is installed. I would love to see it ASAP.

comment:35 Changed 11 months ago by Coffee_fan

The only reason I have VMWare workstation is because of this feature. Understand it may be very hard to implement. FWIW, it is useful for all devops that want to run hypervisor on hypervisor.

comment:36 Changed 11 months ago by frank

It's not a question if this feature is useful or not. The feature is definitely useful. The only question is the benefit in relation to the effort and the comparison of this factor to other roadmap items.

comment:37 Changed 11 months ago by everflux

Any estimation how expensive this feature would be? I imagine it could perhaps get crowd funded... (at least initial development, maintenance is another topic, I know.) I would throw in up to 500 euros if that helps any way.

comment:38 in reply to: ↑ 2 Changed 10 months ago by purpletoad

Replying to sandervl73:

A lot of work for questionable usefulness. Definitely very low on our priority list.

still looking forward for this feature, been a long time user of virtualbox since version 2.2, really don't want to switch to another software, but some times lack of VT-in-VT feature really killing the job on virtualbox. would like to see this feature in your high priority list.

comment:39 Changed 10 months ago by Bernd Vogt


comment:40 Changed 8 months ago by jonseymour

The case for nested virtualisation is only going to grow stronger as tools like boot2docker become more widely deployed.

For an example of a real world scenario, I need to run boot2docker inside a Windows VM running on an OSX host in order to document the process of setting up Boot2Docker on Windows hosts. I can't do this with Virtual Box on the OSX host because boot2docker needs to use Virtual Box to create a 64bit Linux VM and the whole process won't work without nested virtualization.

Yes, I could install a Windows OS on bare metal to enable this scenario, but I really don't want to do this. The other option I have at this point is to migrate my Windows VM into VMWare because apparently this capability is supported there.

2015 isn't 2009 - the advent of lightweight containerisation and tools like boot2docker have created a strong need for VMs to be able to run VMs.


comment:41 Changed 8 months ago by ciekawy


comment:42 Changed 8 months ago by GA

Please implement this feature and stop the monopoly of ESXi. I would like to setup a Oracle VMServer dev environment within VBOX on my win7 laptop, but cannot.

comment:43 Changed 7 months ago by Mobile Geek

I too would like to see this feature. It can already do this at work with VMWare Workstation, but would rather do it with VirtualBox. The usefulness has been questioned. I can give you another use for it.

I run Windows 7 at work but need to develop software for Windows Phone 8 or later. This cannot be done in Windows 7. It has to be Windows 8 or later, because the emulators and debugging require Hyper-V, which comes in Windows 8 or later. I have no choice about the base OS I can use. At work, I have VMWare Workstation 10 with pass through virtualization, with the Windows 7 host, and a Windows 8 VM with Hyper-V enabled in the guest. It works very well. Again, I like VirtualBox better than VMWare, but cannot use it because of this limitation.

Also, Hyper-V will not play nice with other virtualization solutions on the same Host. So when I want to test something with a Hyper-V server in a distributed environment, I cannot do so with VirtualBox. Even if I get multiple virtualization solutions installed, I have to have multiple boot configurations in order to disable the hyper-visor under certain circumstances. It's really a pain.

comment:44 Changed 7 months ago by ThatOneGuySean


I'd like to do mobile development development on a VirtualBox VM using Android Studio. In order to do that I need to run HAXM.

comment:45 Changed 7 months ago by Rainmaker

  • Please ignore this response. I noticed VBox 5 had a "Paravirtualization Interface" and I mixed up Paravirtualization with Nested Virtualization -
Last edited 7 months ago by Rainmaker (previous) (diff)

comment:46 Changed 7 months ago by frank

I don't know what you mean by "lack of response". We commented several times that at the moment we are not working on implementing Nested Virtualization. This feature is definitely interesting but at the moment we have different priorities.

comment:47 Changed 6 months ago by Youness

+1 I too would love to see nested virtualization in VirtualBox.

Not only are emulators (phone, various devices in SDKs) something that force me to use a virtualization platform but it hinders from experimenting with other hypervisors such as KVM, HyperV.

Multiple virtualization platforms support this such as VMWare, KVM, Hyper-V

comment:48 Changed 6 months ago by subdian

+1 need it for phone dev

comment:49 Changed 6 months ago by frank

Deleted the last comment. tuhinchakravorty, if you want to insult people, go elsewhere. If you want help, please create separate tickets for separate problems. Your problem has nothing to do with nested virtualization, at least from what I saw from the rare facts you provided.

comment:50 Changed 5 months ago by veganaiZe


comment:51 Changed 5 months ago by JohnStarich

+1 I am in agreement with jonseymour that this is becoming a much more demanded feature in the world of cloud computing and boot2docker is also gaining ground. I use virtualization tools like Virtual Box regularly, including spinning up virtual cloud hosts. My example use-case is deploying docker-machines with a virtual box driver to create container VM hosts in its own private cloud. This is especially useful in development and system administration of virtual private clouds.

comment:52 Changed 5 months ago by rdx

+1 As another example, we want to distribute a Vagrant box with Android SDK and emulators installed, so that it becomes easy for anybody to run automated emulator tests. Without KVM inside the guest, emulation is uselessly slow. Of course, the other option is we can install Android SDK and everything in the host OS itself, but that's cumbersome and the whole android sdk is a moving target. We could resolve this and many such use cases (as others have pointed out) with VT-inside-VT. Having this feature would be very valuable...

comment:53 Changed 3 months ago by Michal

+1 (to be able to run android emulator inside vm)

comment:54 Changed 3 months ago by todd.vierling

VT-x+SLAT passthrough to allow for better simulation environments for OVM Server (Xen) would be quite useful internally.

comment:55 Changed 8 weeks ago by Mandrel Bent

+1 for this feature. I just set up a whole Linux VM on my corporate mandated Windows box to be able to build and run Cloudius OSv images (similar to Docker but qemu-kvm based), only to realize I could not start VMs within my VM. :( I have an Haswell and am glad to know this will be very fast once it is supported.

comment:56 Changed 7 weeks ago by Brian Cunnie

+1. Android Developer on Mac OS X running Linux VM for CI (continuous integration) which uses a Docker Container which will need HAXM to run an android emulator at reasonable speed. Thanks!

comment:57 Changed 7 weeks ago by emiliano.bonassi

+1. In the same situation of Brian Cunnie. A question, aren't you implement this feature because it conflicts with other Oracle's product (e.g Oracle VM Server for x86)?

comment:58 Changed 5 weeks ago by killmenot


comment:59 Changed 3 weeks ago by Thomas Dreibholz


comment:60 Changed 2 weeks ago by ccarbone

Guest type in this issue is set to Windows but I'm running on a Mac (OSX10.11.1) and I have the same issue with my VirtualBox 5.0.8. Do you have a workaround to be able to get KVM ? Do you think I should clone this issue to have one with Guest type set to Mac Thanks

comment:61 Changed 2 weeks ago by azurtree


comment:62 Changed 2 weeks ago by amclain

+1. I have a Windows host with a Linux VM I use for web development. The projects I'm working on now use Ansible + Vagrant, and since Ansible doesn't support Windows I need to be able to run the Vagrant VMs inside the Linux guest.

Note: See TracTickets for help on using tickets.
ContactPrivacy policyTerms of Use