64-bit guest support is broken on Vista x64 with AMD Phenom CPU -> fixed in SVN/3.0.4

[Julien Pierre]

I am using the latest vbox 2.2.2 . My host is Vista x64 SP2 (SP2, RTM). I have tried several x64 guest OSes.

• Opensolaris 2008.11 . It won't boot in 64-bit mode from the boot CD. It results in an OSOL kernel panic. It does boot in 32-bit mode if I disable AMD-V in vbox.

• Windows 7 release candidate . It gets stuck at "starting windows" when booting the install DVD.

The host machine where I'm seeing this has an AMD Phenom X4 9750 CPU. This is the 2.4 GHz 95W version (low-power). The motherboard is XFX 750a with nVidia 750a chipset. I have "secure virtual machine" enabled in the BIOS - if I disable it, virtual box complains when trying to run 64-bit guests. I have 8 GB DDR2 800 RAM.

I am pretty sure this is a problem in virtual box related to the AMD CPU, maybe with AMD-V.

I have another machine which is intel based (AMD Q6600), also running Vista x64 SP2, also with 8 GB of RAM, with Intel P35 chipset. On that host, both of the above x64 OS'es boot fine as virtualbox guests. In fact I first ran into the problem when copying the hard drive VDI files from the Intel host machine to the AMD host machine, and they didn't work on the later. Then I tried with the install images for these OSes, and they didn't work either on the AMD-based host.

[Sander van Leeuwen]

Without a VBox.log of such a session there's nothing we can say.

[Julien Pierre]

Log of trying to boot OpenSolaris image CD in 64 bits mode

[Julien Pierre]

Log of trying to boot Win7 RC image DVD in x64 mode

[Julien Pierre]

Log of trying to boot Solaris 10 update 6 x64 previously installed on VDI made on Intel box

[Julien Pierre]

Log of trying to boot image DVD of Vista x64 SP1 . It blue screened during boot, then I powered down the VM

[Julien Pierre]

I have attached 4 logs of different x64 operating systems. None of them run as guests on my AMD-based Phenom 9750 64-bit Windows host. All of them work as guests on my Intel-based Q6600 64-bit Windows host.

[Julien Pierre]

This problem is specific to the Windows x64 host with AMD CPU. On my AMD system, I have a dual boot between Vista x64 SP2 and Opensolaris 2008.11 . I just installed virtualbox 2.2.2 in Opensolaris on my AMD host . All the x64 guests OSes that I have tried are working fine under it : Opensolaris 2008.11, Windows 7 RC x64, Vista x64 SP1 . The install CD/DVDs booted fine as guests, and I have them all installing at the same time in different VMs right now.

I think this bug may also be related to ticket 3683 .

[Frank Mehnert]

Is your CPU nested paging capable? If so, does it make any difference when you activate that setting for a VM?

[Julien Pierre]

Yes, the Phenom is nested paging capable. I enabled it, and it solves the problems with the Windows host and x64 guests. Thanks !

What's strange is that on the same machine, when I boot to the Opensolaris 2008.11 host, the same guest OSes are able to run without nested paging.

[Sander van Leeuwen]

Confirmed. Seeing the same here with a Vista64 SP1 host + Win7 RC guest. Enabling nested paging solves the problem. I'll have a closer look.

[Sander van Leeuwen]

Found and fixed the problem. The bug is timing sensitive, so it may not appear on another machine or even every time.