Virtual machines hang if PAE enabled

[Tom Francart]

After an upgrade from Virtualbox 3.2 to 4.0.6, Windows guests hung shortly after starting up. This happened both for Windows 7 (at the point where the moving windows startup logo should have appeared) and for Windows XP (after showing the desktop).

This problem went away when I disabled "Enable PAE/NX"

Host is Ubuntu Natty (all packages up to date on 2 May, kernel 2.6.38-8-generic-pae)

The following error message was shown in the system logs:
[ 3897.461643] BUG: unable to handle kernel NULL pointer dereference at   (null)
[ 3897.461651] IP: [<f8ca4303>] 0xf8ca4303
[ 3897.461657] *pdpt = 0000000026fc6001 *pde = 0000000000000000 
[ 3897.461659] Oops: 0000 [#3] SMP 
[ 3897.461661] last sysfs file: /sys/devices/virtual/net/vboxnet0/statistics/collisions
[ 3897.461664] Modules linked in: vboxnetadp vboxnetflt vboxdrv hidp aesni_intel cryptd aes_i586 aes_generic binfmt_misc rfcomm sco bnep l2cap dm_crypt vesafb nvidia(P) snd_hda_codec_hdmi arc4 snd_hda_codec_idt snd_hda_intel iwlagn snd_hda_codec snd_hwdep snd_pcm iwlcore pcmcia snd_seq_midi mac80211 dell_wmi ppdev sparse_keymap snd_rawmidi snd_seq_midi_event snd_seq dell_laptop usbhid dcdbas btusb uvcvideo bluetooth snd_timer hid snd_seq_device videodev yenta_socket pcmcia_rsrc pcmcia_core cfg80211 psmouse snd parport_pc serio_raw intel_ips soundcore snd_page_alloc lp parport ahci firewire_ohci libahci e1000e video firewire_core crc_itu_t sdhci_pci sdhci
[ 3897.461691] 
[ 3897.461693] Pid: 7755, comm: VirtualBox Tainted: P      D     2.6.38-8-generic-pae #42-Ubuntu Dell Inc. Latitude E6510/0N5KHN
[ 3897.461697] EIP: 0060:[<f8ca4303>] EFLAGS: 00010246 CPU: 1
[ 3897.461700] EIP is at 0xf8ca4303
[ 3897.461701] EAX: 00000000 EBX: e0afbdec ECX: c0103124 EDX: 00000000
[ 3897.461702] ESI: 00000001 EDI: 0000280a EBP: e0afbdcc ESP: e0afbda4
[ 3897.461703]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 3897.461705] Process VirtualBox (pid: 7755, ti=e0afa000 task=e6f98000 task.ti=e0afa000)
[ 3897.461706] Stack:
[ 3897.461707]  fa6ba000 e0afbdc0 e0afbe0c f8cbe327 fa69f000 e0afbdec ffffffff 00000000
[ 3897.461711]  00000004 fa6ba000 e0afbe0c f8c7cbc6 e0afbdec fa6ba000 00000000 00000000
[ 3897.461714]  e0afbe24 c1042325 00a09c4e 00000000 00989680 e7b99900 00000000 00327000
[ 3897.461718] Call Trace:
[ 3897.461730]  [<c1042325>] ? update_cfs_load+0x135/0x290
[ 3897.461736]  [<c104a69f>] ? enqueue_entity+0x18f/0x2c0
[ 3897.461743]  [<c10418d5>] ? check_preempt_curr+0x65/0x90
[ 3897.461746]  [<c1053293>] ? try_to_wake_up+0x223/0x3c0
[ 3897.461761]  [<f8c353c9>] ? supdrvIOCtlFast+0x99/0xb0 [vboxdrv]
[ 3897.461768]  [<f8c32301>] ? VBoxDrvLinuxIOCtl+0x41/0x1c0 [vboxdrv]
[ 3897.461772]  [<c108b41a>] ? do_futex+0x7a/0x230
[ 3897.461778]  [<f8c322c0>] ? VBoxDrvLinuxIOCtl+0x0/0x1c0 [vboxdrv]
[ 3897.461782]  [<c1141ecb>] ? do_vfs_ioctl+0x7b/0x2e0
[ 3897.461784]  [<c11421b7>] ? sys_ioctl+0x87/0x90
[ 3897.461787]  [<c100ab5f>] ? sysenter_do_call+0x12/0x28
[ 3897.461788] Code: 00 74 a7 eb 9e 8d 76 00 55 89 e5 53 83 ec 24 8b 55 0c 8b 5d 08 8b 82 dc 36 00 00 85 c0 89 45 f4 74 1d 8b 55 10 83 e2 03 8d 14 d0 <8b> 02 8b 52 04 89 03 89 d8 89 53 04 83 c4 24 5b 5d c2 04 00 8d 
[ 3897.461808] EIP: [<f8ca4303>] 0xf8ca4303 SS:ESP 0068:e0afbda4
[ 3897.461812] CR2: 0000000000000000
[ 3897.461814] ---[ end trace 24309d039bef3ab4 ]---

[Frank Mehnert]

Please could you attach a VBox.log file for such a VM session with PAE enabled?

[Tom Francart]

Vbox.log of failed Win 7 session (killed after 5min inactivity)

[Frank Mehnert]

Reproduced (Natty host, WinXP guest with 2048MB guest RAM).

[Markus Duft]

i have a very similar problem, but seemingly unrelated to PAE (not sure though, i cannot re-test right now). the virtual machine hangs so that only killing the process helps. i'll attach a log file for the session i had to kill.

i think it's a similar problem, because the kernel log shows this:

May  9 08:35:11 s01en22 kernel: [  416.092646] BUG: unable to handle kernel NULL pointer dereference at 00000004
May  9 08:35:11 s01en22 kernel: [  416.092653] IP: [<f864ee2d>] 0xf864ee2d
May  9 08:35:11 s01en22 kernel: [  416.092658] *pdpt = 0000000030df7001 *pde = 0000000000000000
May  9 08:35:11 s01en22 kernel: [  416.092661] Oops: 0000 [#2] SMP
May  9 08:35:11 s01en22 kernel: [  416.092663] last sysfs file: /sys/devices/virtual/dmi/id/product_version
May  9 08:35:11 s01en22 kernel: [  416.092666] Modules linked in: vboxnetflt vboxnetadp vboxdrv nvidia(P)
May  9 08:35:11 s01en22 kernel: [  416.092669]
May  9 08:35:11 s01en22 kernel: [  416.092672] Pid: 5260, comm: VirtualBox Tainted: P      D     2.6.37-gentoo-r4 #2 Hewlett-Packard HP Z600 Workstation/0AE8h
May  9 08:35:11 s01en22 kernel: [  416.092676] EIP: 0060:[<f864ee2d>] EFLAGS: 00010246 CPU: 7
May  9 08:35:11 s01en22 kernel: [  416.092677] EIP is at 0xf864ee2d
May  9 08:35:11 s01en22 kernel: [  416.092679] EAX: 00000000 EBX: 00000000 ECX: ffffff1c EDX: 00000001
May  9 08:35:11 s01en22 kernel: [  416.092680] ESI: ee4b3dd8 EDI: f8830000 EBP: ee4b3db0 ESP: ee4b3d98
May  9 08:35:11 s01en22 kernel: [  416.092681]  DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
May  9 08:35:11 s01en22 kernel: [  416.092683] Process VirtualBox (pid: 5260, ti=ee4b2000 task=eb66b720 task.ti=ee4b2000)
May  9 08:35:11 s01en22 kernel: [  416.092685] Stack:
May  9 08:35:11 s01en22 kernel: [  416.092686]  ee4b3dc8 f8622a18 00000000 00000000 00000000 0000280a ee4b3df0 f862c2ac
May  9 08:35:11 s01en22 kernel: [  416.092689]  ee4b3dd8 f8830000 00000000 00000001 0000008b ae3b7000 00000000 00000001
May  9 08:35:11 s01en22 kernel: [  416.092693]  f8830000 00000000 ee4b3e40 00000000 00185000 00000000 ee4b3e40 f862cdcf
May  9 08:35:11 s01en22 kernel: [  416.092764] Call Trace:
May  9 08:35:11 s01en22 kernel: [  416.092773]  [<c102c166>] ? select_idle_sibling+0x66/0x120
May  9 08:35:11 s01en22 kernel: [  416.092777]  [<c102af02>] ? enqueue_task+0x32/0x90
May  9 08:35:11 s01en22 kernel: [  416.092787]  [<f9b65bf8>] ? rtR0MemAllocEx+0x68/0xd0 [vboxdrv]
May  9 08:35:11 s01en22 kernel: [  416.092793]  [<f9b5c921>] ? supdrvIOCtlFast+0x91/0x3a0 [vboxdrv]
May  9 08:35:11 s01en22 kernel: [  416.092800]  [<f9b5c2cf>] ? SUPDrvLinuxIDC+0xbf/0x260 [vboxdrv]
May  9 08:35:11 s01en22 kernel: [  416.092803]  [<c10606e5>] ? do_futex+0x85/0xa40
May  9 08:35:11 s01en22 kernel: [  416.092807]  [<f9b5c290>] ? SUPDrvLinuxIDC+0x80/0x260 [vboxdrv]
May  9 08:35:11 s01en22 kernel: [  416.092810]  [<c10ca24e>] ? do_vfs_ioctl+0x8e/0x640
May  9 08:35:11 s01en22 kernel: [  416.092815]  [<c105d66e>] ? tick_program_event+0x1e/0x30
May  9 08:35:11 s01en22 kernel: [  416.092819]  [<c1052f86>] ? hrtimer_interrupt+0x1a6/0x2b0
May  9 08:35:11 s01en22 kernel: [  416.092821]  [<c1061116>] ? sys_futex+0x76/0x140
May  9 08:35:11 s01en22 kernel: [  416.092823]  [<c10ca876>] ? sys_ioctl+0x76/0x90
May  9 08:35:11 s01en22 kernel: [  416.092826]  [<c1002cd0>] ? sysenter_do_call+0x12/0x26
May  9 08:35:11 s01en22 kernel: [  416.092830]  [<c151007b>] ? i801_remove+0x32/0x53
May  9 08:35:11 s01en22 kernel: [  416.092831] Code: 00 00 00 00 e9 3e ff ff ff 90 55 89 e5 56 53 83 ec 10 8b 55 0c 8b 82 dc 36 00 00 85 c0 8b 75 08 8b 5d 10 89 45 f4 74 1a 83 e3 03 <8b> 54 d8 04 8b 04 d8 89 06 89 56 04 8d 65 f8 5b 89 f0 5e c9 c2
May  9 08:35:11 s01en22 kernel: [  416.092850] EIP: [<f864ee2d>] 0xf864ee2d SS:ESP 0068:ee4b3d98
May  9 08:35:11 s01en22 kernel: [  416.092852] CR2: 0000000000000004
May  9 08:35:11 s01en22 kernel: [  416.092854] ---[ end trace 1280b57fcce2755b ]---

cheers, markus

[Markus Duft]

log for (killed) hanging vm

[Frank Mehnert]

Thanks for this report. Actually this is a duplicate of #7930. Disabling either nested paging or PAE for the guest will workaround this bug.