VirtualBox

Ticket #15439 (closed defect: fixed)

Opened 18 months ago

Last modified 17 months ago

5.0.20 + kernel 4.6.0 = Guru Meditation critical error

Reported by: HJarausch Owned by:
Priority: major Component: other
Version: VirtualBox 5.0.20 Keywords:
Cc: Guest type: Windows
Host type: Linux

Description

Hi,

I have a Window7 VDI file which works perfectly with VB 5.0.20 when I run a older kernel like 4.5.

But with the same VDI file VB crashes under kernel 4.6.0 giving VirtualBox - Guru Meditation A critical error has occurred while running the virtual machine and the machine execution has been stopped.

Attachments

VBox.log Download (346.6 KB) - added by HJarausch 18 months ago.
VBox.log
VBox.png Download (31.8 KB) - added by HJarausch 18 months ago.
VBox.png
VBox.log_Windows_7 Download (422.5 KB) - added by Platypus 18 months ago.
VBox.log_openSUSE_421 Download (233.1 KB) - added by Platypus 18 months ago.
VBox.2.log Download (177.8 KB) - added by FredBezies 18 months ago.
Crash log in Archlinux + kernel 4.6
config-archlinux-4.6-kernel Download (176.6 KB) - added by FredBezies 18 months ago.
Config file for linux kernel 4.6 in Archlinux - Hope it helps :)
VBox.log.10 Download (378.8 KB) - added by efferre 18 months ago.
config-4.6.0-gentoo Download (98.4 KB) - added by efferre 18 months ago.
host linux kernel config
config_4.6.0_Intel Download (105.0 KB) - added by Platypus 18 months ago.
VBox.3.log Download (303.5 KB) - added by FredBezies 18 months ago.
Another crash log under Archlinux with linux kernel 4.6.1
VBox.2.png Download (7.4 KB) - added by FredBezies 18 months ago.
Screenshot related to archlinux + linux 4.6.1 crash.
VBox.log-fbe-gentoo-kernel4.6.1 Download (232.7 KB) - added by fbe 18 months ago.
VirtualBox Crash Log (fbe) on Gentoo Kernel 4.6.1 while installing win7 in vm
config-kernel-4.6.1-gentoo-fbe.gz Download (22.0 KB) - added by fbe 18 months ago.
kernel config (fbe) gentoo 4.6.1 corresponding to VBox.log-fbe-gentoo-kernel4.6.1
config-4.6.0-frank Download (117.8 KB) - added by frank 18 months ago.
VBox-crash-disabled-MEMPROTKEYS-Win7VM-Gentoo-Amd64-Vanilla4.6.1.log Download (386.2 KB) - added by fbe 18 months ago.
Crash with Vanilla 4.6.1 Kernel and disabled memory protection keys
NV-CentOS7-crash.log Download (94.1 KB) - added by Anybody 18 months ago.
Crash Log of CentOS 7 on my Gentoo Kernel 4.6.0 machine with VBOX 5.0.20
NV-Gentoo-460-kernel-config Download (93.5 KB) - added by Anybody 18 months ago.
Kernel .config of my Gentoo kernel 4.6.0 machine
config_4.6.2-Intel_WORKS Download (108.8 KB) - added by Platypus 18 months ago.
VBox.log_hekarath.log Download (231.9 KB) - added by Hekarath 17 months ago.
config-4.6.0-pf1_hekarath Download (96.0 KB) - added by Hekarath 17 months ago.

Change History

Changed 18 months ago by HJarausch

VBox.log

Changed 18 months ago by HJarausch

VBox.png

comment:1 Changed 18 months ago by frank

This is Gentoo. Does it use the same configuration for Linux 4.5 as for Linux 4.6 or is there any difference?

Changed 18 months ago by Platypus

Changed 18 months ago by Platypus

comment:2 Changed 18 months ago by Platypus

Hello,

I can confirm this issue running openSUSE 13.2 x64 with kernel 4.6.0. It makes no diffenrence if a Windows-VM or Linux-VM is started. I get error code -2708: Guru Meditation -2708 (VERR_VMM_SET_JMP_ABORTED_RESUME)

Some Information about the host:

[18:01:06 @ Mo Mai 30]root@Rosstheboss[~/]$cat /etc/os-release
NAME=openSUSE
VERSION="13.2 (Harlequin)"
VERSION_ID="13.2"
PRETTY_NAME="openSUSE 13.2 (Harlequin) (x86_64)"
ID=opensuse
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:opensuse:opensuse:13.2"
BUG_REPORT_URL="https://bugs.opensuse.org"
HOME_URL="https://opensuse.org/"                                                                                                                           
ID_LIKE="suse"                                                                                                                                             
[18:01:14 @ Mo Mai 30]root@Rosstheboss[~/]$
[18:01:14 @ Mo Mai 30]root@Rosstheboss[~/]$uname -a
Linux Rosstheboss 4.6.0-Intel #2 SMP PREEMPT Sun May 29 08:20:59 CEST 2016 x86_64 x86_64 x86_64 GNU/Linux
[18:02:05 @ Mo Mai 30]root@Rosstheboss[~/]$

Error-Log of Windows-7-VM: Please check attachement VBox.log_Windows_7

Error-Log of openSUSE-42.1-VM: Please check attachement VBox.log_openSUSE_421

I also did a rollback to kernel 4.5.4 using the same configuration as kernel 4.6.0. There were no crashes anymore. So it seems to be related to some change in the kernel 4.6.0

comment:3 Changed 18 months ago by frank

Please could you attach the configuation of your 4.6 kernel?

Changed 18 months ago by FredBezies

Crash log in Archlinux + kernel 4.6

Changed 18 months ago by FredBezies

Config file for linux kernel 4.6 in Archlinux - Hope it helps :)

comment:4 Changed 18 months ago by efferre

I'm experiencing the same problem with a 64bit Linux host and Win7 64 bit guest.

Changed 18 months ago by efferre

Changed 18 months ago by efferre

host linux kernel config

Changed 18 months ago by Platypus

comment:5 Changed 18 months ago by FredBezies

After looking at Archlinux change for kernel config file, I noticed that CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS seems to be guilty here. It is also enabled in both config-4.6.0-gentoo and config_4.6.0_Intel

Looking at this option man page...

 http://cateee.net/lkddb/web-lkddb/X86_INTEL_MEMORY_PROTECTION_KEYS.html

Just an educated guess, of course :)

Last edited 18 months ago by FredBezies (previous) (diff)

comment:6 Changed 18 months ago by Platypus

Thx for the hint. Compiling a new kernel without CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS enabled at the moment. Will come back to you...

comment:7 follow-up: ↓ 8 Changed 18 months ago by Platypus

No luck. Got same error again...

comment:8 in reply to: ↑ 7 Changed 18 months ago by FredBezies

Replying to Platypus:

No luck. Got same error again...

Ouch. I was looking at changes in kernel config between 4.5.x and 4.6.0. This option was always present.

Could be CONFIG_ARCH_HAS_PKEYS ? CONFIG_KALLSYMS_ABSOLUTE_PERCPU ? CONFIG_KALLSYMS_BASE_RELATIVE ? ONFIG_HAVE_STACK_VALIDATION ? CONFIG_ARCH_USES_HIGH_VMA_FLAGS ?

All options were added in 4.6.x code.

Changed 18 months ago by FredBezies

Another crash log under Archlinux with linux kernel 4.6.1

Changed 18 months ago by FredBezies

Screenshot related to archlinux + linux 4.6.1 crash.

Changed 18 months ago by fbe

VirtualBox Crash Log (fbe) on Gentoo Kernel 4.6.1 while installing win7 in vm

Changed 18 months ago by fbe

kernel config (fbe) gentoo 4.6.1 corresponding to VBox.log-fbe-gentoo-kernel4.6.1

comment:9 Changed 18 months ago by fbe

Having the same problem with Virtualbox 5.0.20 here on kernel 4.6 (gentoo, not sure if it worked with < 4.6).

In my case vbox crashes when running win10 / installing win7 in a fresh vm.

Attached both the log and the kernel config.

comment:10 Changed 18 months ago by FredBezies

Again this source file in every single log or so ? :)

00:02:52.724675 VMSetError: /var/tmp/portage/app-emulation/virtualbox-5.0.20/work/VirtualBox-5.0.20/src/VBox/VMM/VMMR3/MM.cpp(685) int MMR3AdjustFixedReservation(PVM, int32_t, const char*); rc=VERR_VMM_SET_JMP_ABORTED_RESUME

comment:11 Changed 18 months ago by frank

Unfortunately I'm still unable to reproduce this problem on a Debian host with a self-compiled vanilla 4.6.0 kernel. So it's most likely that a compile option is responsible for these problems. So far no time to check every single difference between my config and the Gentoo / ArchLinux config.

comment:12 Changed 18 months ago by fbe

Hi frank,

can you post the kernel configuration and your compiler you were using on your debian test? I'll retry it with the vanilla sources (unpatched) 4.6.1 and report if this doesn't work.

Update: problem exists with the unpatched vanilla sources and my posted kernel configuration in 4.6.1, too.

GCC-Version used for kernel / vbox compilation: 4.9.3

Last edited 18 months ago by fbe (previous) (diff)

Changed 18 months ago by frank

comment:13 Changed 18 months ago by frank

fbe, done. I'm using gcc 5.3.1 on Debian.

comment:14 Changed 18 months ago by fbe

Additional information: my vbox doesn't crash immediately. For example i can perform the first setup steps for a windows 7 installation and it crashes after the first reboot of the installation.

My win10 machine crashes suddently, too.

comment:15 Changed 18 months ago by fbe

as mentioned by FredBezies i can confirm that deactivating

CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS

in my vanilla 4.6.1 helps (vbox working stable now).

Edit: Problem still exists, but my windows 7 installation in virtualbox crashes "later" than with activated MEMORY_PROTECTION_KEYS :(

Last edited 18 months ago by fbe (previous) (diff)

Changed 18 months ago by fbe

Crash with Vanilla 4.6.1 Kernel and disabled memory protection keys

comment:16 Changed 18 months ago by FredBezies

Every single time, same file and same line leading to crash :

-> src/VBox/VMM/VMMR3/MM.cpp(685)

00:08:11.015098 VUSB: Detached 'HidMouse' from port 1
00:08:11.016864 VMSetError: /var/tmp/portage/app-emulation/virtualbox-5.0.20/work/VirtualBox-5.0.20/src/VBox/VMM/VMMR3/MM.cpp(685) int MMR3AdjustFixedReservation(PVM, int32_t, const char*); rc=VERR_VMM_SET_JMP_ABORTED_RESUME
00:08:11.016878 VMSetError: Failed to reserve physical memory (0x3706 -> 0x3702; VMMDev Heap)
00:08:11.017559 VMSetError: /var/tmp/portage/app-emulation/virtualbox-5.0.20/work/VirtualBox-5.0.20/src/VBox/VMM/VMMR3/MM.cpp(685) int MMR3AdjustFixedReservation(PVM, int32_t, const char*); rc=VERR_VMM_SET_JMP_ABORTED_RESUME
00:08:11.017568 VMSetError: Failed to reserve physical memory (0x3706 -> 0x3306; VMMDev)
00:08:11.025315 VMSetError: /var/tmp/portage/app-emulation/virtualbox-5.0.20/work/VirtualBox-5.0.20/src/VBox/VMM/VMMR3/MM.cpp(685) int MMR3AdjustFixedReservation(PVM, int32_t, const char*); rc=VERR_VMM_SET_JMP_ABORTED_RESUME
00:08:11.025328 VMSetError: Failed to reserve physical memory (0x3706 -> 0x406; VRam)
00:08:11.025887 NAT: Zone(nm:mbuf_cluster, used:0)
00:08:11.026814 NAT: Zone(nm:mbuf_packet, used:0)
00:08:11.026828 NAT: Zone(nm:mbuf, used:1)
00:08:11.026836 NAT: Zone(nm:mbuf_jumbo_pagesize, used:0)
00:08:11.027696 NAT: Zone(nm:mbuf_jumbo_9k, used:0)
00:08:11.028138 NAT: Zone(nm:mbuf_jumbo_16k, used:0)
00:08:11.028364 NAT: Zone(nm:mbuf_ext_refcnt, used:0)
00:08:11.028570 VMSetError: /var/tmp/portage/app-emulation/virtualbox-5.0.20/work/VirtualBox-5.0.20/src/VBox/VMM/VMMR3/MM.cpp(685) int MMR3AdjustFixedReservation(PVM, int32_t, const char*); rc=VERR_VMM_SET_JMP_ABORTED_RESUME
00:08:11.028583 VMSetError: Failed to reserve physical memory (0x3706 -> 0x3705; Hyper-V hypercall page)
00:08:11.028626 VMSetError: /var/tmp/portage/app-emulation/virtualbox-5.0.20/work/VirtualBox-5.0.20/src/VBox/VMM/VMMR3/MM.cpp(685) int MMR3AdjustFixedReservation(PVM, int32_t, const char*); rc=VERR_VMM_SET_JMP_ABORTED_RESUME
00:08:11.028633 VMSetError: Failed to reserve physical memory (0x3706 -> 0x3705; Hyper-V TSC page)
00:08:11.048018 VMM: VMMR3Term: R0 term failed, rc=VERR_VMM_SET_JMP_ABORTED_RESUME (-2708) - VMM set jump resume error.. (warning)
00:08:11.049861 Changing the VM state from 'DESTROYING' to 'TERMINATED'
00:08:11.052949 Console: Machine state changed to 'PoweredOff'
00:08:11.406420 GUI: Passing request to close Runtime UI from machine-logic to UI session.

Changed 18 months ago by Anybody

Crash Log of CentOS 7 on my Gentoo Kernel 4.6.0 machine with VBOX 5.0.20

Changed 18 months ago by Anybody

Kernel .config of my Gentoo kernel 4.6.0 machine

comment:17 Changed 18 months ago by Anybody

I'm also suffering from this problem. Running a Gentoo Host with 4.6.0 kernel and two guests. Guest "Windows 10" has not crashed so far, while guest "CentOS 7" gets stuck about once a day at random times of day. The reason why the Windows guest has not crashed might be that it's not used heavily. But then again the CentOS guest crashes at random times and not at times of high usage. So perhaps it is something else.

The difference in guest configuration is: more RAM (7GB for CentOS, 2 for Windows), more CPU (4 for CentOS, 1 for Windows) and two options under System: PAE/NX and KVM Paravirtualization was on for CentOS and off for the Windows guests.

You can find my kernel .config and the crashlog in the attachments directly above. I have now downgraded to kernel 4.5.7.

Last edited 18 months ago by Anybody (previous) (diff)

comment:18 Changed 18 months ago by Platypus

Hi all,

for me it seesm to to work stable now:

  • updated to VirtualBox 5.1 BETA2
  • rebuild my kernel and played around. You can find my working config as attachement

My Windows 7 VM is running stable now since a few hours. Also the openSUSE VMs do not crash anymore. I had three running VMs in parallel and produced heavy load. Everything is working fine so far.

Changed 18 months ago by Platypus

comment:19 Changed 17 months ago by frank

Looks to me like CONFIG_PREEMPT_NOTIFIERS=y is responsible. If this option is set, all is fine. Without this option, the above Guru is raised. This option depends on CONFIG_VIRTUALIZATION.

comment:20 Changed 17 months ago by frank

In other words: I believe that enabling CONFIG_VIRTUALIZATION works around this bug.

comment:21 Changed 17 months ago by fbe

Hi frank,

i enabled CONFIG_PREEMPT_NOTIFIERS by activating KVM support (which is required for MPREEMPT_NOTIFIERS, too).

Result is, that my virtualbox kernel module crashes when starting a new machine..

[   23.140685] capability: warning: `VirtualBox' uses 32-bit capabilities (legacy support in use)
[   27.964520] BUG: unable to handle kernel NULL pointer dereference at 0000000000000068
[   27.964537] IP: [<ffffffff81826341>] down_write+0x11/0x30
[   27.964542] PGD 832748067 PUD 80bcef067 PMD 0 
[   27.964546] Oops: 0002 [#1] PREEMPT SMP 
[   27.964549] Modules linked in: xt_nat veth ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 xt_addrtype nf_nat br_netfilter bridge stp llc overlay x86_pkg_temp_thermal kvm_intel kvm irqbypass snd_usb_audio snd_usbmidi_lib snd_rawmidi algif_skcipher af_alg crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel aes_x86_64 glue_helper lrw ablk_helper cryptd vboxnetflt(O) vboxnetadp(O) vboxdrv(O)
[   27.964580] CPU: 2 PID: 2672 Comm: VirtualBox Tainted: G           O    4.6.2 #3
[   27.964582] Hardware name: Gigabyte Technology Co., Ltd. Z170XP-SLI/Z170XP-SLI-CF, BIOS F2 07/17/2015
[   27.964585] task: ffff88082d6a6580 ti: ffff88080e7a8000 task.ti: ffff88080e7a8000
[   27.964588] RIP: 0010:[<ffffffff81826341>]  [<ffffffff81826341>] down_write+0x11/0x30
[   27.964591] RSP: 0018:ffff88080e7abc80  EFLAGS: 00050202
[   27.964593] RAX: 0000000000000068 RBX: 0000000000000001 RCX: 8000000000000025
[   27.964596] RDX: ffffffff00000001 RSI: 0000000000000000 RDI: 0000000000000068
[   27.964598] RBP: ffff88080e7abc80 R08: 0000000000000000 R09: 00007f2d3c1c8000
[   27.964601] R10: ffff880832419398 R11: 0000000000019228 R12: ffff88082d6a6580
[   27.964603] R13: ffff8808405e8510 R14: ffffffffffffffff R15: ffff8808326ba290
[   27.964605] FS:  00007f2d3c19b700(0000) GS:ffff880866480000(0000) knlGS:0000000000000000
[   27.964608] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   27.964610] CR2: 0000000000000068 CR3: 000000080f309000 CR4: 00000000003406e0
[   27.964612] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[   27.964614] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[   27.964616] Stack:
[   27.964618]  ffff88080e7abcf8 ffffffffa0013d76 0000000000000002 0000000000000002
[   27.964623]  ffff88080e7abd20 8000000000000025 0000000000040246 00007f2d3c1c8000
[   27.964630]  0000000000000000 0000000000000001 ffff8808405e8510 ffff880821d5e068
[   27.964634] Call Trace:
[   27.964640]  [<ffffffffa0013d76>] rtR0MemObjNativeMapUser+0xb6/0x2a0 [vboxdrv]
[   27.964645]  [<ffffffffa0011950>] VBoxHost_RTR0MemObjMapUserTag+0x110/0x190 [vboxdrv]
[   27.964649]  [<ffffffffa000abe9>] SUPR0GipMap+0x469/0x560 [vboxdrv]
[   27.964652]  [<ffffffff811812ed>] ? __kmalloc+0xfd/0x250
[   27.964656]  [<ffffffffa00051f9>] supdrvIOCtl+0xfd9/0x3ad0 [vboxdrv]
[   27.964659]  [<ffffffff81180405>] ? kfree+0x165/0x190
[   27.964662]  [<ffffffffa0000510>] SUPR0Printf+0x1c0/0x2c0 [vboxdrv]
[   27.964665]  [<ffffffff811a55d9>] do_vfs_ioctl+0x89/0x580
[   27.964669]  [<ffffffff8104b02b>] ? __do_page_fault+0x1cb/0x520
[   27.964671]  [<ffffffff811a5b44>] SyS_ioctl+0x74/0x80
[   27.964674]  [<ffffffff81827edb>] entry_SYSCALL_64_fastpath+0x13/0x8f
[   27.964676] Code: e5 f0 48 ff 00 79 05 e8 0e 1c b1 ff 5d c3 66 90 66 2e 0f 1f 84 00 00 00 00 00 55 48 ba 01 00 00 00 ff ff ff ff 48 89 f8 48 89 e5 <f0> 48 0f c1 10 85 d2 74 05 e8 11 1c b1 ff 65 48 8b 04 25 00 c4 
[   27.964722] RIP  [<ffffffff81826341>] down_write+0x11/0x30
[   27.964725]  RSP <ffff88080e7abc80>
[   27.964727] CR2: 0000000000000068
[   27.970886] ---[ end trace 681d2941eba239bd ]---

Kernel config is the same as before only with kvm enabled..

sorry, messed up my installed kernels while debugging, didn't rebuild the virtualbox modules properly, so please ignore this.

Last edited 17 months ago by fbe (previous) (diff)

comment:22 Changed 17 months ago by frank

fbe, did you recompile your VBox kernel modules after you booted into the kernel with the new configuration?

comment:23 Changed 17 months ago by fbe

Hi frank,

sorry, my fault, messed up my kernel builds.

With CONFIG_PREEMPT_NOTIFIERS enabled i can confirm that it works here on gentoo with 5.0.20 and kernel 4.6.2 (amd64), no more crashes yet.

comment:24 Changed 17 months ago by frank

FYI, 5.0.22 has not a fix for this problem. At the moment we only have the above workaround but no fix available (yet).

comment:25 Changed 17 months ago by FredBezies

Well, CONFIG_PREEMPT_NOTIFIERS=y is not a working workaround for Archlinux based system.

It is enabled on archlinux kernel, and even with 5.0.22, I got 2 crashes in 5 minutes while trying to install a Manjaro based system.

Changed 17 months ago by Hekarath

Changed 17 months ago by Hekarath

comment:26 Changed 17 months ago by Hekarath

Issue still present with 5.0.22. Checked on Gentoo Linux with kernel 4.6.0-pf1 and 4.6.2-gentoo. I'm using voluntary preemption, since preemptive kernel is not supported by Broadcom STA driver:

$ grep PREEMPT .config
# CONFIG_PREEMPT_NONE is not set
CONFIG_PREEMPT_VOLUNTARY=y
# CONFIG_PREEMPT is not set

VirtualBox version:

$ vboxmanage -v
5.0.22_Gentoor108108

Module versions:

$ modinfo vboxdrv vboxnetflt vboxnetadp | grep '^filename:\|^version:'
filename:       /lib/modules/4.6.0-pf1/misc/vboxdrv.ko
version:        5.0.22 r108108 (0x00240000)
filename:       /lib/modules/4.6.0-pf1/misc/vboxnetflt.ko
version:        5.0.22 r108108 ((0xA2CDe001U))
filename:       /lib/modules/4.6.0-pf1/misc/vboxnetadp.ko
version:        5.0.22 r108108 ((0xA2CDe001U))

VBox.log (VBox.log_hekarath.log) and kernel config (config-4.6.0-pf1_hekarath) attached.

Last edited 17 months ago by frank (previous) (diff)

comment:27 Changed 17 months ago by HMLi

For 5.0.22r, the CONFIG_PREEMPT_NOTIFIERS=y is work for the kernel 4.6.2 customized on Debian sid, x86_64. But disabling 2d and 3d hardware acceleration is also needed.

comment:28 Changed 17 months ago by frank

The most recent 5.0 test builds have a fix for this problem, at least it works here. Could you confirm?

comment:29 Changed 17 months ago by frank

Any feedback yet for these builds?

comment:30 Changed 17 months ago by Rainmaker

For what it's worth, on my setup, the 5.0.23-108298 build no longer has Guru Mediations. I've only been testing it for an hour though, but had a pretty sure way to reproduce them.

Host kernel is 4.6.2.

comment:31 Changed 17 months ago by frank

  • Status changed from new to closed
  • Resolution set to fixed

Fix is part of VBox 5.0.24.

Note: See TracTickets for help on using tickets.

www.oracle.com
ContactPrivacy policyTerms of Use