5.0.20 + kernel 4.6.0 = Guru Meditation critical error

[HJarausch]

Hi,

I have a Window7 VDI file which works perfectly with VB 5.0.20 when I run a older kernel like 4.5.

But with the same VDI file VB crashes under kernel 4.6.0 giving VirtualBox - Guru Meditation A critical error has occurred while running the virtual machine and the machine execution has been stopped.

[HJarausch]

VBox.log

[HJarausch]

VBox.png

[Frank Mehnert]

This is Gentoo. Does it use the same configuration for Linux 4.5 as for Linux 4.6 or is there any difference?

[Platypus]

Hello,

I can confirm this issue running openSUSE 13.2 x64 with kernel 4.6.0. It makes no diffenrence if a Windows-VM or Linux-VM is started. I get error code -2708: Guru Meditation -2708 (VERR_VMM_SET_JMP_ABORTED_RESUME)

Some Information about the host:

[18:01:06 @ Mo Mai 30]root@Rosstheboss[~/]$cat /etc/os-release
NAME=openSUSE
VERSION="13.2 (Harlequin)"
VERSION_ID="13.2"
PRETTY_NAME="openSUSE 13.2 (Harlequin) (x86_64)"
ID=opensuse
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:opensuse:opensuse:13.2"
BUG_REPORT_URL="https://bugs.opensuse.org"
HOME_URL="https://opensuse.org/"                                                                                                                           
ID_LIKE="suse"                                                                                                                                             
[18:01:14 @ Mo Mai 30]root@Rosstheboss[~/]$

[18:01:14 @ Mo Mai 30]root@Rosstheboss[~/]$uname -a
Linux Rosstheboss 4.6.0-Intel #2 SMP PREEMPT Sun May 29 08:20:59 CEST 2016 x86_64 x86_64 x86_64 GNU/Linux
[18:02:05 @ Mo Mai 30]root@Rosstheboss[~/]$

Error-Log of Windows-7-VM: Please check attachement VBox.log_Windows_7

Error-Log of openSUSE-42.1-VM: Please check attachement VBox.log_openSUSE_421

I also did a rollback to kernel 4.5.4 using the same configuration as kernel 4.6.0. There were no crashes anymore. So it seems to be related to some change in the kernel 4.6.0

[Frank Mehnert]

Please could you attach the configuation of your 4.6 kernel?

[FredBezies]

Crash log in Archlinux + kernel 4.6

[FredBezies]

Config file for linux kernel 4.6 in Archlinux - Hope it helps :)

[efferre]

I'm experiencing the same problem with a 64bit Linux host and Win7 64 bit guest.

[efferre]

host linux kernel config

[FredBezies]

After looking at Archlinux change for kernel config file, I noticed that CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS seems to be guilty here. It is also enabled in both config-4.6.0-gentoo and config_4.6.0_Intel

Looking at this option man page...

​http://cateee.net/lkddb/web-lkddb/X86_INTEL_MEMORY_PROTECTION_KEYS.html

Just an educated guess, of course :)

[Platypus]

Thx for the hint. Compiling a new kernel without CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS enabled at the moment. Will come back to you...

[Platypus]

No luck. Got same error again...

[FredBezies]

Replying to Platypus:

No luck. Got same error again...

Ouch. I was looking at changes in kernel config between 4.5.x and 4.6.0. This option was always present.

Could be CONFIG_ARCH_HAS_PKEYS ? CONFIG_KALLSYMS_ABSOLUTE_PERCPU ? CONFIG_KALLSYMS_BASE_RELATIVE ? ONFIG_HAVE_STACK_VALIDATION ? CONFIG_ARCH_USES_HIGH_VMA_FLAGS ?

All options were added in 4.6.x code.

[FredBezies]

Another crash log under Archlinux with linux kernel 4.6.1

[FredBezies]

Screenshot related to archlinux + linux 4.6.1 crash.

[fbe]

VirtualBox Crash Log (fbe) on Gentoo Kernel 4.6.1 while installing win7 in vm

[fbe]

kernel config (fbe) gentoo 4.6.1 corresponding to VBox.log-fbe-gentoo-kernel4.6.1

[fbe]

Having the same problem with Virtualbox 5.0.20 here on kernel 4.6 (gentoo, not sure if it worked with < 4.6).

In my case vbox crashes when running win10 / installing win7 in a fresh vm.

Attached both the log and the kernel config.

[FredBezies]

Again this source file in every single log or so ? :)

00:02:52.724675 VMSetError: /var/tmp/portage/app-emulation/virtualbox-5.0.20/work/VirtualBox-5.0.20/src/VBox/VMM/VMMR3/MM.cpp(685) int MMR3AdjustFixedReservation(PVM, int32_t, const char*); rc=VERR_VMM_SET_JMP_ABORTED_RESUME

[Frank Mehnert]

Unfortunately I'm still unable to reproduce this problem on a Debian host with a self-compiled vanilla 4.6.0 kernel. So it's most likely that a compile option is responsible for these problems. So far no time to check every single difference between my config and the Gentoo / ArchLinux config.

[fbe]

Hi frank,

can you post the kernel configuration and your compiler you were using on your debian test? I'll retry it with the vanilla sources (unpatched) 4.6.1 and report if this doesn't work.

Update: problem exists with the unpatched vanilla sources and my posted kernel configuration in 4.6.1, too.

GCC-Version used for kernel / vbox compilation: 4.9.3

[Frank Mehnert]

fbe, done. I'm using gcc 5.3.1 on Debian.

[fbe]

Additional information: my vbox doesn't crash immediately. For example i can perform the first setup steps for a windows 7 installation and it crashes after the first reboot of the installation.

My win10 machine crashes suddently, too.

[fbe]

as mentioned by FredBezies i can confirm that deactivating

CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS

in my vanilla 4.6.1 helps (vbox working stable now).

Edit: Problem still exists, but my windows 7 installation in virtualbox crashes "later" than with activated MEMORY_PROTECTION_KEYS :(

[fbe]

Crash with Vanilla 4.6.1 Kernel and disabled memory protection keys

[FredBezies]

Every single time, same file and same line leading to crash :

-> src/VBox/VMM/VMMR3/MM.cpp(685)

00:08:11.015098 VUSB: Detached 'HidMouse' from port 1
00:08:11.016864 VMSetError: /var/tmp/portage/app-emulation/virtualbox-5.0.20/work/VirtualBox-5.0.20/src/VBox/VMM/VMMR3/MM.cpp(685) int MMR3AdjustFixedReservation(PVM, int32_t, const char*); rc=VERR_VMM_SET_JMP_ABORTED_RESUME
00:08:11.016878 VMSetError: Failed to reserve physical memory (0x3706 -> 0x3702; VMMDev Heap)
00:08:11.017559 VMSetError: /var/tmp/portage/app-emulation/virtualbox-5.0.20/work/VirtualBox-5.0.20/src/VBox/VMM/VMMR3/MM.cpp(685) int MMR3AdjustFixedReservation(PVM, int32_t, const char*); rc=VERR_VMM_SET_JMP_ABORTED_RESUME
00:08:11.017568 VMSetError: Failed to reserve physical memory (0x3706 -> 0x3306; VMMDev)
00:08:11.025315 VMSetError: /var/tmp/portage/app-emulation/virtualbox-5.0.20/work/VirtualBox-5.0.20/src/VBox/VMM/VMMR3/MM.cpp(685) int MMR3AdjustFixedReservation(PVM, int32_t, const char*); rc=VERR_VMM_SET_JMP_ABORTED_RESUME
00:08:11.025328 VMSetError: Failed to reserve physical memory (0x3706 -> 0x406; VRam)
00:08:11.025887 NAT: Zone(nm:mbuf_cluster, used:0)
00:08:11.026814 NAT: Zone(nm:mbuf_packet, used:0)
00:08:11.026828 NAT: Zone(nm:mbuf, used:1)
00:08:11.026836 NAT: Zone(nm:mbuf_jumbo_pagesize, used:0)
00:08:11.027696 NAT: Zone(nm:mbuf_jumbo_9k, used:0)
00:08:11.028138 NAT: Zone(nm:mbuf_jumbo_16k, used:0)
00:08:11.028364 NAT: Zone(nm:mbuf_ext_refcnt, used:0)
00:08:11.028570 VMSetError: /var/tmp/portage/app-emulation/virtualbox-5.0.20/work/VirtualBox-5.0.20/src/VBox/VMM/VMMR3/MM.cpp(685) int MMR3AdjustFixedReservation(PVM, int32_t, const char*); rc=VERR_VMM_SET_JMP_ABORTED_RESUME
00:08:11.028583 VMSetError: Failed to reserve physical memory (0x3706 -> 0x3705; Hyper-V hypercall page)
00:08:11.028626 VMSetError: /var/tmp/portage/app-emulation/virtualbox-5.0.20/work/VirtualBox-5.0.20/src/VBox/VMM/VMMR3/MM.cpp(685) int MMR3AdjustFixedReservation(PVM, int32_t, const char*); rc=VERR_VMM_SET_JMP_ABORTED_RESUME
00:08:11.028633 VMSetError: Failed to reserve physical memory (0x3706 -> 0x3705; Hyper-V TSC page)
00:08:11.048018 VMM: VMMR3Term: R0 term failed, rc=VERR_VMM_SET_JMP_ABORTED_RESUME (-2708) - VMM set jump resume error.. (warning)
00:08:11.049861 Changing the VM state from 'DESTROYING' to 'TERMINATED'
00:08:11.052949 Console: Machine state changed to 'PoweredOff'
00:08:11.406420 GUI: Passing request to close Runtime UI from machine-logic to UI session.

[Anybody]

Crash Log of CentOS 7 on my Gentoo Kernel 4.6.0 machine with VBOX 5.0.20

[Anybody]

Kernel .config of my Gentoo kernel 4.6.0 machine

[Anybody]

I'm also suffering from this problem. Running a Gentoo Host with 4.6.0 kernel and two guests. Guest "Windows 10" has not crashed so far, while guest "CentOS 7" gets stuck about once a day at random times of day. The reason why the Windows guest has not crashed might be that it's not used heavily. But then again the CentOS guest crashes at random times and not at times of high usage. So perhaps it is something else.

The difference in guest configuration is: more RAM (7GB for CentOS, 2 for Windows), more CPU (4 for CentOS, 1 for Windows) and two options under System: PAE/NX and KVM Paravirtualization was on for CentOS and off for the Windows guests.

You can find my kernel .config and the crashlog in the attachments directly above. I have now downgraded to kernel 4.5.7.

[Platypus]

Hi all,

for me it seesm to to work stable now:

• updated to VirtualBox 5.1 BETA2
• rebuild my kernel and played around. You can find my working config as attachement

My Windows 7 VM is running stable now since a few hours. Also the openSUSE VMs do not crash anymore. I had three running VMs in parallel and produced heavy load. Everything is working fine so far.

[Frank Mehnert]

Looks to me like CONFIG_PREEMPT_NOTIFIERS=y is responsible. If this option is set, all is fine. Without this option, the above Guru is raised. This option depends on CONFIG_VIRTUALIZATION.

[Frank Mehnert]

In other words: I believe that enabling CONFIG_VIRTUALIZATION works around this bug.

[fbe]

Hi frank,

i enabled CONFIG_PREEMPT_NOTIFIERS by activating KVM support (which is required for MPREEMPT_NOTIFIERS, too).

Result is, that my virtualbox kernel module crashes when starting a new machine..

[   23.140685] capability: warning: `VirtualBox' uses 32-bit capabilities (legacy support in use)
[   27.964520] BUG: unable to handle kernel NULL pointer dereference at 0000000000000068
[   27.964537] IP: [<ffffffff81826341>] down_write+0x11/0x30
[   27.964542] PGD 832748067 PUD 80bcef067 PMD 0 
[   27.964546] Oops: 0002 [#1] PREEMPT SMP 
[   27.964549] Modules linked in: xt_nat veth ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 xt_addrtype nf_nat br_netfilter bridge stp llc overlay x86_pkg_temp_thermal kvm_intel kvm irqbypass snd_usb_audio snd_usbmidi_lib snd_rawmidi algif_skcipher af_alg crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel aes_x86_64 glue_helper lrw ablk_helper cryptd vboxnetflt(O) vboxnetadp(O) vboxdrv(O)
[   27.964580] CPU: 2 PID: 2672 Comm: VirtualBox Tainted: G           O    4.6.2 #3
[   27.964582] Hardware name: Gigabyte Technology Co., Ltd. Z170XP-SLI/Z170XP-SLI-CF, BIOS F2 07/17/2015
[   27.964585] task: ffff88082d6a6580 ti: ffff88080e7a8000 task.ti: ffff88080e7a8000
[   27.964588] RIP: 0010:[<ffffffff81826341>]  [<ffffffff81826341>] down_write+0x11/0x30
[   27.964591] RSP: 0018:ffff88080e7abc80  EFLAGS: 00050202
[   27.964593] RAX: 0000000000000068 RBX: 0000000000000001 RCX: 8000000000000025
[   27.964596] RDX: ffffffff00000001 RSI: 0000000000000000 RDI: 0000000000000068
[   27.964598] RBP: ffff88080e7abc80 R08: 0000000000000000 R09: 00007f2d3c1c8000
[   27.964601] R10: ffff880832419398 R11: 0000000000019228 R12: ffff88082d6a6580
[   27.964603] R13: ffff8808405e8510 R14: ffffffffffffffff R15: ffff8808326ba290
[   27.964605] FS:  00007f2d3c19b700(0000) GS:ffff880866480000(0000) knlGS:0000000000000000
[   27.964608] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   27.964610] CR2: 0000000000000068 CR3: 000000080f309000 CR4: 00000000003406e0
[   27.964612] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[   27.964614] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[   27.964616] Stack:
[   27.964618]  ffff88080e7abcf8 ffffffffa0013d76 0000000000000002 0000000000000002
[   27.964623]  ffff88080e7abd20 8000000000000025 0000000000040246 00007f2d3c1c8000
[   27.964630]  0000000000000000 0000000000000001 ffff8808405e8510 ffff880821d5e068
[   27.964634] Call Trace:
[   27.964640]  [<ffffffffa0013d76>] rtR0MemObjNativeMapUser+0xb6/0x2a0 [vboxdrv]
[   27.964645]  [<ffffffffa0011950>] VBoxHost_RTR0MemObjMapUserTag+0x110/0x190 [vboxdrv]
[   27.964649]  [<ffffffffa000abe9>] SUPR0GipMap+0x469/0x560 [vboxdrv]
[   27.964652]  [<ffffffff811812ed>] ? __kmalloc+0xfd/0x250
[   27.964656]  [<ffffffffa00051f9>] supdrvIOCtl+0xfd9/0x3ad0 [vboxdrv]
[   27.964659]  [<ffffffff81180405>] ? kfree+0x165/0x190
[   27.964662]  [<ffffffffa0000510>] SUPR0Printf+0x1c0/0x2c0 [vboxdrv]
[   27.964665]  [<ffffffff811a55d9>] do_vfs_ioctl+0x89/0x580
[   27.964669]  [<ffffffff8104b02b>] ? __do_page_fault+0x1cb/0x520
[   27.964671]  [<ffffffff811a5b44>] SyS_ioctl+0x74/0x80
[   27.964674]  [<ffffffff81827edb>] entry_SYSCALL_64_fastpath+0x13/0x8f
[   27.964676] Code: e5 f0 48 ff 00 79 05 e8 0e 1c b1 ff 5d c3 66 90 66 2e 0f 1f 84 00 00 00 00 00 55 48 ba 01 00 00 00 ff ff ff ff 48 89 f8 48 89 e5 <f0> 48 0f c1 10 85 d2 74 05 e8 11 1c b1 ff 65 48 8b 04 25 00 c4 
[   27.964722] RIP  [<ffffffff81826341>] down_write+0x11/0x30
[   27.964725]  RSP <ffff88080e7abc80>
[   27.964727] CR2: 0000000000000068
[   27.970886] ---[ end trace 681d2941eba239bd ]---

Kernel config is the same as before only with kvm enabled..

sorry, messed up my installed kernels while debugging, didn't rebuild the virtualbox modules properly, so please ignore this.

[Frank Mehnert]

fbe, did you recompile your VBox kernel modules after you booted into the kernel with the new configuration?

[fbe]

Hi frank,

sorry, my fault, messed up my kernel builds.

With CONFIG_PREEMPT_NOTIFIERS enabled i can confirm that it works here on gentoo with 5.0.20 and kernel 4.6.2 (amd64), no more crashes yet.

[Frank Mehnert]

FYI, 5.0.22 has not a fix for this problem. At the moment we only have the above workaround but no fix available (yet).

[FredBezies]

Well, CONFIG_PREEMPT_NOTIFIERS=y is not a working workaround for Archlinux based system.

It is enabled on archlinux kernel, and even with 5.0.22, I got 2 crashes in 5 minutes while trying to install a Manjaro based system.

[Hekarath]

Issue still present with 5.0.22. Checked on Gentoo Linux with kernel 4.6.0-pf1 and 4.6.2-gentoo. I'm using voluntary preemption, since preemptive kernel is not supported by Broadcom STA driver:

$ grep PREEMPT .config
# CONFIG_PREEMPT_NONE is not set
CONFIG_PREEMPT_VOLUNTARY=y
# CONFIG_PREEMPT is not set

VirtualBox version:

$ vboxmanage -v
5.0.22_Gentoor108108

Module versions:

$ modinfo vboxdrv vboxnetflt vboxnetadp | grep '^filename:\|^version:'
filename:       /lib/modules/4.6.0-pf1/misc/vboxdrv.ko
version:        5.0.22 r108108 (0x00240000)
filename:       /lib/modules/4.6.0-pf1/misc/vboxnetflt.ko
version:        5.0.22 r108108 ((0xA2CDe001U))
filename:       /lib/modules/4.6.0-pf1/misc/vboxnetadp.ko
version:        5.0.22 r108108 ((0xA2CDe001U))

VBox.log (VBox.log_hekarath.log) and kernel config (config-4.6.0-pf1_hekarath) attached.

[HMLi]

For 5.0.22r, the CONFIG_PREEMPT_NOTIFIERS=y is work for the kernel 4.6.2 customized on Debian sid, x86_64. But disabling 2d and 3d hardware acceleration is also needed.

[Frank Mehnert]

The most recent 5.0 test builds have a fix for this problem, at least it works here. Could you confirm?

[Frank Mehnert]

Any feedback yet for these builds?

[Rainmaker]

For what it's worth, on my setup, the 5.0.23-108298 build no longer has Guru Mediations. I've only been testing it for an hour though, but had a pretty sure way to reproduce them.

Host kernel is 4.6.2.

[Frank Mehnert]

Fix is part of VBox 5.0.24.