Kernel (4.5.0) failure on VM start

[th0br0]

VM startup repeatedly fails, regardless of the guest OS.

[50763.014158] ------------[ cut here ]------------
[50763.016334] kernel BUG at include/linux/page-flags.h:272!
[50763.018488] invalid opcode: 0000 [#3] SMP 
[50763.020664] Modules linked in: vboxpci(OE) vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) cpufreq_stats rfcomm fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_broute bridge stp llc ebtable_filter ebtable_nat ebtables ip6table_raw ip6table_security ip6table_mangle ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_filter ip6_tables iptable_raw iptable_security iptable_mangle iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack bnep vfat fat btusb btrtl uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_core videodev media i2c_designware_platform i2c_designware_core dell_wmi snd_soc_skl snd_hda_codec_hdmi snd_soc_skl_ipc
[50763.025429]  snd_soc_sst_ipc snd_soc_sst_dsp dell_led intel_rapl dell_laptop snd_hda_ext_core dcdbas snd_soc_sst_match x86_pkg_temp_thermal snd_soc_core snd_hda_codec_realtek coretemp snd_hda_codec_generic kvm_intel snd_compress snd_pcm_dmaengine kvm ac97_bus dw_dmac_core irqbypass snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device brcmfmac snd_pcm brcmutil joydev cfg80211 snd_timer rtsx_pci_ms memstick snd mei_me soundcore i2c_i801 mei processor_thermal_device idma64 shpchp intel_lpss_pci intel_soc_dts_iosf wmi hci_uart btbcm btqca btintel bluetooth pinctrl_sunrisepoint pinctrl_intel rfkill intel_lpss_acpi intel_lpss int3400_thermal int3403_thermal int340x_thermal_zone acpi_thermal_rel intel_hid acpi_pad sparse_keymap acpi_als tpm_tis kfifo_buf tpm industrialio nfsd auth_rpcgss
[50763.030432]  nfs_acl lockd grace sunrpc dm_crypt rtsx_pci_sdmmc mmc_core i915 crct10dif_pclmul crc32_pclmul crc32c_intel i2c_algo_bit ghash_clmulni_intel drm_kms_helper serio_raw rtsx_pci drm nvme i2c_hid video fjes hid_multitouch
[50763.033000] CPU: 1 PID: 17188 Comm: EMT Tainted: G      D W  OE   4.5.0-0.rc7.git3.1.fc25.x86_64 #1
[50763.035479] Hardware name: Dell Inc. XPS 13 9350/0J07MR, BIOS 1.2.3 01/08/2016
[50763.037931] task: ffff8803e1943300 ti: ffff880145640000 task.ti: ffff880145640000
[50763.040319] RIP: 0010:[<ffffffffa0a7ec28>]  [<ffffffffa0a7ec28>] rtR0MemObjLinuxAllocPages+0x178/0x290 [vboxdrv]
[50763.042798] RSP: 0018:ffff880145643b90  EFLAGS: 00050246
[50763.045177] RAX: 000000000000003c RBX: 0000000000000200 RCX: 0000000000000006
[50763.047569] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff8804701ce1e0
[50763.050051] RBP: ffff880145643bc8 R08: 0000000000000001 R09: 0000000000000001
[50763.052565] R10: ffffffff81cb2c16 R11: ffffffff830cb0e8 R12: 0000000000200000
[50763.054988] R13: 00000000024304c2 R14: ffff88023f322010 R15: 0000000000000001
[50763.057911] FS:  00007f3cfbdff700(0000) GS:ffff880470000000(0000) knlGS:0000000000000000
[50763.060392] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[50763.062868] CR2: 0000000000000000 CR3: 00000002972a9000 CR4: 00000000003426e0
[50763.065267] Stack:
[50763.067683]  ffff880145643be0 0000000000200000 ffff880145643c88 ffffffffffffffff
[50763.070152]  0000000000000005 0000000000040296 0000000000200000 ffff880145643c08
[50763.072658]  ffffffffa0a7ed87 ffff8803ffffffb7 ffff8803ee977698 0000000065226c8a
[50763.075093] Call Trace:
[50763.077505]  [<ffffffffa0a7ed87>] rtR0MemObjLinuxAllocPhysSub2+0x47/0xe0 [vboxdrv]
[50763.079865]  [<ffffffffa0a7eee8>] rtR0MemObjLinuxAllocPhysSub+0xc8/0xe0 [vboxdrv]
[50763.082192]  [<ffffffffa0a7f3e9>] rtR0MemObjNativeAllocPhys+0x19/0x20 [vboxdrv]
[50763.084437]  [<ffffffffa0a7ce66>] VBoxHost_RTR0MemObjAllocPhysExTag+0x76/0xb0 [vboxdrv]
[50763.086617]  [<ffffffff813c5ffb>] ? avc_has_extended_perms+0x3b/0x620
[50763.088721]  [<ffffffff8122891d>] ? __might_fault+0x4d/0xa0
[50763.090943]  [<ffffffffa0a70e77>] ? supdrvIOCtl+0x17d7/0x32c0 [vboxdrv]
[50763.093039]  [<ffffffff81228966>] ? __might_fault+0x96/0xa0
[50763.095179]  [<ffffffff8122891d>] ? __might_fault+0x4d/0xa0
[50763.097231]  [<ffffffffa0a6b600>] ? VBoxDrvLinuxIOCtl_5_0_16+0x150/0x250 [vboxdrv]
[50763.099304]  [<ffffffff812a64f4>] ? do_vfs_ioctl+0xa4/0x6d0
[50763.101373]  [<ffffffff812b2cf2>] ? __fget+0x122/0x210
[50763.103400]  [<ffffffff812b2bd5>] ? __fget+0x5/0x210
[50763.105431]  [<ffffffff812a6b99>] ? SyS_ioctl+0x79/0x90
[50763.107494]  [<ffffffff818bb732>] ? entry_SYSCALL_64_fastpath+0x12/0x72
[50763.109574] Code: 40 48 83 c2 08 48 39 c8 75 f0 b9 01 00 00 00 49 8b 7e 50 41 88 4e 40 48 8b 07 f6 c4 40 74 0e 48 c7 c6 88 3d aa a0 e8 28 78 7a e0 <0f> 0b 48 8b 47 20 a8 01 75 ea 31 c0 f0 80 4f 01 04 48 83 c0 01 
[50763.111785] RIP  [<ffffffffa0a7ec28>] rtR0MemObjLinuxAllocPages+0x178/0x290 [vboxdrv]
[50763.113864]  RSP <ffff880145643b90>
[50763.116049] ---[ end trace 7acbc558b74d55ec ]---

[Zdenek Sedlak]

I can confirm the same behavior - Fedora 24, kernel 4.5.0-302.fc24.x86_64, VirtualBox 5.0.16

[21657.385945] ------------[ cut here ]------------
[21657.388011] kernel BUG at include/linux/page-flags.h:272!
[21657.390071] invalid opcode: 0000 [#4] SMP 
[21657.391283] Modules linked in: vboxpci(OE) vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) uas usb_storage uinput btrfs xor raid6_pq ufs hfsplus hfs minix vfat msdos fat jfs rfcomm fuse cmac xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat tun bridge ebtable_filter ebtables ip6table_filter ip6_tables nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack bnep b43 mac80211 cfg80211 snd_hda_codec_hdmi intel_rapl x86_pkg_temp_thermal coretemp ssb kvm_intel iTCO_wdt iTCO_vendor_support kvm ppdev dell_wmi irqbypass sparse_keymap snd_hda_codec_realtek snd_hda_codec_generic btusb dell_laptop btrtl btbcm dcdbas uvcvideo snd_soc_rt5640 btintel dell_smm_hwmon snd_hda_intel videobuf2_vmalloc snd_soc_rl6231 videobuf2_memops bluetooth snd_soc_core snd_hda_codec videobuf2_v4l2
[21657.395230]  videobuf2_core videodev snd_hda_core snd_compress snd_pcm_dmaengine snd_hwdep ac97_bus snd_seq media snd_seq_device snd_pcm bcma i2c_i801 joydev snd_timer mei_me ie31200_edac snd mei parport_pc edac_core shpchp soundcore parport lpc_ich dell_rbtn dell_smo8800 elan_i2c snd_soc_sst_acpi snd_soc_sst_match dw_dmac tpm_tis dw_dmac_core i2c_designware_platform rfkill i2c_designware_core tpm spi_pxa2xx_platform nfsd binfmt_misc auth_rpcgss nfs_acl lockd grace xfs libcrc32c dm_crypt 8021q garp stp llc mrp nouveau crct10dif_pclmul crc32_pclmul crc32c_intel mxm_wmi i2c_algo_bit ghash_clmulni_intel drm_kms_helper e1000e ttm serio_raw drm sdhci_pci ptp pps_core sdhci_acpi wmi sdhci mmc_core video i2c_hid fjes sunrpc scsi_transport_iscsi
[21657.401077] CPU: 7 PID: 24738 Comm: EMT-0 Tainted: G      D    OE   4.5.0-302.fc24.x86_64 #1
[21657.402526] Hardware name: Dell Inc. Precision M6800/0XD1M5, BIOS A16 12/01/2015
[21657.403865] task: ffff88074d2d5b80 ti: ffff880475fec000 task.ti: ffff880475fec000
[21657.405269] RIP: 0010:[<ffffffffa0d6eb68>]  [<ffffffffa0d6eb68>] rtR0MemObjLinuxAllocPages+0x178/0x290 [vboxdrv]
[21657.406760] RSP: 0018:ffff880475fefc98  EFLAGS: 00010246
[21657.408126] RAX: 000000000000003c RBX: 0000000000000002 RCX: 0000000000000006
[21657.409465] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff88082ddcdfb0
[21657.410916] RBP: ffff880475fefcd0 R08: 000000000005e369 R09: 000000000000052d
[21657.412532] R10: ffffffff81a80399 R11: ffffffff81f3f0cd R12: 0000000000002000
[21657.414202] R13: 00000000024200c2 R14: ffff8806bbdfe310 R15: 0000000000000000
[21657.415613] FS:  00007f362c30e700(0000) GS:ffff88082ddc0000(0000) knlGS:0000000000000000
[21657.416977] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[21657.418404] CR2: 00007f3636a04d88 CR3: 00000006bbef3000 CR4: 00000000001406e0
[21657.419773] Stack:
[21657.421135]  ffff880475fefcf0 0000000000001000 0000000000000001 0000000000000246
[21657.422600]  ffff880475fefd50 ffff8806b9c34cf0 ffff8800335a9010 ffff880475fefd18
[21657.424010]  ffffffffa0d6f0cf 00000000fffffff8 0000000000009d04 ffff880475fefd80
[21657.425372] Call Trace:
[21657.426791]  [<ffffffffa0d6f0cf>] rtR0MemObjNativeAllocPage+0x4f/0xb0 [vboxdrv]
[21657.428282]  [<ffffffffa0d6ca72>] VBoxHost_RTR0MemObjAllocPageTag+0x62/0x70 [vboxdrv]
[21657.430117]  [<ffffffffa0d5d4d9>] SUPR0PageAllocEx+0x149/0x2d0 [vboxdrv]
[21657.431601]  [<ffffffffa0d60e57>] supdrvIOCtl+0x17f7/0x32c0 [vboxdrv]
[21657.432952]  [<ffffffffa0d5b5c0>] VBoxDrvLinuxIOCtl_5_0_16+0x150/0x250 [vboxdrv]
[21657.434339]  [<ffffffff81256763>] do_vfs_ioctl+0xa3/0x5d0
[21657.435720]  [<ffffffff81256d09>] SyS_ioctl+0x79/0x90
[21657.437123]  [<ffffffff817cd0ee>] entry_SYSCALL_64_fastpath+0x12/0x6d
[21657.438483] Code: 40 48 83 c2 08 48 39 c8 75 f0 b9 01 00 00 00 49 8b 7e 50 41 88 4e 40 48 8b 07 f6 c4 40 74 0e 48 c7 c6 30 3c d9 a0 e8 f8 4c 47 e0 <0f> 0b 48 8b 47 20 a8 01 75 ea 31 c0 f0 80 4f 01 04 48 83 c0 01 
[21657.439889] RIP  [<ffffffffa0d6eb68>] rtR0MemObjLinuxAllocPages+0x178/0x290 [vboxdrv]
[21657.441241]  RSP <ffff880475fefc98>
[21657.442656] ---[ end trace b8e1aa855bc172b9 ]---

[frank]

Please could you attach the corresponding vboxdrv.ko kernel module to this ticket? Thanks.

[Zdenek Sedlak]

Sure, here it comes.

Well, I tried, but: File (size limit 512.0 KB):

The *.ko packed with xz has still 1.6M

[frank]

Could you send it to me via private e-mail (frank _dot_ mehnert _at_ oracle _dot_ com)?

[Zdenek Sedlak]

Yep, sent the e-mail right now...

[frank]

Thanks. I'm now able to reproduce the problem. These panics do only happen due to additional sanity checks which are enabled in the current FC24 kernels (CONFIG_DEBUG_VM=y, CONFIG_DEBUG_VM_PGFLAGS=y). We need to think about how to adapt the code.

[g_tirloni]

I can consistently cause this crash (with exact same stack trace) by booting a VM with 2 or more CPUs. Everything is fine if the VM has only one CPU.

Tested on FC24 host, VirtualBox 5.0.16 and kernel 4.5.0-302

[frank]

One or two CPUs shouldn't really matter. r60372 should fix the problem.

[Zdenek Sedlak]

Is there any fixed version available? Like 5.0.17, so I could test it?

[frank]

The latest universal Linux builds from the test builds page contain the fix and here is a Fedora 24 package.

[Zdenek Sedlak]

I can confirm that the 5.0.17 fixed the problem

[frank]

Thanks for testing!

[Zdenek Sedlak]

Well, this is strange - got a kernel update to 4.5.1 and the error is back:

[ 2939.805490] ------------[ cut here ]------------
[ 2939.805512] kernel BUG at include/linux/page-flags.h:272!
[ 2939.805526] invalid opcode: 0000 [#2] SMP 
[ 2939.805538] Modules linked in: vboxpci(OE) vboxnetadp(OE) vboxnetflt(OE) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat tun fuse bridge ebtable_filter ebtables ip6table_filter ip6_tables nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack vboxdrv(OE) nf_conntrack b43 mac80211 cfg80211 pn544_mei mei_phy pn544 hci ssb nfc iTCO_wdt iTCO_vendor_support ppdev dell_wmi snd_hda_codec_hdmi sparse_keymap dell_laptop dcdbas dell_smm_hwmon intel_rapl x86_pkg_temp_thermal coretemp kvm_intel kvm snd_hda_codec_realtek snd_hda_codec_generic snd_soc_rt5640 snd_soc_rl6231 irqbypass snd_soc_core snd_hda_intel snd_hda_codec snd_hda_core snd_compress snd_pcm_dmaengine snd_hwdep ac97_bus snd_seq joydev i2c_i801 snd_seq_device snd_pcm bcma snd_timer snd mei_me ie31200_edac mei
[ 2939.805754]  lpc_ich edac_core shpchp soundcore elan_i2c parport_pc parport i2c_designware_platform dell_rbtn dell_smo8800 snd_soc_sst_acpi dw_dmac i2c_designware_core spi_pxa2xx_platform snd_soc_sst_match tpm_tis dw_dmac_core rfkill tpm nfsd binfmt_misc auth_rpcgss nfs_acl lockd grace xfs libcrc32c dm_crypt 8021q garp stp llc mrp nouveau uas usb_storage crct10dif_pclmul crc32_pclmul crc32c_intel mxm_wmi i2c_algo_bit ghash_clmulni_intel drm_kms_helper ttm serio_raw e1000e drm sdhci_pci ptp pps_core sdhci_acpi wmi sdhci mmc_core video i2c_hid fjes sunrpc scsi_transport_iscsi [last unloaded: vboxnetflt]
[ 2939.805928] CPU: 4 PID: 19523 Comm: EMT-0 Tainted: G      D    OE   4.5.1-300.fc24.x86_64 #1
[ 2939.805948] Hardware name: Dell Inc. Precision M6800/0VCHYK, BIOS A16 12/01/2015
[ 2939.805965] task: ffff88071de4db80 ti: ffff88071dbe4000 task.ti: ffff88071dbe4000
[ 2939.805983] RIP: 0010:[<ffffffffa0aaab68>]  [<ffffffffa0aaab68>] rtR0MemObjLinuxAllocPages+0x178/0x290 [vboxdrv]
[ 2939.806019] RSP: 0018:ffff88071dbe7c98  EFLAGS: 00010246
[ 2939.806032] RAX: 000000000000003c RBX: 0000000000000002 RCX: 0000000000000006
[ 2939.806049] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff88082dd0dfb0
[ 2939.806065] RBP: ffff88071dbe7cd0 R08: 0000000000000005 R09: 00000000000004ad
[ 2939.806082] R10: ffffffff81a80459 R11: ffffffff81f400cd R12: 0000000000002000
[ 2939.806099] R13: 00000000024200c2 R14: ffff88076975f410 R15: 0000000000000000
[ 2939.806116] FS:  00007fd4efc51700(0000) GS:ffff88082dd00000(0000) knlGS:0000000000000000
[ 2939.806134] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 2939.806167] CR2: 00007fd50a613808 CR3: 000000072219a000 CR4: 00000000001406e0
[ 2939.809451] Stack:
[ 2939.810653]  ffff88071dbe7cf0 0000000000001000 0000000000000001 0000000000000246
[ 2939.812005]  ffff88071dbe7d50 ffff8808029f6690 ffff8807132dc810 ffff88071dbe7d18
[ 2939.814666]  ffffffffa0aab0cf 00000000fffffff8 00000000000081c0 ffff88071dbe7d80
[ 2939.816006] Call Trace:
[ 2939.817158]  [<ffffffffa0aab0cf>] rtR0MemObjNativeAllocPage+0x4f/0xb0 [vboxdrv]
[ 2939.818210]  [<ffffffffa0aa8a72>] VBoxHost_RTR0MemObjAllocPageTag+0x62/0x70 [vboxdrv]
[ 2939.819261]  [<ffffffffa0a994d9>] SUPR0PageAllocEx+0x149/0x2d0 [vboxdrv]
[ 2939.820273]  [<ffffffffa0a9ce57>] supdrvIOCtl+0x17f7/0x32c0 [vboxdrv]
[ 2939.821260]  [<ffffffffa0a975c0>] VBoxDrvLinuxIOCtl_5_0_16+0x150/0x250 [vboxdrv]
[ 2939.822235]  [<ffffffff81256863>] do_vfs_ioctl+0xa3/0x5d0
[ 2939.823192]  [<ffffffff81256e09>] SyS_ioctl+0x79/0x90
[ 2939.824142]  [<ffffffff817cd2ae>] entry_SYSCALL_64_fastpath+0x12/0x6d
[ 2939.825145] Code: 40 48 83 c2 08 48 39 c8 75 f0 b9 01 00 00 00 49 8b 7e 50 41 88 4e 40 48 8b 07 f6 c4 40 74 0e 48 c7 c6 68 fe ac a0 e8 f8 8b 73 e0 <0f> 0b 48 8b 47 20 a8 01 75 ea 31 c0 f0 80 4f 01 04 48 83 c0 01 
[ 2939.827966] RIP  [<ffffffffa0aaab68>] rtR0MemObjLinuxAllocPages+0x178/0x290 [vboxdrv]
[ 2939.829015]  RSP <ffff88071dbe7c98>
[ 2939.830113] ---[ end trace 82f6c325e48a7ea2 ]---

[Zdenek Sedlak]

Hmm, just to be sure, I tried to recompile the vbox modules using rcvboxdrv script and suddenly it works again....

[frank]

Please reopen if still relevant with VBox 5.0.18.