X.Org crashes when starting VirtualBox VM with dual display setup

[Frans Pop]

The X.Org crash itself is likely to be an X.Org issue, but it looks as if there may be an issue in VirtualBox as well.

I have described the circumstances of the crash in detail here: ​http://bugs.freedesktop.org/show_bug.cgi?id=19643

The reason that I think there's also a VirtualBox issue is that with 2.6.29-rc2 I also get a kernel WARN just before the crash:

kernel: vboxdrv: Trying to deactivate the NMI watchdog permanently...
kernel: vboxdrv: Successfully done.
kernel: vboxdrv: Found 2 processor cores.
kernel: VBoxDrv: dbg - g_abExecMemory=ffffffffa0572140
kernel: vboxdrv: fAsync=0 offMin=0x19a offMax=0x776
kernel: vboxdrv: TSC mode is 'synchronous', kernel timer mode is 'normal'.
kernel: vboxdrv: Successfully loaded version 2.1.0_OSE (interface 0x000a0008).
kernel: ------------[ cut here ]------------
kernel: WARNING: at arch/x86/mm/pat.c:620 reserve_pfn_range+0x5b/0x26d()
kernel: Hardware name: HP Compaq 2510p Notebook PC
kernel: Modules linked in: vboxnetflt vboxdrv tcp_diag inet_diag i915 drm i2c_algo_bit i2c_core ppdev parport_pc lp parport ipv6 nfs lockd nfs_acl auth_rpcgss sunrpc ext2 coretemp hp_wmi acpi_cpufreq loop snd_hda_codec_analog snd_hda_intel snd_hda_codec snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss arc4 snd_seq_midi snd_rawmidi ecb snd_seq_midi_event snd_seq iwlagn iwlcore snd_timer snd_seq_device rfkill snd lib80211 mac80211 serio_raw soundcore pcmcia snd_page_alloc hp_accel cfg80211 psmouse pcspkr iTCO_wdt yenta_socket rsrc_nonstatic pcmcia_core usblp intel_agp wmi video output lis3lv02d container battery ac button joydev evdev ext3 jbd mbcache sha256_generic aes_x86_64 aes_generic cbc usbhid hid dm_crypt dm_mirror dm_region_hash dm_log dm_snapshot dm_mod sg sr_mod cdrom sd_mod piix ide_pci_generic ata_piix ide_core pata_acpi ohci1394 sdhci_pci sdhci ieee1394 mmc_core led_class ricoh_mmc ata_generic ehci_hcd libata uhci_hcd e1000e scsi_mod thermal processor fan thermal_sys
kernel: Pid: 29812, comm: VirtualBox Tainted: G        W  2.6.29-rc2 #7
kernel: Call Trace:
kernel:  [<ffffffff802398cf>] warn_slowpath+0xd3/0x10f
kernel:  [<ffffffff8027bb33>] ? shmem_alloc_inode+0x15/0x2a
kernel:  [<ffffffff8024e701>] ? __mutex_init+0x4d/0x58
kernel:  [<ffffffff80323a40>] ? __init_rwsem+0x29/0x39
kernel:  [<ffffffff802a896d>] ? inode_init_always+0x146/0x1db
kernel:  [<ffffffff804467a2>] ? _spin_unlock+0x9/0xb
kernel:  [<ffffffff802aac76>] ? mnt_want_write+0x7a/0x8d
kernel:  [<ffffffff802255fa>] ? pagerange_is_ram+0x31/0x5c
kernel:  [<ffffffff80227b65>] reserve_pfn_range+0x5b/0x26d
kernel:  [<ffffffff80227dc2>] track_pfn_vma_new+0x4b/0xb7
kernel:  [<ffffffff8028357f>] remap_pfn_range+0x8e/0x2eb
kernel:  [<ffffffffa056b3c1>] rtR0MemObjNativeMapUser+0x14c/0x267 [vboxdrv]
kernel:  [<ffffffffa056903e>] RTR0MemObjMapUser+0x130/0x1dc [vboxdrv]
kernel:  [<ffffffffa05651ea>] SUPR0GipMap+0x125/0x216 [vboxdrv]
kernel:  [<ffffffffa05682de>] supdrvIOCtl+0x101c/0x152c [vboxdrv]
kernel:  [<ffffffffa056a89e>] ? rtMemAlloc+0x92/0xc7 [vboxdrv]
kernel:  [<ffffffffa056a805>] ? rtMemFree+0x65/0x6c [vboxdrv]
kernel:  [<ffffffffa0564260>] VBoxDrvLinuxIOCtl+0x120/0x19f [vboxdrv]
kernel:  [<ffffffff802a2504>] vfs_ioctl+0x2a/0x78
kernel:  [<ffffffff802a29d9>] do_vfs_ioctl+0x46b/0x4ab
kernel:  [<ffffffff80296650>] ? fsnotify_access+0x62/0x6a
kernel:  [<ffffffff802a2a6e>] sys_ioctl+0x55/0x77
kernel:  [<ffffffff8020c25b>] system_call_fastpath+0x16/0x1b
kernel: ---[ end trace 7b44d6d0be5818c3 ]---
kernel: Xorg[4039]: segfault at 1b8 ip 000000000054d1b5 sp 00007fff894e47a0 error 4 in Xorg[400000+1bf000]
kdm[4029]: X server for display :0 terminated unexpectedly

If I look at arch/x86/mm/pat.c line 620, I see:

static int reserve_pfn_range(u64 paddr, unsigned long size, pgprot_t *vma_prot,
                                int strict_prot)
{
        int is_ram = 0;
        int id_sz, ret;
        unsigned long flags;
        unsigned long want_flags = (pgprot_val(*vma_prot) & _PAGE_CACHE_MASK);

        is_ram = pagerange_is_ram(paddr, paddr + size);

        if (is_ram != 0) {
                /*
                 * For mapping RAM pages, drivers need to call
                 * set_memory_[uc|wc|wb] directly, for reserve and free, before
                 * setting up the PTE.
                 */
                WARN_ON_ONCE(1);
                return 0;
        }

Looking at that comment, is there maybe a call to set_memory_[uc|wc|wb] missing before the call to remap_pfn_range in r0drv/linux/memobj-r0drv-linux.c around line 1074 (in function rtR0MemObjNativeMapUser)?

[Frans Pop]

I'm now also seeing the WARN witout the X.Org crash, so it looks like they are separate issues.

[Frans Pop]

The warning happens every time I first start a VBox guest and can still be reproduced with 2.6.29-rc5. The X.Org crash has now been solved in the xserver and is thus definitely unrelated.

It looks like this warning could be fixed by the following kernel patch: ​http://lkml.org/lkml/2009/2/11/317

However, if I apply that patch on top of 2.6.29-rc5, I get:

Failed to load VMMR0.r0 (VERR_SYMBOL_NOT_FOUND).
Unknown error creating VM (VERR_SYMBOL_NOT_FOUND).

Result Code: 
NS_ERROR_FAILURE (0x80004005)
Component: 
Console
Interface: 
IConsole {e3c6d4a1-a935-47ca-b16d-f9e9c496e53e}

And the end of the log shows:

00:00:00.749 102: RTLogLogger
00:00:00.749 103: RTLogLoggerEx
00:00:00.749 104: RTLogLoggerExV
00:00:00.749 105: RTLogPrintf
00:00:00.749 106: RTLogPrintfV
00:00:00.749 107: AssertMsg1
00:00:00.749 108: AssertMsg2
00:00:00.749 VMMR0.r0 is importing g_SUPGlobalInfoPage which we couldn't find
00:00:00.749 pdmR3LoadR0U: pszName="VMMR0.r0" rc=VERR_SYMBOL_NOT_FOUND
00:00:00.751 ERROR [COM]: aRC=NS_ERROR_FAILURE (0x80004005) aIID={e3c6d4a1-a935-47ca-b16d-f9e9c496e53e} aComponent={Console} aText={Failed to load VMMR0.r0 (VERR_SYMBOL_NOT_FOUND).
00:00:00.751 Unknown error creating VM (VERR_SYMBOL_NOT_FOUND)} aWarning=false, preserve=false
00:00:00.761 Power up failed (vrc=VERR_SYMBOL_NOT_FOUND, rc=NS_ERROR_FAILURE (0X80004005))

Both vbox modules build and load correctly. The failure occurs when the guest is started. If I boot this same kernel with the "nopat" option, the guest starts correctly.

I've looked as include/VBox/sup.h where g_SUPGlobalInfoPage is defined, but that's where it gets over my head. I don't see how the kernel patch would cause this failure.

[Frans Pop]

The patch mentioned in my previous comment has now been included in the upstream kernel (commit be03d9e8022030c16abf5). This means that I now always get the "Failed to load VMMR0.r0" error if I boot the host kernel with pat enabled.

I'm still using VBox 2.1.2 (Debian's -ose package).

[Frank Mehnert]

Let's continue talking on #3403.