PAX-related trace in dmesg on Gentoo AMD64

[youngage]

Commands issued:

youngage ~ $ rmmod vboxdrv
youngage ~ $ rmmod vboxnetflt
youngage ~ $ modprobe vboxdrv
youngage ~ $ modprobe vboxnetflt
youngage ~ $ VBoxManage createvm -name Test -register
VirtualBox Command Line Management Interface Version 2.2.0
(C) 2005-2009 Sun Microsystems, Inc.
All rights reserved.

Virtual machine 'Test' is created and registered.
UUID: 0c260006-0eef-4a15-bdad-b07ef31f5485
Settings file: '/root/.VirtualBox/Machines/Test/Test.xml'
youngage ~ $ VBoxHeadless -s Test
VirtualBox Headless Interface 2.2.0
(C) 2008-2009 Sun Microsystems, Inc.
All rights reserved.

Listening on port 3389

dmesg:

[  113.355487] vboxdrv: Trying to deactivate the NMI watchdog permanently...
[  113.355492] vboxdrv: Successfully done.
[  113.355494] vboxdrv: Found 1 processor cores.
[  113.355626] VBoxDrv: dbg - g_abExecMemory=ffffffffa01c1000
[  113.355674] vboxdrv: TSC mode is 'synchronous', kernel timer mode is 'normal'.
[  113.355676] vboxdrv: Successfully loaded version 2.2.0 (interface 0x000a0009).
[  117.444748] VBoxNetFlt: dbg - g_abExecMemory=ffffffffa0360000
[  349.179253] PAX: From 89.245.105.223: VBoxHeadless:8353, uid/euid: 0/0, attempted to modify kernel code
[  349.179522] BUG: unable to handle kernel paging request at ffffffffa01cfc00
[  349.179757] IP: [<ffffffffa01cfc00>] 0xffffffffa01cfc00
[  349.179950] PGD 721067 PUD 72a063 PMD 7dab2063 PTE 800000007d163163
[  349.180005] Oops: 0011 [#1] SMP 
[  349.180005] last sysfs file: /sys/class/net/vboxnet0/address
[  349.180005] CPU 0 
[  349.180005] Modules linked in: vboxnetflt vboxdrv k8temp [last unloaded: vboxdrv]
[  349.180005] Pid: 8353, comm: VBoxHeadless Not tainted 2.6.28-hardened-r7 #3
[  349.180005] RIP: 0010:[<ffffffffa01cfc00>]  [<ffffffffa01cfc00>] 0xffffffffa01cfc00
[  349.180005] RSP: 0018:ffff88007d183db0  EFLAGS: 00010286
[  349.180005] RAX: ffffffffa01cfc00 RBX: ffffffffa01c1070 RCX: ffffffffa01cfd70
[  349.180005] RDX: ffffffffa01c10e0 RSI: ffffffffa01cfd80 RDI: ffffffffa01d0810
[  349.180005] RBP: ffffc20001785010 R08: 000000000000f730 R09: 00000000ffffffff
[  349.180005] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000000
[  349.180005] R13: ffffffffa03435a0 R14: 0000000000000494 R15: ffff88007d102810
[  349.180005] FS:  00000000406f4950(0063) GS:ffffffff80bad040(0000) knlGS:0000000000000000
[  349.180005] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[  349.180005] CR2: ffffffffa01cfc00 CR3: 0000000075183000 CR4: 00000000000006e0
[  349.180005] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  349.180005] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  349.180005] Process VBoxHeadless (pid: 8353, threadinfo ffff88007d182000, task ffff88007d0fa680)
[  349.180005] Stack:
[  349.180005]  ffffffffa034ba8b 8000000000000163 0000000000005686 ffffc20001785078
[  349.180005]  ffffc200017e1478 0000000000065018 0000000000000000 0000000000065018
[  349.180005]  0000000000005686 000072ba9beab010 ffffffffa034e66f ffff88007519f600
[  349.180005] Call Trace:
[  349.180005]  [<ffffffffa034ba8b>] ? 0xffffffffa034ba8b
[  349.180005]  [<ffffffffa034e66f>] ? 0xffffffffa034e66f
[  349.180005]  [<ffffffffa0348231>] ? 0xffffffffa0348231
[  349.180005]  [<ffffffff802880f5>] ? 0xffffffff802880f5
[  349.180005]  [<ffffffff802afbfe>] ? 0xffffffff802afbfe
[  349.180005]  [<ffffffff802affc2>] ? 0xffffffff802affc2
[  349.180005]  [<ffffffff802b005a>] ? 0xffffffff802b005a
[  349.180005]  [<ffffffff80218c4b>] ? 0xffffffff80218c4b
[  349.180005] Code: f7 c3 cc cd f8 c3 cc cd f9 c3 cc cd fa c3 cc cd fb c3 cc cd fc c3 cc cd fd c3 cc cd fe c3 cc cd ff c3 cc cc cc cc 90 90 90 90 90 <55> 48 89 e5 53 48 83 ec 08 e8 62 49 ff ff 85 c0 89 c3 78 66 e8 
[  349.180005] RIP  [<ffffffffa01cfc00>] 0xffffffffa01cfc00
[  349.180005]  RSP <ffff88007d183db0>
[  349.180005] CR2: ffffffffa01cfc00
[  349.180005] ---[ end trace 216d471fba277aea ]---

Abort VBoxHeadless with CTRL+C and type:

youngage ~ $ VBoxTunctl
Set 'tap0' persistent and owned by uid 0
youngage ~ $ VBoxManage modifyvm Test --nic1 bridged --bridgeadapter1 tap0
VirtualBox Command Line Management Interface Version 2.2.0
(C) 2005-2009 Sun Microsystems, Inc.
All rights reserved.

youngage ~ $ VBoxHeadless -s Test
VirtualBox Headless Interface 2.2.0
(C) 2008-2009 Sun Microsystems, Inc.
All rights reserved.

Listening on port 3389

But despite the message the port is not open and the VM is not stoppable by VBoxManage controlvm Test poweroff.

PAX was configured by Gentoo Portage:

 paxctl -v /opt/VirtualBox/VBoxHeadless
PaX control v0.5
Copyright 2004,2005,2006,2007 PaX Team <pageexec@freemail.hu>

- PaX flags: -----m-x-e-- [/opt/VirtualBox/VBoxHeadless]
        MPROTECT is disabled
        RANDEXEC is disabled
        EMUTRAMP is disabled

Issuing "paxctl -pemcxs /opt/VirtualBox/VBoxHeadless" made no difference.

[youngage]

Wanted to add the VBox.log but got a TRAC error with a Python trace. The content of VBox.log is short:

00:00:00.144 VirtualBox 2.2.0 r45846 linux.amd64 (Apr  7 2009 21:26:29) release log
00:00:00.144 Log opened 2009-04-10T02:01:36.710457000Z
00:00:00.144 OS Product: Linux
00:00:00.144 OS Release: 2.6.28-hardened-r7
00:00:00.144 OS Version: #3 SMP Fri Apr 10 03:47:38 CEST 2009
00:00:00.144 Package type: LINUX_64BITS_GENERIC
00:00:00.188 VRDP: TCP server listening on port 3389.

[youngage]

Disabling PAX (and GrSecurity) in the kernel results in the whole machine freezing without any comment (e.g. on console) - need to hard reset.

[Paul Kronenwetter]

I'm seeing the same with VB 3.0.0. Though I am able to recompile the kernel without PAX (but still with grsecurity) and it boots just fine.

[Frank Mehnert]

Marking this as duplicate of #2652 as that ticket is older.