Opened 4 years ago
Last modified 4 years ago
#20163 new defect
kernel 5.10.11 panic at VM start, when CONFIG_DEBUG_SPINLOCK=y in kernel .config
| Reported by: | ozugzug | Owned by: | |
|---|---|---|---|
| Component: | other | Version: | VirtualBox 6.1.18 |
| Keywords: | Cc: | ||
| Guest type: | X11 | Host type: | Linux |
Description
this is on app-emulation/virtualbox-6.1.18::gentoo
kernel 5.10.11
No panic when CONFIG_DEBUG_SPINLOCK=n (which means CONFIG_DEBUG_LOCK_ALLOC=n also)
Also panic when using CONFIG_DEBUG_LOCK_ALLOC=y (which forces CONFIG_DEBUG_SPINLOCK=y)
Still panic when CONFIG_DEBUG_SPINLOCK=y and CONFIG_DEBUG_LOCK_ALLOC=n
This has been mentioned in a comment(thanks!) in https://www.virtualbox.org/ticket/20055#comment:16
I'm happy to test any patches, or if you think the issue is on the kernel side...
Here are my notes, I'm not sure how useful all this is but I'm happy to do more things if you tell me what:
Using /usr/src/.config.prev18_DEBUGlocks
KERNEL: /var/crash/vmlinux-5.10.11-gentoo-x86_64-2021-01-29-01_35_09
DUMPFILE: /var/crash/crashdump-2021-01-29-01_42_27 [PARTIAL DUMP]
Below, the word "crash" references https://github.com/crash-utility/crash
/ dmesg seen by crash's 'log' command:
... [ 27.789537] zram1: detected capacity change from 0 to 206158430208 [ 27.792783] zram2: detected capacity change from 0 to 206158430208 [ 27.807234] vboxdrv: loading out-of-tree module taints kernel. [ 27.807944] vboxdrv: Found 12 processor cores [ 27.808318] ------------[ cut here ]------------ [ 27.808318] DEBUG_LOCKS_WARN_ON(lockdep_hardirqs_enabled()) [ 27.808325] WARNING: CPU: 4 PID: 12692 at kernel/locking/lockdep.c:5281 check_flags.part.0+0x157/0x160 [ 27.808325] Modules linked in: vboxdrv(O+) pcspkr [ 27.808328] CPU: 4 PID: 12692 Comm: modprobe Tainted: G U O 5.10.11-gentoo-x86_64 #1 [ 27.808329] Hardware name: System manufacturer System Product Name/PRIME Z370-A, BIOS 2401 07/12/2019 [ 27.808330] RIP: 0010:check_flags.part.0+0x157/0x160 [ 27.808331] Code: c0 0f 84 3f 9c 04 01 44 8b 0d 0d cf 7c 02 45 85 c9 0f 85 2f 9c 04 01 48 c7 c6 2f 22 d3 9a 48 c7 c7 cb e5 d1 9a e8 82 52 04 01 <0f> 0b e9 15 9c 04 01 66 90 41 57 41 56 49 89 fe 41 55 41 89 d5 41 [ 27.808332] RSP: 0018:ffffb32948e37a10 EFLAGS: 00010082 [ 27.808333] RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 [ 27.808334] RDX: 0000000000000000 RSI: ffffffff991424fc RDI: ffffffff9b065d40 [ 27.808334] RBP: ffffffff9b768300 R08: 0000000000000001 R09: 0000000000000000 [ 27.808335] R10: 0000000000000000 R11: fffffffffe02e9f8 R12: 0000000000000001 [ 27.808335] R13: 0000000000000000 R14: ffffffff9b76c688 R15: 0000000000000046 [ 27.808336] FS: 00007bcdb219fb80(0000) GS:ffff996609800000(0000) knlGS:0000000000000000 [ 27.808337] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 27.808338] CR2: 00007bcdb232f445 CR3: 00000002d416e005 CR4: 00000000001706e0 [ 27.808338] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 27.808339] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 27.808339] Call Trace: [ 27.808342] ? lock_is_held_type+0x65/0x110 [ 27.808344] ? rcu_read_lock_sched_held+0x3a/0x70 [ 27.808345] ? lock_acquire+0x39a/0x480 [ 27.808347] ? ktime_get_ts64+0x46/0x280 [ 27.808355] ? VBoxHost_RTTimeSystemNanoTS+0x30/0x60 [vboxdrv] [ 27.808360] ? VBoxHost_RTTimeSystemNanoTS+0x30/0x60 [vboxdrv] [ 27.808365] ? supdrvLdrGetExportedSymbol+0x3342/0x3640 [vboxdrv] [ 27.808369] ? supdrvGipCreate+0x820/0xc50 [vboxdrv] [ 27.808370] ? rcu_read_lock_sched_held+0x3a/0x70 [ 27.808371] ? lockdep_init_map_waits+0x42/0x200 [ 27.808375] ? supdrvInitDevExt+0x14a/0x310 [vboxdrv] [ 27.808380] ? init_module+0x91/0x1000 [vboxdrv] [ 27.808380] ? 0xffffffffc0297000 [ 27.808382] ? do_one_initcall+0x7a/0x2d0 [ 27.808383] ? do_init_module+0x57/0x230 [ 27.808384] ? load_module+0x2445/0x26e0 [ 27.808386] ? __do_sys_finit_module+0xc0/0x100 [ 27.808387] ? do_syscall_64+0x33/0x40 [ 27.808388] ? entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 27.808388] irq event stamp: 5085 [ 27.808390] hardirqs last enabled at (5085): [<ffffffff99167cd5>] ktime_get_ts64+0x205/0x280 [ 27.808390] hardirqs last disabled at (5084): [<ffffffff99167c9d>] ktime_get_ts64+0x1cd/0x280 [ 27.808391] softirqs last enabled at (5000): [<ffffffff9a400eff>] asm_call_irq_on_stack+0xf/0x20 [ 27.808392] softirqs last disabled at (4995): [<ffffffff9a400eff>] asm_call_irq_on_stack+0xf/0x20 [ 27.808392] ---[ end trace 3f8fdc7480d9e05c ]--- [ 27.808393] possible reason: unannotated irqs-off. [ 27.808393] irq event stamp: 5085 [ 27.808394] hardirqs last enabled at (5085): [<ffffffff99167cd5>] ktime_get_ts64+0x205/0x280 [ 27.808395] hardirqs last disabled at (5084): [<ffffffff99167c9d>] ktime_get_ts64+0x1cd/0x280 [ 27.808395] softirqs last enabled at (5000): [<ffffffff9a400eff>] asm_call_irq_on_stack+0xf/0x20 [ 27.808396] softirqs last disabled at (4995): [<ffffffff9a400eff>] asm_call_irq_on_stack+0xf/0x20 [ 27.823374] zram3: detected capacity change from 0 to 206158430208 [ 27.824878] vboxdrv: TSC mode is Invariant, tentative frequency 3714564013 Hz [ 27.830217] xhci_hcd 0000:00:14.0: Cancel URB 00000000493c99df, dev 14, ep 0x81, starting at offset 0xfff92010 [ 27.831594] vboxdrv: Successfully loaded version 6.1.18 (interface 0x00300000) [ 27.833271] xhci_hcd 0000:00:14.0: // Ding dong! ... [ 57.854519] elogind-daemon[13194]: New session 2 of user user. [ 62.295045] i2c i2c-1: NAK from device addr 0x50 msg #0 [ 62.298587] i2c i2c-0: NAK from device addr 0x50 msg #0 [ 62.302129] i2c i2c-2: NAK from device addr 0x50 msg #0 [ 62.331645] snd_hda_intel 0000:00:1f.3: power state changed by ACPI to D0 [ 62.344256] snd_hda_intel 0000:00:1f.3: PME# disabled [ 71.473189] snd_hda_intel 0000:00:1f.3: saving config space at offset 0x0 (reading 0xa2f08086) [ 71.473206] snd_hda_intel 0000:00:1f.3: saving config space at offset 0x4 (reading 0x100406) [ 71.473215] snd_hda_intel 0000:00:1f.3: saving config space at offset 0x8 (reading 0x4030000) [ 71.473223] snd_hda_intel 0000:00:1f.3: saving config space at offset 0xc (reading 0x2010) [ 71.473230] snd_hda_intel 0000:00:1f.3: saving config space at offset 0x10 (reading 0xfff20004) [ 71.473238] snd_hda_intel 0000:00:1f.3: saving config space at offset 0x14 (reading 0x2f) [ 71.473258] snd_hda_intel 0000:00:1f.3: saving config space at offset 0x18 (reading 0x0) [ 71.473260] snd_hda_intel 0000:00:1f.3: saving config space at offset 0x1c (reading 0x0) [ 71.473263] snd_hda_intel 0000:00:1f.3: saving config space at offset 0x20 (reading 0xfff00004) [ 71.473265] snd_hda_intel 0000:00:1f.3: saving config space at offset 0x24 (reading 0x2f) [ 71.473267] snd_hda_intel 0000:00:1f.3: saving config space at offset 0x28 (reading 0x0) [ 71.473270] snd_hda_intel 0000:00:1f.3: saving config space at offset 0x2c (reading 0x87241043) [ 71.473272] snd_hda_intel 0000:00:1f.3: saving config space at offset 0x30 (reading 0x0) [ 71.473274] snd_hda_intel 0000:00:1f.3: saving config space at offset 0x34 (reading 0x50) [ 71.473277] snd_hda_intel 0000:00:1f.3: saving config space at offset 0x38 (reading 0x0) [ 71.473279] snd_hda_intel 0000:00:1f.3: saving config space at offset 0x3c (reading 0x10b) [ 71.473297] snd_hda_intel 0000:00:1f.3: PME# enabled [ 71.485469] snd_hda_intel 0000:00:1f.3: power state changed by ACPI to D3hot [ 72.024850] SUPR0GipMap: fGetGipCpu=0x1b [ 73.474107] BUGGY: kernel NULL pointer dereference, address: 0000000000000004 [ 73.474120] #PF: supervisor read access in kernel mode [ 73.474126] #PF: error_code(0x0000) - not-present page [ 73.474131] PGD 0 P4D 0 [ 73.474143] Oops: 0000 [#1] SMP NOPTI [ 73.474151] CPU: 2 PID: 14833 Comm: EMT-0 Kdump: loaded Tainted: G U W O 5.10.11-gentoo-x86_64 #1 [ 73.474157] Hardware name: System manufacturer System Product Name/PRIME Z370-A, BIOS 2401 07/12/2019 [ 73.474174] RIP: 0010:do_raw_spin_lock+0x4/0x90 [ 73.474181] Code: 48 8d 88 b8 06 00 00 48 c7 c7 58 48 d3 9a e8 79 3a 04 01 e9 5b 3b 06 01 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 53 48 89 fb <8b> 47 04 3d ad 4e ad de 75 46 48 8b 53 10 65 48 8b 04 25 00 6f 01 [ 73.474187] RSP: 0018:ffffb3294a0ebd30 EFLAGS: 00010296 [ 73.474194] RAX: ffffffff992838a1 RBX: 0000000000000000 RCX: 0000000000000000 [ 73.474199] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 [ 73.474204] RBP: ffffb3294a0ebdf8 R08: 0000000000000001 R09: 0000000000000000 [ 73.474209] R10: ffffffffc026f760 R11: 000000000007b438 R12: ffffffffc0240080 [ 73.474213] R13: ffffb3294bea1000 R14: 0000000000000001 R15: ffffb3294a0ebde0 [ 73.474220] FS: 0000754d3c1f4640(0000) GS:ffff996609600000(0000) knlGS:0000000000000000 [ 73.474225] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 73.474230] CR2: 0000000000000004 CR3: 00000002d7102004 CR4: 00000000001706e0 [ 73.474234] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 73.474239] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 73.474243] Call Trace: [ 73.474255] ? __apply_to_page_range+0x2e1/0x6a0 [ 73.474286] ? rtR0TermNative+0xd0/0x220 [vboxdrv] [ 73.474313] ? rtR0MemObjNativeProtect+0x74/0xa0 [vboxdrv] [ 73.474338] ? VBoxHost_RTR0MemObjProtect+0x81/0xc0 [vboxdrv] [ 73.474360] ? supdrvIOCtl+0x3265/0x3800 [vboxdrv] [ 73.474379] ? SUPR0Printf+0x22f/0x330 [vboxdrv] [ 73.474388] ? __x64_sys_ioctl+0x7e/0xb0 [ 73.474395] ? do_syscall_64+0x33/0x40 [ 73.474402] ? entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 73.474407] Modules linked in: vboxnetadp(O) vboxnetflt(O) vboxdrv(O) pcspkr [ 73.474425] CR2: 0000000000000004
(it ended here)
/ Partial output of crash's "bt -sFlxg":
PID: 14833 TASK: ffff9959886acf80 CPU: 2 COMMAND: "EMT-0"
#0 [ffffb3294a0eba68] machine_kexec+0x191 at ffffffff9904d3b1
/usr/src/linux-5.10.11-gentoo/arch/x86/include/asm/mem_encrypt.h: 77
ffffb3294a0eba70: 0000000000000018 0000ffff99574000
ffffb3294a0eba80: ffff995740000000 0000000f0a001000
ffffb3294a0eba90: ffff99664a001000 0000000f0a000000
ffffb3294a0ebaa0: 0000000000000000 cc4567c5b1908400
ffffb3294a0ebab0: ffffb3294a0ebc88 ffffb3294a0ebc88
ffffb3294a0ebac0: 0000000000000009 __crash_kexec+225
#1 [ffffb3294a0ebac8] __crash_kexec+0xe1 at ffffffff99188a21
/usr/src/linux-5.10.11-gentoo/kernel/kexec_core.c: 963
ffffb3294a0ebad0: ffffb3294a0ebde0 0000000000000001
ffffb3294a0ebae0: ffffb3294bea1000 rtR0TermNative+208
ffffb3294a0ebaf0: ffffb3294a0ebdf8 0000000000000000
ffffb3294a0ebb00: 000000000007b438 __this_module+992
ffffb3294a0ebb10: 0000000000000000 0000000000000001
ffffb3294a0ebb20: __apply_to_page_range+737 0000000000000000
ffffb3294a0ebb30: 0000000000000000 0000000000000000
ffffb3294a0ebb40: 0000000000000000 ffffffffffffffff
ffffb3294a0ebb50: do_raw_spin_lock+4 0000000000000010
ffffb3294a0ebb60: 0000000000010296 ffffb3294a0ebd30
ffffb3294a0ebb70: 0000000000000018 cc4567c5b1908400
ffffb3294a0ebb80: 0000000000000046 crash_kexec+52
#2 [ffffb3294a0ebb88] crash_kexec+0x34 at ffffffff99189954
/usr/src/linux-5.10.11-gentoo/arch/x86/include/asm/atomic.h: 41
ffffb3294a0ebb90: ffffb3294a0ebc88 oops_end+132
#3 [ffffb3294a0ebb98] oops_end+0x84 at ffffffff99022034
/usr/src/linux-5.10.11-gentoo/arch/x86/kernel/dumpstack.c: 359
ffffb3294a0ebba0: [task_struct] ffffb3294a0ebc88
ffffb3294a0ebbb0: 0000000000000004 no_context+572
ffffb3294a0ebbc0: 0000000000000000 contig_page_data+3600
ffffb3294a0ebbd0: 0000000000000004 0000000000000000
ffffb3294a0ebbe0: 0000000000000000 0000000000000000
ffffb3294a0ebbf0: cc4567c5b1908400 0000000000000000
ffffb3294a0ebc00: ffffb3294a0ebc88 0000000000000000
ffffb3294a0ebc10: 0000000000000004 0000000000000000
ffffb3294a0ebc20: [mm_struct] exc_page_fault+732
#4 [ffffb3294a0ebc28] exc_page_fault+0x2dc at ffffffff9a22a69c
/usr/src/linux-5.10.11-gentoo/arch/x86/mm/fault.c: 1320
ffffb3294a0ebc30: 0000000000000000 0000000000000000
ffffb3294a0ebc40: 0000000000000000 00042cc000000000
ffffb3294a0ebc50: 0000000000000000 0000000000000000
ffffb3294a0ebc60: 0000000000000000 0000000000000000
ffffb3294a0ebc70: 0000000000000000 0000000000000000
ffffb3294a0ebc80: asm_exc_page_fault+27
#5 [ffffb3294a0ebc80] asm_exc_page_fault+0x1b at ffffffff9a400acb
/usr/src/linux-5.10.11-gentoo/arch/x86/include/asm/idtentry.h: 583
ffffb3294a0ebc88: ffffb3294a0ebde0 0000000000000001
ffffb3294a0ebc98: ffffb3294bea1000 rtR0TermNative+208
#6 [ffffb3294a0ebca0] rtR0TermNative+0xd0 at ffffffffc0240080 [vboxdrv]
ffffb3294a0ebca8: ffffb3294a0ebdf8 0000000000000000
ffffb3294a0ebcb8: 000000000007b438 __this_module+992
ffffb3294a0ebcc8: 0000000000000000 0000000000000001
ffffb3294a0ebcd8: __apply_to_page_range+737
#7 [ffffb3294a0ebcd8] __apply_to_page_range+0x2e1 at ffffffff992838a1
/usr/src/linux-5.10.11-gentoo/include/linux/spinlock.h: 354
ffffb3294a0ebce0: 0000000000000000 0000000000000000
ffffb3294a0ebcf0: 0000000000000000 0000000000000000
ffffb3294a0ebd00: ffffffffffffffff do_raw_spin_lock+4
ffffb3294a0ebd10: 0000000000000010 0000000000010296
ffffb3294a0ebd20: ffffb3294a0ebd30 0000000000000018
ffffb3294a0ebd30: ffffb3294beed000 __apply_to_page_range+737
ffffb3294a0ebd40: ffff9958401012f8 ffffb3294beed000
ffffb3294a0ebd50: [mm_struct] ffff995858f00508
ffffb3294a0ebd60: 0000000000000000 ffffb3294beecfff
ffffb3294a0ebd70: ffffb3294beed000 ffff995840000528
ffffb3294a0ebd80: ffffb3294beed000 ffff995a17102b30
ffffb3294a0ebd90: ffffb3294beecfff rtR0TermNative+208
#8 [ffffb3294a0ebd98] rtR0TermNative+0xd0 at ffffffffc0240080 [vboxdrv]
ffffb3294a0ebda0: ffffb3294beecfff [kmalloc-8k]
ffffb3294a0ebdb0: ffffb3294a0ebdf8 0000000000000000
ffffb3294a0ebdc0: 0000000000000000 ffffb3294c609010
ffffb3294a0ebdd0: [kmalloc-192] rtR0MemObjNativeProtect+116
#9 [ffffb3294a0ebdd8] rtR0MemObjNativeProtect+0x74 at ffffffffc0241774 [vboxdrv]
ffffb3294a0ebde0: [kmalloc-8k] 8000000000000161
ffffb3294a0ebdf0: cc4567c5b1908400 ffffb3294a0ebe10
ffffb3294a0ebe00: VBoxHost_RTR0MemObjProtect+129
#10 [ffffb3294a0ebe00] VBoxHost_RTR0MemObjProtect+0x81 at ffffffffc023ecd1 [vboxdrv]
ffffb3294a0ebe08: __this_module+992 ffffb3294a0ebea8
ffffb3294a0ebe18: supdrvIOCtl+12901
#11 [ffffb3294a0ebe18] supdrvIOCtl+0x3265 at ffffffffc02339a5 [vboxdrv]
ffffb3294a0ebe20: 0000000000000004 [kmalloc-2k]
ffffb3294a0ebe30: __this_module+992 00000004001fc060
ffffb3294a0ebe40: 00000000000004ff 0000000000205900
ffffb3294a0ebe50: __this_module+992 0000000000000000
ffffb3294a0ebe60: 0000000000000000 0000000000000000
ffffb3294a0ebe70: ffffb3294a0ebea8 cc4567c5b1908400
ffffb3294a0ebe80: 0000000000205978 0000000000005684
ffffb3294a0ebe90: 0000754d24911010 [kmalloc-2k]
ffffb3294a0ebea0: ffffb3294c609010 ffffb3294a0ebf08
ffffb3294a0ebeb0: SUPR0Printf+559
#12 [ffffb3294a0ebeb0] SUPR0Printf+0x22f at ffffffffc022b67f [vboxdrv]
RIP: 0000754d82eb9e37 RSP: 0000754d3c1f2b58 RFLAGS: 00000246
RAX: ffffffffffffffda RBX: 0000754d24911010 RCX: 0000754d82eb9e37
RDX: 0000754d24911010 RSI: 0000000000005684 RDI: 000000000000000b
RBP: 0000754d3c1f2b70 R8: 0000000000000000 R9: 00000000fffffffc
R10: 0000000000000000 R11: 0000000000000246 R12: 0000754d6ef3afcf
R13: 0000000000000000 R14: 0000754d3c1f2e20 R15: 0000000000000004
ORIG_RAX: 0000000000000010 CS: 0033 SS: 002b
/
crash> mod
MODULE NAME BASE SIZE OBJECT FILE
ffffffffc0225100 pcspkr ffffffffc0223000 16384 (not loaded) [CONFIG_KALLSYMS]
ffffffffc026f380 vboxdrv ffffffffc022b000 438272 (not loaded) [CONFIG_KALLSYMS]
ffffffffc02b0040 vboxnetflt ffffffffc02ab000 28672 (not loaded) [CONFIG_KALLSYMS]
ffffffffc02b9180 vboxnetadp ffffffffc02b7000 28672 (not loaded) [CONFIG_KALLSYMS]
/
crash> add-symbol-file /usr/lib/debug/lib/modules/5.10.11-gentoo-x86_64/misc/vboxdrv.ko.debug 0xffffffffc022b000
add symbol table from file "/usr/lib/debug/lib/modules/5.10.11-gentoo-x86_64/misc/vboxdrv.ko.debug" at
.text_addr = 0xffffffffc022b000
Reading symbols from /usr/lib/debug/lib/modules/5.10.11-gentoo-x86_64/misc/vboxdrv.ko.debug...done.
/
crash> info line *rtR0TermNative+0xd0 Line 540 of "/usr/src/debug/app-emulation/virtualbox-modules-6.1.18/vboxdrv/r0drv/linux/memobj-r0drv-linux.c" starts at address 0xffffffffc0240080 <rtR0MemObjLinuxApplyPageRange> and ends at 0xffffffffc0240084 <rtR0MemObjLinuxApplyPageRange+4>. crash> info line *rtR0MemObjLinuxApplyPageRange+4 Line 542 of "/usr/src/debug/app-emulation/virtualbox-modules-6.1.18/vboxdrv/r0drv/linux/memobj-r0drv-linux.c" starts at address 0xffffffffc0240084 <rtR0MemObjLinuxApplyPageRange+4> and ends at 0xffffffffc0240087 <rtR0MemObjLinuxApplyPageRange+7>.
/**
* Callback called in apply_to_page_range().
*
* @returns Linux status code.
* @param pPte Pointer to the page table entry for the given address.
* @param uAddr The address to apply the new protection to.
* @param pvUser The opaque user data.
*/
static DECLCALLBACK(int) rtR0MemObjLinuxApplyPageRange(pte_t *pPte, unsigned long uAddr, void *pvUser)
{ //this is line 540
PCLNXAPPLYPGRANGE pArgs = (PCLNXAPPLYPGRANGE)pvUser;
PRTR0MEMOBJLNX pMemLnx = pArgs->pMemLnx; //this is line 542
size_t idxPg = (uAddr - (unsigned long)pMemLnx->Core.pv) >> PAGE_SHIFT;
set_pte(pPte, mk_pte(pMemLnx->apPages[idxPg], pArgs->fPg));
return 0;
}
#endif
/
crash> info line *rtR0TermNative+208 Line 540 of "/usr/src/debug/app-emulation/virtualbox-modules-6.1.18/vboxdrv/r0drv/linux/memobj-r0drv-linux.c" starts at address 0xffffffffc0240080 <rtR0MemObjLinuxApplyPageRange> and ends at 0xffffffffc0240084 <rtR0MemObjLinuxApplyPageRange+4>. (ok this doesn't seem to be useful?)
/
crash> info line *rtR0TermNative
Line 117 of "/usr/src/debug/app-emulation/virtualbox-modules-6.1.18/vboxdrv/r0drv/linux/initterm-r0drv-linux.c" starts at address 0xffffffffc023ffb0 <rtR0TermNative> and ends at 0xffffffffc023ffb4 <rtR0TermNative+4>.
DECLHIDDEN(void) rtR0TermNative(void)
{ //this is line 117
IPRT_LINUX_SAVE_EFL_AC();
rtR0LnxWorkqueueFlush();
#if RTLNX_VER_MIN(2,5,41)
destroy_workqueue(g_prtR0LnxWorkQueue);
g_prtR0LnxWorkQueue = NULL;
#endif
rtR0MemExecCleanup();
IPRT_LINUX_RESTORE_EFL_AC();
}
/
crash> info line *__apply_to_page_range+0x2e1
No line number information available for address 0xffffffff992838a1
(don't know how to make this work, tho it does work in the output of 'bt -l')
crash> sym 0xffffffff992838a1
ffffffff992838a1 (t) __apply_to_page_range+737 /usr/src/linux-5.10.11-gentoo/include/linux/spinlock.h: 354
crash> info line *__apply_to_page_range+737
No line number information available for address 0xffffffff992838a1
//ok so 0x2e1 is 737 in decimal
crash> sym 0xffffffff992838a1
ffffffff992838a1 (t) __apply_to_page_range+737 /usr/src/linux-5.10.11-gentoo/include/linux/spinlock.h: 354
#7 [ffffb3294a0ebcd8] __apply_to_page_range+0x2e1 at ffffffff992838a1
/usr/src/linux-5.10.11-gentoo/include/linux/spinlock.h: 354
static __always_inline void spin_lock(spinlock_t *lock)
{
raw_spin_lock(&lock->rlock); // this is line 354
}
/
Another crash, now with the following .config diff (compared to /usr/src/.config.prev18_nondebug):
-CONFIG_DEBUG_LOCK_ALLOC=y -CONFIG_LOCKDEP=y -# CONFIG_DEBUG_LOCKDEP is not set +# CONFIG_DEBUG_LOCK_ALLOC is not set
KERNEL: /var/crash/vmlinux-5.10.11-gentoo-x86_64-2021-01-29-19_54_43
DUMPFILE: /var/crash/crashdump-2021-01-29-20_22_50 [PARTIAL DUMP]
'log' partial output:
... [ 14.857357] vboxdrv: loading out-of-tree module taints kernel. [ 14.857929] vboxdrv: Found 12 processor cores [ 14.873771] vboxdrv: TSC mode is Invariant, tentative frequency 3695998457 Hz [ 14.873771] vboxdrv: Successfully loaded version 6.1.18 (interface 0x00300000) ... [ 56.083785] snd_hda_intel 0000:00:1f.3: power state changed by ACPI to D3hot [ 61.037910] SUPR0GipMap: fGetGipCpu=0x1b [ 62.739845] BUGGY: kernel NULL pointer dereference, address: 0000000000000004 [ 62.739856] #PF: supervisor read access in kernel mode [ 62.739862] #PF: error_code(0x0000) - not-present page [ 62.739866] PGD 0 P4D 0 [ 62.739876] Oops: 0000 [#1] SMP NOPTI [ 62.739883] CPU: 9 PID: 14846 Comm: EMT-0 Kdump: loaded Tainted: G U O 5.10.11-gentoo-x86_64 #1 [ 62.739887] Hardware name: System manufacturer System Product Name/PRIME Z370-A, BIOS 2401 07/12/2019 [ 62.739897] RIP: 0010:do_raw_spin_lock+0x4/0x90 [ 62.739904] Code: 48 8d 88 90 06 00 00 48 c7 c7 f0 c7 aa 94 e8 f6 be e4 00 e9 33 b4 e6 00 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 53 48 89 fb <8b> 47 04 3d ad 4e ad de 75 46 48 8b 53 10 65 48 8b 04 25 40 6d 01 [ 62.739909] RSP: 0018:ffffb532498a7d30 EFLAGS: 00010286 [ 62.739915] RAX: ffff997800000588 RBX: 0000000000000000 RCX: 0000000000000000 [ 62.739920] RDX: 0000000101207067 RSI: 00000000040481c0 RDI: 0000000000000000 [ 62.739923] RBP: ffffb532498a7df8 R08: ffff997901207588 R09: 0000000000000001 [ 62.739927] R10: ffffffffc0273560 R11: 0000000000061918 R12: ffffffffc0244000 [ 62.739931] R13: ffffb532498b1000 R14: 0000000000000001 R15: ffffb532498a7de0 [ 62.739937] FS: 0000751ec8366640(0000) GS:ffff99880a480000(0000) knlGS:0000000000000000 [ 62.739941] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 62.739945] CR2: 0000000000000004 CR3: 000000032b99c004 CR4: 00000000001706e0 [ 62.739949] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 62.739954] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 62.739957] Call Trace: [ 62.739967] ? __apply_to_page_range+0x2e1/0x6a0 [ 62.739999] ? rtR0TermNative+0xd0/0x220 [vboxdrv] [ 62.740026] ? rtR0MemObjNativeProtect+0x74/0xa0 [vboxdrv] [ 62.740051] ? VBoxHost_RTR0MemObjProtect+0x81/0xc0 [vboxdrv] [ 62.740074] ? supdrvIOCtl+0x3265/0x3800 [vboxdrv] [ 62.740094] ? SUPR0Printf+0x22f/0x330 [vboxdrv] [ 62.740101] ? __x64_sys_ioctl+0x7e/0xb0 [ 62.740109] ? do_syscall_64+0x33/0x40 [ 62.740117] ? entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 62.740121] Modules linked in: vboxnetadp(O) vboxnetflt(O) vboxdrv(O) pcspkr [ 62.740136] CR2: 0000000000000004
crash> bt -l
PID: 14846 TASK: ffff99792fa78000 CPU: 9 COMMAND: "EMT-0"
#0 [ffffb532498a7a68] machine_kexec at ffffffff93042956
/usr/src/linux-5.10.11-gentoo/arch/x86/include/asm/mem_encrypt.h: 77
#1 [ffffb532498a7ac8] __crash_kexec at ffffffff9314bee1
/usr/src/linux-5.10.11-gentoo/kernel/kexec_core.c: 963
#2 [ffffb532498a7b88] crash_kexec at ffffffff9314cdf4
/usr/src/linux-5.10.11-gentoo/arch/x86/include/asm/atomic.h: 41
#3 [ffffb532498a7b98] oops_end at ffffffff9301cd84
/usr/src/linux-5.10.11-gentoo/arch/x86/kernel/dumpstack.c: 359
#4 [ffffb532498a7c28] exc_page_fault at ffffffff94002016
/usr/src/linux-5.10.11-gentoo/arch/x86/mm/fault.c: 1346
#5 [ffffb532498a7c80] asm_exc_page_fault at ffffffff94200acb
/usr/src/linux-5.10.11-gentoo/arch/x86/include/asm/idtentry.h: 583
#6 [ffffb532498a7ca0] rtR0TermNative at ffffffffc0244000 [vboxdrv]
#7 [ffffb532498a7d08] do_raw_spin_lock at ffffffff9310dc54
/usr/src/linux-5.10.11-gentoo/kernel/locking/spinlock_debug.c: 112
#8 [ffffb532498a7d38] __apply_to_page_range at ffffffff931feb71
/usr/src/linux-5.10.11-gentoo/include/linux/spinlock.h: 354
#9 [ffffb532498a7dd8] rtR0MemObjNativeProtect at ffffffffc02456f4 [vboxdrv]
#10 [ffffb532498a7e00] VBoxHost_RTR0MemObjProtect at ffffffffc0242c81 [vboxdrv]
#11 [ffffb532498a7e18] supdrvIOCtl at ffffffffc0237955 [vboxdrv]
#12 [ffffb532498a7eb0] SUPR0Printf at ffffffffc022f62f [vboxdrv]
RIP: 0000751f0b85ae37 RSP: 0000751ec8364b58 RFLAGS: 00000246
RAX: ffffffffffffffda RBX: 0000751ead314010 RCX: 0000751f0b85ae37
RDX: 0000751ead314010 RSI: 0000000000005684 RDI: 000000000000000b
RBP: 0000751ec8364b70 R8: 0000000000000000 R9: 00000000fffffffc
R10: 0000000000000000 R11: 0000000000000246 R12: 0000751efc3aefcf
R13: 0000000000000000 R14: 0000751ec8364e20 R15: 0000000000000004
ORIG_RAX: 0000000000000010 CS: 0033 SS: 002b
/
/*
* We are now relying on the NMI watchdog to detect lockup instead of doing
* the detection here with an unfair lock which can cause problem of its own.
*/
void do_raw_spin_lock(raw_spinlock_t *lock)
{
debug_spin_lock_before(lock); // this is line 112
arch_spin_lock(&lock->raw_lock);
mmiowb_spin_lock();
debug_spin_lock_after(lock);
}
/
PID: 14846 TASK: ffff99792fa78000 CPU: 9 COMMAND: "EMT-0"
#0 [ffffb532498a7a68] machine_kexec+0x186 at ffffffff93042956
/usr/src/linux-5.10.11-gentoo/arch/x86/include/asm/mem_encrypt.h: 77
ffffb532498a7a70: 00000000498a7d30 0000ffff99780000
ffffb532498a7a80: ffff997800000000 000000003d001000
ffffb532498a7a90: ffff99783d001000 000000003d000000
ffffb532498a7aa0: 0000000000000000 c2c563b4d1c59e00
ffffb532498a7ab0: ffffb532498a7c88 ffffb532498a7c88
ffffb532498a7ac0: 0000000000000009 __crash_kexec+225
#1 [ffffb532498a7ac8] __crash_kexec+0xe1 at ffffffff9314bee1
/usr/src/linux-5.10.11-gentoo/kernel/kexec_core.c: 963
ffffb532498a7ad0: ffffb532498a7de0 0000000000000001
ffffb532498a7ae0: ffffb532498b1000 rtR0TermNative+208
ffffb532498a7af0: ffffb532498a7df8 0000000000000000
ffffb532498a7b00: 0000000000061918 __this_module+864
ffffb532498a7b10: 0000000000000001 ffff997901207588
ffffb532498a7b20: ffff997800000588 0000000000000000
ffffb532498a7b30: 0000000101207067 00000000040481c0
ffffb532498a7b40: 0000000000000000 ffffffffffffffff
ffffb532498a7b50: do_raw_spin_lock+4 0000000000000010
ffffb532498a7b60: 0000000000010286 ffffb532498a7d30
ffffb532498a7b70: 0000000000000018 c2c563b4d1c59e00
ffffb532498a7b80: 0000000000000246 crash_kexec+52
#2 [ffffb532498a7b88] crash_kexec+0x34 at ffffffff9314cdf4
/usr/src/linux-5.10.11-gentoo/arch/x86/include/asm/atomic.h: 41
ffffb532498a7b90: ffffb532498a7c88 oops_end+132
#3 [ffffb532498a7b98] oops_end+0x84 at ffffffff9301cd84
/usr/src/linux-5.10.11-gentoo/arch/x86/kernel/dumpstack.c: 359
ffffb532498a7ba0: [task_struct] ffffb532498a7c88
ffffb532498a7bb0: 0000000000000004 no_context+572
ffffb532498a7bc0: 0000000000000000 0000000000000000
ffffb532498a7bd0: 0000000000000000 0000000000000000
ffffb532498a7be0: 0000000000000000 0000000000000000
ffffb532498a7bf0: c2c563b4d1c59e00 0000000000000214
ffffb532498a7c00: 0000000000000000 ffffb532498a7c88
ffffb532498a7c10: 0000000000000004 0000000000000000
ffffb532498a7c20: [mm_struct] exc_page_fault+790
#4 [ffffb532498a7c28] exc_page_fault+0x316 at ffffffff94002016
/usr/src/linux-5.10.11-gentoo/arch/x86/mm/fault.c: 1346
ffffb532498a7c30: [mm_struct] 0000000000000000
ffffb532498a7c40: 0000000000000000 00000000000001d0
ffffb532498a7c50: 0000000000000000 0000000000000000
ffffb532498a7c60: 0000000000000000 0000000000000000
ffffb532498a7c70: 0000000000000000 0000000000000000
ffffb532498a7c80: asm_exc_page_fault+27
#5 [ffffb532498a7c80] asm_exc_page_fault+0x1b at ffffffff94200acb
/usr/src/linux-5.10.11-gentoo/arch/x86/include/asm/idtentry.h: 583
ffffb532498a7c88: ffffb532498a7de0 0000000000000001
ffffb532498a7c98: ffffb532498b1000 rtR0TermNative+208
#6 [ffffb532498a7ca0] rtR0TermNative+0xd0 at ffffffffc0244000 [vboxdrv]
ffffb532498a7ca8: ffffb532498a7df8 0000000000000000
ffffb532498a7cb8: 0000000000061918 __this_module+864
ffffb532498a7cc8: 0000000000000001 ffff997901207588
ffffb532498a7cd8: ffff997800000588 0000000000000000
ffffb532498a7ce8: 0000000101207067 00000000040481c0
ffffb532498a7cf8: 0000000000000000 ffffffffffffffff
ffffb532498a7d08: do_raw_spin_lock+4
#7 [ffffb532498a7d08] do_raw_spin_lock+0x4 at ffffffff9310dc54
/usr/src/linux-5.10.11-gentoo/kernel/locking/spinlock_debug.c: 112
ffffb532498a7d10: 0000000000000010 0000000000010286
ffffb532498a7d20: ffffb532498a7d30 0000000000000018
ffffb532498a7d30: ffffb532498fd000 __apply_to_page_range+737
#8 [ffffb532498a7d38] __apply_to_page_range+0x2e1 at ffffffff931feb71
/usr/src/linux-5.10.11-gentoo/include/linux/spinlock.h: 354
ffffb532498a7d40: ffff997900067260 ffffb532498fd000
ffffb532498a7d50: [mm_struct] ffff997901207588
ffffb532498a7d60: 0000000000000000 ffffb532498fcfff
ffffb532498a7d70: ffffb532498fd000 ffff997900000648
ffffb532498a7d80: ffffb532498fd000 ffff997b2b99cb50
ffffb532498a7d90: ffffb532498fcfff rtR0TermNative+208
ffffb532498a7da0: ffffb532498fcfff [kmalloc-8k]
ffffb532498a7db0: ffffb532498a7df8 0000000000000000
ffffb532498a7dc0: 0000000000000000 ffffb5324a109010
ffffb532498a7dd0: [kmalloc-192] rtR0MemObjNativeProtect+116
#9 [ffffb532498a7dd8] rtR0MemObjNativeProtect+0x74 at ffffffffc02456f4 [vboxdrv]
ffffb532498a7de0: [kmalloc-8k] 8000000000000161
ffffb532498a7df0: c2c563b4d1c59e00 ffffb532498a7e10
ffffb532498a7e00: VBoxHost_RTR0MemObjProtect+129
#10 [ffffb532498a7e00] VBoxHost_RTR0MemObjProtect+0x81 at ffffffffc0242c81 [vboxdrv]
ffffb532498a7e08: __this_module+864 ffffb532498a7ea8
ffffb532498a7e18: supdrvIOCtl+12901
#11 [ffffb532498a7e18] supdrvIOCtl+0x3265 at ffffffffc0237955 [vboxdrv]
ffffb532498a7e20: 0000000000000004 [kmalloc-2k]
ffffb532498a7e30: __this_module+864 00000004001fc060
ffffb532498a7e40: 00000000000004ff 0000000000205900
ffffb532498a7e50: __this_module+864 0000000000000000
ffffb532498a7e60: 0000000000000000 0000000000000000
ffffb532498a7e70: 0000000000205978 c2c563b4d1c59e00
ffffb532498a7e80: 0000000000205978 0000000000005684
ffffb532498a7e90: 0000751ead314010 [kmalloc-2k]
ffffb532498a7ea0: ffffb5324a109010 ffffb532498a7f08
ffffb532498a7eb0: SUPR0Printf+559
#12 [ffffb532498a7eb0] SUPR0Printf+0x22f at ffffffffc022f62f [vboxdrv]
RIP: 0000751f0b85ae37 RSP: 0000751ec8364b58 RFLAGS: 00000246
RAX: ffffffffffffffda RBX: 0000751ead314010 RCX: 0000751f0b85ae37
RDX: 0000751ead314010 RSI: 0000000000005684 RDI: 000000000000000b
RBP: 0000751ec8364b70 R8: 0000000000000000 R9: 00000000fffffffc
R10: 0000000000000000 R11: 0000000000000246 R12: 0000751efc3aefcf
R13: 0000000000000000 R14: 0000751ec8364e20 R15: 0000000000000004
ORIG_RAX: 0000000000000010 CS: 0033 SS: 002b
/
#9 [ffffb532498a7dd8] rtR0MemObjNativeProtect+0x74 at ffffffffc02456f4 [vboxdrv]
crash> info line *0xffffffffc02456f4
Line 1919 of "/usr/src/debug/app-emulation/virtualbox-modules-6.1.18/vboxdrv/r0drv/linux/memobj-r0drv-linux.c" starts at address 0xffffffffc02456f4 <rtR0MemObjNativeProtect+116> and ends at 0xffffffffc0245720 <rtR0MemObjNativeGetPagePhysAddr>.
# elif defined(IPRT_USE_APPLY_TO_PAGE_RANGE_FOR_EXEC)
PRTR0MEMOBJLNX pMemLnx = (PRTR0MEMOBJLNX)pMem;
if ( pMemLnx->fExecutable
&& pMemLnx->fMappedToRing0)
{
LNXAPPLYPGRANGE Args;
Args.pMemLnx = pMemLnx;
Args.fPg = rtR0MemObjLinuxConvertProt(fProt, true /*fKernel*/);
int rcLnx = apply_to_page_range(current->active_mm, (unsigned long)pMemLnx->Core.pv + offSub, cbSub,
rtR0MemObjLinuxApplyPageRange, (void *)&Args);
if (rcLnx) //this is line 1919 !!!!!!!!!!!!!!!!!!!!!!!!!!! which seems off by one (a normal thing)
return VERR_NOT_SUPPORTED;
return VINF_SUCCESS;
}
# endif
no crash now with the following .config diff (compared to previous):
-CONFIG_UNINLINE_SPIN_UNLOCK=y +CONFIG_INLINE_SPIN_UNLOCK_IRQ=y +CONFIG_INLINE_READ_UNLOCK=y +CONFIG_INLINE_READ_UNLOCK_IRQ=y +CONFIG_INLINE_WRITE_UNLOCK=y +CONFIG_INLINE_WRITE_UNLOCK_IRQ=y -CONFIG_DEBUG_SPINLOCK=y +# CONFIG_DEBUG_SPINLOCK is not set
so to avoid crash, you need(at least):
CONFIG_DEBUG_SPINLOCK=n (which forces CONFIG_DEBUG_LOCK_ALLOC=n)
but doesn't matter if =y on the following:
CONFIG_DEBUG_RT_MUTEXES=y CONFIG_DEBUG_MUTEXES=y CONFIG_DEBUG_LOCKING_API_SELFTESTS=y
nvmTODO: try with:
+# CONFIG_DEBUG_SPINLOCK is not set
but with
CONFIG_DEBUG_LOCK_ALLOC=y
Can't, because if CONFIG_DEBUG_LOCK_ALLOC=y then it's forced that CONFIG_DEBUG_SPINLOCK=y !
Note:
See TracTickets
for help on using tickets.
Also seeing a kernel panic with Fedora 32 and 5.10.x kernel. No issues with 5.9.x kernels.
Pretty much a show stopper.