VirtualBox

Ticket #10097 (new defect)

Opened 2 years ago

Last modified 16 months ago

Win7 VM's crash when a debugger is connected via virtualized COM port pipe (ICH9 only)

Reported by: mfrobben Owned by:
Priority: major Component: other
Version: VirtualBox 4.1.8 Keywords: HAL
Cc: Guest type: Windows
Host type: Windows

Description (last modified by frank) (diff)

I've got a 64 bit win7 box running latest version of VirtualBox. I have a Win7 x64 VM that I'd like to use for development of kernel drivers. The VM has a virtualized COM port that creates a host pipe called VMCOM. Whenever I connect a kernel debugger (in this case WINDBG) to the VM over that pipe, after breaking and continuing execution a few times I see the VM crash in HALHandleNMI after a secondaryclockinterrupt occurs. The bugcheck details given via !analyze are also given.

Opened \.\pipe\vmcom
Waiting to reconnect...
Connected to Windows 7 7600 x64 target at (Wed Dec 28 10:26:18.444 2011 (UTC - 5:00)), ptr64 TRUE
Kernel Debugger connection established.
Symbol search path is: *** Invalid ***
****************************************************************************
* Symbol loading may be unreliable without a symbol search path.           *
* Use .symfix to have the debugger choose a symbol path.                   *
* After setting your symbol path, use .reload to refresh symbol locations. *
****************************************************************************
Executable search path is: 
*********************************************************************
* Symbols can not be loaded because symbol path is not initialized. *
*                                                                   *
* The Symbol Path can be set by:                                    *
*   using the _NT_SYMBOL_PATH environment variable.                 *
*   using the -y <symbol_path> argument when starting the debugger. *
*   using .sympath and .sympath+                                    *
*********************************************************************
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for ntkrnlmp.exe - 
Windows 7 Kernel Version 7600 MP (4 procs) Free x64
Product: WinNt, suite: TerminalServer SingleUserTS
Built by: 7600.16385.amd64fre.win7_rtm.090713-1255
Machine Name:
Kernel base = 0xfffff800`0284a000 PsLoadedModuleList = 0xfffff800`02a87e50
Debug session time: Wed Dec 28 10:26:12.029 2011 (UTC - 5:00)
System Uptime: 0 days 5:12:49.358
Break instruction exception - code 80000003 (first chance)
*******************************************************************************
*                                                                             *
*   You are seeing this message because you pressed either                    *
*       CTRL+C (if you run console kernel debugger) or,                       *
*       CTRL+BREAK (if you run GUI kernel debugger),                          *
*   on your debugger machine's keyboard.                                      *
*                                                                             *
*                   THIS IS NOT A BUG OR A SYSTEM CRASH                       *
*                                                                             *
* If you did not intend to break into the debugger, press the "g" key, then   *
* press the "Enter" key now.  This message might immediately reappear.  If it *
* does, press "g" and "Enter" again.                                          *
*                                                                             *
*******************************************************************************
nt!DbgBreakPointWithStatus:
fffff800`028b3f60 cc              int     3
1: kd> .symfix
1: kd> .reload
Connected to Windows 7 7600 x64 target at (Wed Dec 28 10:33:42.784 2011 (UTC - 5:00)), ptr64 TRUE
Loading Kernel Symbols
. -- User interrupt
1: kd> g
Break instruction exception - code 80000003 (first chance)
*******************************************************************************
*                                                                             *
*   You are seeing this message because you pressed either                    *
*       CTRL+C (if you run console kernel debugger) or,                       *
*       CTRL+BREAK (if you run GUI kernel debugger),                          *
*   on your debugger machine's keyboard.                                      *
*                                                                             *
*                   THIS IS NOT A BUG OR A SYSTEM CRASH                       *
*                                                                             *
* If you did not intend to break into the debugger, press the "g" key, then   *
* press the "Enter" key now.  This message might immediately reappear.  If it *
* does, press "g" and "Enter" again.                                          *
*                                                                             *
*******************************************************************************
nt!RtlpBreakWithStatusInstruction:
fffff800`028b3f60 cc              int     3
3: kd> !stack
No export stack found
3: kd> !ps
No export ps found
3: kd> !eip
No export eip found
3: kd> !thread
THREAD fffff88002f1dfc0  Cid 0000.0000  Teb: 0000000000000000 Win32Thread: 0000000000000000 RUNNING on processor 3
Not impersonating
DeviceMap                 fffff8a0000060c0
Owning Process            fffff80002a43140       Image:         Idle
Attached Process          fffffa8000c9e040       Image:         System
Wait Start TickCount      0              Ticks: 1203174 (0:05:12:49.634)
Context Switch Count      1255366        IdealProcessor: 3             
UserTime                  00:00:00.000
KernelTime                05:11:32.149
Win32 Start Address nt!KiIdleLoop (0xfffff800028c46a0)
Stack Init fffff88002f3bdb0 Current fffff88002f3bd40
Base fffff88002f3c000 Limit fffff88002f36000 Call 0
Priority 16 BasePriority 0 UnusualBoost 0 ForegroundBoost 0 IoPriority 0 PagePriority 0
Child-SP          RetAddr           : Args to Child                                                           : Call Site
fffff880`02f3bac8 fffff800`02882d73 : 00000000`00000000 fffff880`02f13180 00000000`00000000 00000000`00026161 : nt!RtlpBreakWithStatusInstruction
fffff880`02f3bad0 fffff800`028c8ba1 : 00000000`00000000 fffff880`02f3bb80 fffff880`02f13180 00000000`00000001 : nt! ?? ::FNODOBFM::`string'+0x5dd4
fffff880`02f3bb00 fffff880`0450f9c2 : fffff800`028c9a3a 00000000`ffffffed fffffa80`0203b2b8 fffff880`02f1dfc0 : nt!KiSecondaryClockInterrupt+0x131 (TrapFrame @ fffff880`02f3bb00)
fffff880`02f3bc98 fffff800`028c9a3a : 00000000`ffffffed fffffa80`0203b2b8 fffff880`02f1dfc0 00000000`00000001 : 0xfffff880`0450f9c2
fffff880`02f3bca0 fffff800`028c46cc : fffff880`02f13180 fffff880`00000000 00000000`00000000 fffff800`02950cf0 : nt!PoIdle+0x53a
fffff880`02f3bd80 00000000`00000000 : fffff880`02f3c000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiIdleLoop+0x2c

3: kd> g
Break instruction exception - code 80000003 (first chance)
*******************************************************************************
*                                                                             *
*   You are seeing this message because you pressed either                    *
*       CTRL+C (if you run console kernel debugger) or,                       *
*       CTRL+BREAK (if you run GUI kernel debugger),                          *
*   on your debugger machine's keyboard.                                      *
*                                                                             *
*                   THIS IS NOT A BUG OR A SYSTEM CRASH                       *
*                                                                             *
* If you did not intend to break into the debugger, press the "g" key, then   *
* press the "Enter" key now.  This message might immediately reappear.  If it *
* does, press "g" and "Enter" again.                                          *
*                                                                             *
*******************************************************************************
nt!RtlpBreakWithStatusInstruction:
fffff800`028b3f60 cc              int     3
2: kd> g
Break instruction exception - code 80000003 (first chance)
nt!RtlpBreakWithStatusInstruction:
fffff800`028b3f60 cc              int     3
3: kd> !thread
THREAD fffff88002f1dfc0  Cid 0000.0000  Teb: 0000000000000000 Win32Thread: 0000000000000000 RUNNING on processor 3
Not impersonating
DeviceMap                 fffff8a0000060c0
Owning Process            fffff80002a43140       Image:         Idle
Attached Process          fffffa8000c9e040       Image:         System
Wait Start TickCount      0              Ticks: 1204506 (0:05:13:10.414)
Context Switch Count      1260552        IdealProcessor: 3             
UserTime                  00:00:00.000
KernelTime                05:11:52.335
Win32 Start Address nt!KiIdleLoop (0xfffff800028c46a0)
Stack Init fffff88002f3bdb0 Current fffff88002f3bd40
Base fffff88002f3c000 Limit fffff88002f36000 Call 0
Priority 16 BasePriority 0 UnusualBoost 0 ForegroundBoost 0 IoPriority 0 PagePriority 0
Child-SP          RetAddr           : Args to Child                                                           : Call Site
fffff880`02f1db58 fffff800`029b16d2 : fffff800`00000010 fffffa80`01be2a20 00000000`00000000 fffff800`029b17e2 : nt!RtlpBreakWithStatusInstruction
fffff880`02f1db60 fffff800`028138da : fffff800`00000005 00000028`00000025 00000000`0000027f fffff800`028292b0 : nt!KiBugCheckDebugBreak+0x12
fffff880`02f1dbc0 fffff800`029d0513 : 00000000`00000001 fffff800`028292b0 00000000`00000000 00000000`0000005c : hal!HalBugCheckSystem+0x1ba
fffff880`02f1dc00 fffff800`0280d6c1 : fffffa80`000006c0 fffff880`02f1de20 fffff880`02f1dcf0 fffff800`028292b0 : nt!WheaReportHwError+0x263
fffff880`02f1dc60 fffff800`02974311 : fffff880`02f1de30 00000000`00000001 00000000`00000001 fffffa80`0203b200 : hal!HalHandleNMI+0x149
fffff880`02f1dc90 fffff800`028b9202 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000003 : nt!KiProcessNMI+0x131
fffff880`02f1dcf0 fffff800`028b9063 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KxNmiInterrupt+0x82
fffff880`02f1de30 fffff800`028c8acf : 00000000`00000000 fffff880`02f3bb80 fffff880`02f13180 00000000`00000001 : nt!KiNmiInterrupt+0x163 (TrapFrame @ fffff880`02f1de30)
fffff880`02f3bb00 fffff880`0450f9c2 : fffff800`028c9a3a 00000000`ffffffed fffffa80`0203b2b8 fffff880`02f1dfc0 : nt!KiSecondaryClockInterrupt+0x5f (TrapFrame @ fffff880`02f3bb00)
fffff880`02f3bc98 fffff800`028c9a3a : 00000000`ffffffed fffffa80`0203b2b8 fffff880`02f1dfc0 00000000`00000001 : 0xfffff880`0450f9c2
fffff880`02f3bca0 fffff800`028c46cc : fffff880`02f13180 fffff880`00000000 00000000`00000000 fffff800`02950cf0 : nt!PoIdle+0x53a
fffff880`02f3bd80 00000000`00000000 : fffff880`02f3c000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiIdleLoop+0x2c


3: kd> !analyze -v
TRIAGER: Could not open triage file : C:\Program Files\Windows Kits\8.0\Debuggers\x64\triage\oca.ini, error 2
TRIAGER: Could not open triage file : C:\Program Files\Windows Kits\8.0\Debuggers\x64\winxp\triage.ini, error 2
TRIAGER: Could not open triage file : C:\Program Files\Windows Kits\8.0\Debuggers\x64\triage\user.ini, error 2
Connected to Windows 7 7600 x64 target at (Wed Dec 28 10:49:24.239 2011 (UTC - 5:00)), ptr64 TRUE
Loading Kernel Symbols
...............................................................
................................................................
.............
Loading User Symbols

Loading unloaded module list
................
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

Unknown bugcheck code (111)
Unknown bugcheck description
Arguments:
Arg1: 0000000000000000
Arg2: 0000000000000000
Arg3: 0000000000000000
Arg4: 0000000000000000

Debugging Details:
------------------

*** ERROR: Module load completed but symbols could not be loaded for intelppm.sys
TRIAGER: Could not open triage file : C:\Program Files\Windows Kits\8.0\Debuggers\x64\triage\modclass.ini, error 2

DEFAULT_BUCKET_ID:  WIN7_DRIVER_FAULT

BUGCHECK_STR:  0x111

PROCESS_NAME:  System

CURRENT_IRQL:  f

LAST_CONTROL_TRANSFER:  from fffff800029b16d2 to fffff800028b3f60

STACK_TEXT:  
fffff880`02f1db58 fffff800`029b16d2 : fffff800`00000010 fffffa80`01be2a20 00000000`00000000 fffff800`029b17e2 : nt!RtlpBreakWithStatusInstruction
fffff880`02f1db60 fffff800`028138da : fffff800`00000005 00000028`00000025 00000000`0000027f fffff800`028292b0 : nt!KiBugCheckDebugBreak+0x12
fffff880`02f1dbc0 fffff800`029d0513 : 00000000`00000001 fffff800`028292b0 00000000`00000000 00000000`0000005c : hal!HalBugCheckSystem+0x1ba
fffff880`02f1dc00 fffff800`0280d6c1 : fffffa80`000006c0 fffff880`02f1de20 fffff880`02f1dcf0 fffff800`028292b0 : nt!WheaReportHwError+0x263
fffff880`02f1dc60 fffff800`02974311 : fffff880`02f1de30 00000000`00000001 00000000`00000001 fffffa80`0203b200 : hal!HalHandleNMI+0x149
fffff880`02f1dc90 fffff800`028b9202 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000003 : nt!KiProcessNMI+0x131
fffff880`02f1dcf0 fffff800`028b9063 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KxNmiInterrupt+0x82
fffff880`02f1de30 fffff800`028c8acf : 00000000`00000000 fffff880`02f3bb80 fffff880`02f13180 00000000`00000001 : nt!KiNmiInterrupt+0x163
fffff880`02f3bb00 fffff880`0450f9c2 : fffff800`028c9a3a 00000000`ffffffed fffffa80`0203b2b8 fffff880`02f1dfc0 : nt!KiSecondaryClockInterrupt+0x5f
fffff880`02f3bc98 fffff800`028c9a3a : 00000000`ffffffed fffffa80`0203b2b8 fffff880`02f1dfc0 00000000`00000001 : intelppm+0x39c2
fffff880`02f3bca0 fffff800`028c46cc : fffff880`02f13180 fffff880`00000000 00000000`00000000 fffff800`02950cf0 : nt!PoIdle+0x53a
fffff880`02f3bd80 00000000`00000000 : fffff880`02f3c000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiIdleLoop+0x2c


STACK_COMMAND:  kb

FOLLOWUP_IP: 
intelppm+39c2
fffff880`0450f9c2 c3              ret

SYMBOL_STACK_INDEX:  9

SYMBOL_NAME:  intelppm+39c2

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: intelppm

IMAGE_NAME:  intelppm.sys

DEBUG_FLR_IMAGE_TIMESTAMP:  4a5bc0fd

FAILURE_BUCKET_ID:  X64_0x111_intelppm+39c2

BUCKET_ID:  X64_0x111_intelppm+39c2

Followup: MachineOwner
---------
}}}[[BR]]

Attachments

Win7_Basic-2011-12-28-11-09-54.log Download (55.0 KB) - added by mfrobben 2 years ago.
vbox log
output.txt Download (14.3 KB) - added by mfrobben 2 years ago.
pretty printed text of the debugger
bugcheck2.txt Download (2.7 KB) - added by mfrobben 2 years ago.
another bugcheck (pretty print)
dump1.zip Download (11.1 KB) - added by mhanor 16 months ago.

Change History

Changed 2 years ago by mfrobben

vbox log

Changed 2 years ago by mfrobben

pretty printed text of the debugger

comment:1 Changed 2 years ago by mfrobben

Another crash occurred with a different stack: *

  • *
  • Bugcheck Analysis *
  • *

*

Use !analyze -v to get detailed debugging information.

BugCheck 111, {0, 0, 0, 0}

TRIAGER: Could not open triage file : C:\Program Files\Windows Kits\8.0\Debuggers\x64\triage\modclass.ini, error 2 Probably caused by : ntkrnlmp.exe ( ntKiNmiInterruptEnd+15 )

Followup: MachineOwner


ntRtlpBreakWithStatusInstruction: fffff800`028b3f60 cc int 3 3: kd> !thread THREAD fffff88002f1dfc0 Cid 0000.0000 Teb: 0000000000000000 Win32Thread: 0000000000000000 RUNNING on processor 3 Not impersonating DeviceMap fffff8a0000060c0 Owning Process fffff80002a43140 Image: Idle Attached Process fffffa8000c9e040 Image: System Wait Start TickCount 0 Ticks: 917455 (0:03:58:32.389) Context Switch Count 1161932 IdealProcessor: 3 UserTime 00:00:00.000 KernelTime 03:57:26.900 Win32 Start Address ntKiIdleLoop (0xfffff800028c46a0) Stack Init fffff88002f3bdb0 Current fffff88002f3bd40 Base fffff88002f3c000 Limit fffff88002f36000 Call 0 Priority 16 BasePriority 0 UnusualBoost 0 ForegroundBoost 0 IoPriority 0 PagePriority 0 Child-SP RetAddr : Args to Child : Call Site fffff88002f1d578 fffff800029b16d2 : 0000000000000000 fffff88002f1dfc0 0000000000000065 fffff800028fa314 : ntRtlpBreakWithStatusInstruction fffff88002f1d580 fffff800029b24be : 0000000000000003 0000000000000000 fffff800028f6ee0 0000000000000111 : ntKiBugCheckDebugBreak+0x12 fffff88002f1d5e0 fffff800028bc004 : fffffa8001bde9a0 fffff80002960aaf fffffa800203b200 fffffa8001bde9a0 : ntKeBugCheck2+0x71e fffff88002f1dcb0 fffff800028bb469 : 0000000000000111 0000000000000000 0000000000000000 0000000000000000 : ntKeBugCheckEx+0x104 fffff88002f1dcf0 fffff800028b914f : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : ntKiBugCheckDispatch+0x69 fffff88002f1de30 fffff8000281a41c : 0000000000000000 0000000000000000 fffffa8002c59b70 0000000000000000 : ntKiNmiInterruptEnd+0x15 (TrapFrame @ fffff880`02f1de30) fffff88002f1da58 0000000000000000 : 0000000000000000 fffffa8002c59b70 0000000000000000 0000000000000002 : halXmOpcodeRegister+0x28

Version 0, edited 2 years ago by mfrobben (next)

Changed 2 years ago by mfrobben

another bugcheck (pretty print)

comment:2 Changed 2 years ago by mfrobben

I was able to resolve this by changing the number of CPU's from 4 to 1. This loads the single processor HAL and doesn't crash due to NMI issues. The bug looks like it's in the multiprocessor HAL or handling of NMI's in the presence of more than one cpu.

comment:3 Changed 2 years ago by frank

  • Description modified (diff)

You should also change the chipset emulation from ICH9 to PIIX3 since the former has known bugs and is marked as experimental.

comment:4 Changed 2 years ago by frank

  • Description modified (diff)

comment:5 Changed 2 years ago by mfrobben

Thanks Frank. Yes, changing to PIIX3 also resolved the issue.

~Matt

comment:6 Changed 22 months ago by frank

  • Summary changed from Win7 VM's crash when a debugger is connected via virtualized COM port pipe to Win7 VM's crash when a debugger is connected via virtualized COM port pipe (ICH9 only)

comment:7 Changed 20 months ago by frank

Could you check if you still can reproduce this bug with ICH9 and VBox 4.1.20?

comment:8 Changed 16 months ago by daveb

I also have come across this issue with version 4.2.4. Window 7 SP1 64 bit host and client.

Settings...

Motherboard:
Base Memory 1024Mb
Chipset: PIIX3
Enabled IO APIC - Enabled
Enabled EFI - Disabled
hardware clock in UTC time - Disabled
Enabled absolute pointing device - Enabled
Processor:
2 CPUs
Execution Cap - 100%
Enable PAE/NX - Disabled
Acceleration:
Enabled VT-x/AMD-V - Enabled
Enabled Nested Paging - Enabled
Serial Ports:
Port1:
Enabled Serial Port
Port Number: COM1 IRQ 4 i/o port 0x3f8
Port Mode: Host Pipe
Create Pipe - Enabled
Port/File Path: \\.\pipe\com_debug_1

Connect Windbg through COM port. Many seemingly random crashes booting up. Never been able to see login screen. Boots fine is Windbg if not connected. Windbg version 6.12.0002.633 64 bit.

If I change the number of processors to 1 then the system is stable.

Thanks
Dave

comment:9 Changed 16 months ago by mhanor

It seems I can reproduce a guest crash (Win7 x64 SP1 unpatched) while it just started booting, just after the external debugger (windbg) has connected to the guest (through the pipe). VirtualBox self-build from svn rev44059, host Win7 x64 SP1. I'm attaching the guest kernel minidump. ICH9 or PIIX3, it doesn't seem to matter.

Last edited 16 months ago by mhanor (previous) (diff)

Changed 16 months ago by mhanor

Note: See TracTickets for help on using tickets.

www.oracle.com
ContactPrivacy policyTerms of Use