Guru Meditation -2701 (VERR_VMM_RING0_ASSERTION) Windows 7 x64 Guest on Linux 64-bit host

[Walter Nicholls]

Host computer: M51S lapop (core2duo, NVidia adaptor) running Ubuntu 11.10 (oneiric) 64-bit; 4GB RAM. 3.0.0-15-generic #26-Ubuntu SMP. Virtualbox 4.1.10 Guest: Windows 7 SP1 64-bit, 2GB RAM allocated, 256MB video RAM. VirtualBox Guest Additions 4.1.10 (more on this later). Also similar problems with Guest Windows XP 32bit, 512MB RAM, not sure of which guest additions version but probably 4.1.10).

These problems started about a month ago when VB 4.1.8 was current. However this exact report (today) is as above.

This maybe same ultimate issue as #10331 but I'm reporting as separate ticket because his/her problem went away with removing 3D whereas mine didn't.

This exact report; If I read the log correctly, the Guest VM crashed with error (first recorded in log Guru Meditation -2701 (VERR_VMM_RING0_ASSERTION)) after 15 hours of operation - it was IDLE at the time, in fact I was not even home at the time! Several programs open, I attach screenshot as there is nothing incriminating or confidential in it .. you can see it was running in full-screen mode and a command window was in the foreground, with Visual Studio 2010 (and probably SQL Management Studio) running as well. Possibly a few other programs but generally lightly loaded. (attached VBox.log and VBox.png from the Windows 7 VM)

The history: I have been having problems with this VM locking up when closing the virtualbox window (normally get the savestate/send shutdown/poweroff dialog when I do this). No repaints. I've had windows in the guest lock up and become unkillable zombie processes. To the point where killall VirtualBox is the only way to get things closed. However as far as I remember the host OS is always ok. (I may have restored to reboot at some point for another reason to see if it did voodoo)

Also this VM did have troubles installing 3D video support this MAY be related. (See below).

NOTE: I have also had trouble at least once with another Windows XP VM locking up (but as I don't use that for regular work, tests only, I can't really give a decent history on that one. The Vbox.log has rotated out so it was more than 4 sessions ago) That does suggest a VirtualBox host-related problem NOT the VM itself!

Now - Guest Additions installer has been complaining "invalid DirectX installation detected" for the last few times, usually I ignore. However this time I uninstalled guest additions, uninstalled "Virtualbox graphics adaptor" through multiple iterations until I got back to standard VGA. Rebooted into safe mode, restored d3d8.dll and d3d9.dll and deleted every VBox*.* copy I can find ... in short just about everything suggested in the thread ​https://forums.virtualbox.org/viewtopic.php?f=2&t=46869 carefully!) And then reinstalled guest additions 4.1.10 with 3d support carefully turned off.

I also temporarily booted the host under 3.0.0-15 kernel but no joy there.

Only thing I have not yet tried is changing VM settings to less available RAM, as I had recently increased from 1GB to 2GB - loath to do this but it is something to try. As this last crash is about the tenth time this week, I guess I should try anything.

So - nothing above has fixed anything, may even have made it worse. Also as before, I suspect anything guest-related (3D support, guest additions, RAM size etc) is irrelevant but I tell anyway.

Hope all that long story helps. And PLEASE HELP. I have lost many hours of work here, almost ready to give up and purchase VMWare - seriously!

[Walter Nicholls]

Screenshot when crashed.

[Walter Nicholls]

Most recent crash log (as reported in ticket description)

[Walter Nicholls]

Crash 16/3/2012 at 21:36 comments refer

[Walter Nicholls]

Crash 16/3/2012 at 21:36 comments refer

[Walter Nicholls]

Well, looking back at the logs I see another crash when the VM hadn't even been running for half a minute! Attached files VBox-crash-20120316-2136.*: from screenshot note I had probably pressed F8 during boot to do a Safe Mode install or uninstall of Guest additions or video drivers.

Error this time: Guru Meditation -1015 (VERR_RAW_MODE_INVALID_SMP)

after this were two tidy reboots, then the crash in the main bug report.

Aargh! Also for completeness I should mention the two VMS also have different networking (XP one is NAT to any active interface; Win7 is bridged on eth0)

[Walter Nicholls]

Three possibly unrelated things:

• Reduced VM memory to 1.5GB and seems stabler. No total freeze yet
• However I just had 32-bit programs fail to start, instantly turn into zombies (unkillable processes) (eg notepad starts ok, notepad2 does not) so something died in WoW subsystem.
• sfc /scannow reports unfixable problems with D3d9.dll as per ticket #10071.

I foresee a Windows in-place upgrade off CD or worse a reinstall.

[Walter Nicholls]

On shutting down Windows, it hung on 'Shutting down' screen. Fullscreen toolbar does not do anything (only option under Machine is Close and that does nothing). vboxmanage controlvm Scorpio poweroff results in 0%...10%...20%... (then hangs)

kill -9 of the VirtualBox process required. Even if Windows gets its drivers in a twist, surely this should never be necessary?

[Ramshankar Venkataraman]

The VERR_RAW_MODE_INVALID_SMP guru meditation is a bug in the vbox 4.1 branch. It has been fixed and the fix should be available in the next maintenance release for 4.1.

[Frank Mehnert]

Could you check if ​this test build fixes the problem for you?

[Walter Nicholls]

Yes I'll have a go but you're asking me to deliberately crash a work machine, so maybe not today (I have a customer to show something *working* <g>). Since reducing the VM memory allocation to 1.5GB the problems seem to have gone away - both vbox crash and the zombie processes - so maybe it is the 2GB threshold, or the 'more than half the total memory of the machine' .. which is warned as "suboptimal" (3D driver problem remains of course).

Just for my comfort, does it seem to be memory-size related?

[Walter Nicholls]

I spoke too soon - went to shut my computer down to take it to work this morning and the Windows VM was frozen. However different behaviour - the VB toolbar still responded, selected "ACPI Shutdown" and the VM quit instantly. Never seen this before. Still on 4.1.10 haven't put the test one in yet.

[Ramshankar Venkataraman]

What was fixed was the VERR_RAW_MODE_INVALID_SMP guru due to rescheduling to raw mode execution while being interrupted in an STI or other situations that inhibit interrupt delivery to the guest. It is specific to Intel CPUs that do not have unrestricted guest execution support in VT-x. So it's not related to the memory size but to SMP guests. Reducing the vCPUs to 1 would avoid the guru and might work although if the guest is unlucky enough to be executing real mode code and we reschedule to raw mode even with a single vCPU it wouldn't work but I can't say for sure now if we adjust it later on that it doesn't happen.

[Walter Nicholls]

Crash on Mon Mar 26

[Walter Nicholls]

4.1.11 hasn't fixed but may have changed behaviour.

Now running 4.1.11 as suggested, guest VM still has 4.1.10 additions and is has 1.5 GB RAM allocated. It ran ok during weekend, and I was planning at next opportunity to bump the RAM up, but this afternoon it beat me to it and crashed. killall VirtualBox process from full-screen console was required - could not even Alt-Tab away from the guest VM and certainly nothing on the mini-toolbar worked.

If I read this right, the sequence of events was:

• Two days ago: fresh boot up of the Windows VM.
• 3:25pm resume from SaveState (from shell prompt : vboxmanage startvm Scorpio, log shows "Log opened 2012-03-26T02:25:17.259989000Z") Run in full-screen with mini-toolbar.
• about 4pm-ish, switched back to host desktop, did some stuff
• 5:03pm switched back to guest, discovered screen frozen. Scrambled red block along top of screen. Log at this point shows "01:38:18.522 Changing the VM state from 'RUNNING' to 'SUSPENDING'"

Log attached - VBox-crash-20120326-1703.zip

Apart from the log no longer having interesting crash info, this is the same symptom as the first time. I may be more aware of the context now: I can't categorically state that switching back to the VM is what crashed it (this time), but the timing is VERY suspicious.

[Walter Nicholls]

March 26 at 18:43

[Walter Nicholls]

Tried rebooting after last comment.

1. processes failed to start, so...
2. inside windows selected Shutdown from Start menu
3. message "background processes" - force close
4. Windows shows "Shutting down.." for next 5 minutes Host+F un-full-screens, menus work
5. select "Reset" from the Machine window; Warning about premature shutdown, say OK
6. Now can still resize the VB frame window, but menus don't work anymore. clicking here does nothing.

'top' shows VirtualBox process consuming 43% of available memory and 0% of CPU

tail of VBox.log ends at:

01:09:23.070 Changing the VM state from 'RUNNING' to 'RESETTING'.
01:09:23.124 CPUMSetGuestCpuIdFeature: Enabled APIC
01:09:23.124 CPUMSetGuestCpuIdFeature: Disabled x2APIC
01:09:23.124 PIT: mode=3 count=0x10000 (65536) - 18.20 Hz (ch=0)

After 10 minutes, no change - so killed VirtualBox process on host. Log attached.

[Geoff Nordli]

i am getting the same error.

Do you have a test build I can try for: Ubuntu 10.04.4 LTS

thanks,

Geoff

[Walter Nicholls]

Progress report on my tinkering

• Went back to VB 4.1.8 but did not magically fix anything
• Also got an I/O error reading my E: drive (which is a VHD file connected as a second SATA drive to Win7) (new symptom although I may have seen this before).

*SO* I have now re-upgraded to VB 4.1.10, and have created a new Windows 7 VM from scratch, 3D&2D acceleration turned on. It seems stable for last 24 hours, but I have not attached the E: drive yet! Time yet to tell..

[Walter Nicholls]

Speak too soon. On reboot got this again the moment I pressed F8: Guru Meditation -1015 (VERR_RAW_MODE_INVALID_SMP), as mentioned by ramshankar this fixed post-4.1.10.

Windows then reboot with E: attached got BSOD STOP: 0x0000024, select from mini-toolbar Machine->Reset .. Virtualbox locks up, host process using 0% CPU. Last entry in VBox.log is:

00:02:27.316 Changing the VM state from 'RUNNING' to 'RESETTING'.
00:02:27.363 CPUMSetGuestCpuIdFeature: Enabled APIC
00:02:27.363 CPUMSetGuestCpuIdFeature: Disabled x2APIC
00:02:27.363 PIT: mode=3 count=0x10000 (65536) - 18.20 Hz (ch=0)

Had to kill -9 (virtualboxpid), then tried booting Windows again, apart from the usual 'windows has recovered from a serious error' message it seems once again fine.

My opinion of VirtualBox is going through the floor. Although that said I'm not fond of Windows/Microsoft having spent many hours reinstalling various things.

[Walter Nicholls]

Further update

• Completely failed to reboot. VBox process using max allocated to guest CPU (170% = 2 cores at 85%)
• I took the guest back to 1 CPU. (Which it was, once) However still got I/O timeouts accessing the VHD drive. This drive *may* be corrupt now.

I will convert the VHD file to a VDI and see if that improves things. Meanwhile my Windows VM is shutting down with "installing update 1 of 70" ... and it has been on 1 for some time. Time for an extended lunch...

[Geoff Nordli]

4.1.12 was released today.

I would upgrade your host to see if that fixes the problem.

[Frank Mehnert]

Yes, this should be fixed in 4.1.12. In the meantime, 4.1.14 was released.

[Walter Nicholls]

Just to follow up, unfortunately the need to get some work done trumped diagnoses, so I made a lot of changes at once including some drastic ones. Everything seems stable so presumably one of : VB went to 4.1.12; I stopped using .vhd drive image; went back to a single processor; and most significantly started with a completely new Windows Guest install; also now have 3D acceleration turned off. Over the last couple of weeks I haven't run Windows as much as usual so it hasn't had a lot of testing but seems good. (Also note 4.1.14 has a 3D+SMP bug fixed and that might also have been, an issue).