vbox consumes non-trivial CPU when guests are idle

[Paul Millar]

I have Debian sid, running on an Intel Pentium D dual-core 3GHz machine.

I'd like to run multiple guest-OSes: ScientificLinux [similar to CentOS] versions 3.0.x, 4.x and 5.x. I notice about 25%--50% (of a single-core) load on the host OS despite the guest OSes being idle.

With a single guest OS running, I noticed ~50% CPU usage (which linux load onto a single core). With two guest-OSes, the load is around 75% CPU (distributed between the two cores).

This is a problem as it limits the number of guest OSes that I can run comfortably.

If I attach strace to one of the guest OSes, I see many select()s timing-out followed by a call to gettimeofday(). This looks like a polling routine, which perhaps is responsible for host CPU consumption.

So, I guess the questions are:

1. is this a known problem/issue?

2. how (in general) does one investigate this class of problem? Would profiling the virtualbox code (using -pg/gprof) be of any use?

Cheers,

Paul

[the4thchild]

I also see this 20-40% CPU utilization with a Windows XP guest on Fedora 7/8 hosts. Others have noted such findings here:

​http://www.virtualbox.org/ticket/751

(and ​http://forums.virtualbox.org/viewtopic.php?t=2045)

​http://forums.virtualbox.org/viewtopic.php?t=2756

​http://forums.virtualbox.org/viewtopic.php?t=2496

As noted here (​http://forums.virtualbox.org/viewtopic.php?t=2936&highlight=cpu), there appears to be a distinction between CPU usage when the VirtualBox console is open but no VM is on, and CPU usage when the VM is on. The bug with usage when no VM is on seems to have been fixed in 1.5.2, but basal usage when a VM is on still seems to hover around 30% or more in 1.5.x. Posts have also described a 100% CPU usage bug that has been fixed for 1.5.4, but this appears to be related to the clipboard and might be unrelated to this 30% usage. The "nohz=off" fix also has not worked for me.

When I first used VirtualBox at version 1.4.0, I didn't encounter this problem, so I wasn't sure if this is a regression, a consequence of the tickless kernel, or something else. Is this a known issue? Are there other tests that I could perform to help track down this issue?

Best regards,

David

[Kerem ERSOY]

Hi,

I have the smilar problem. I have 4 different setups: Setup 1: Host OS Windows 2003 Server R2 32-bit/ 2 GB RAM / Celeron 2.4 CPU / 250 GB SATA 2 disk. Guest OS Centos 5 using 512 MB memory 16 MB for Graphic adapter. / 200 GB SATA2 Disk (using the second disk as aphysical drive) / guest Additions installed.

When I don't run the VB everything is fine CPU is i1le %1 - %8 utilization. When I run the VirtualBox the utilization goes up to %100 during startup. then When the Guest settles and dislpating %0 utilization. The host has %50 - %60 utilization.

Setup 2: Host OS Windows Vista Ultimate 32 Bit / 2 GB DDRII Dual Channel CAS4 800 RAM / Dual Core AMD 6400+ CPU / 1,3 TB storage SATA2 in 3 drives Guest OS Centos 5 using 512 MB memory 16 MB for Graphic adapter. / 200 GB SATA2 Disk (using the second disk as aphysical drive) / guest Additions installed.

When I don't run the CPU utilzation is %1 - %8 fluctuating. But wehn I start the VirtualBox then the utilization jumps to %40 and fluctuates between %35 - %60. While the guest sits idle.

Setup 3: When I move the guest to XP-SP 2 with an identical HOST utilization is normal Fluctuating between %5 - %15 when both system are idle only a negligable overhead. (I have 2 of these and they are all similar)

Setup 4: Host OS Windows XPSP2 32-bit / 1,5 GB DDRII 533 RAM / Centrino 1,70 / 80 BB storage SATA Notebook Guest OS Fedora Core 7 using 512 MB memory 16 MB for Graphic adapter. / guest Additions installed.

System is %1 - %5 ütlized withot VB and %5 - %15 when Guest is running and idle.

So When using Windows 2003 Server or Vista Ultimate the overhead of VB is nearly %50 more CPU load when guest is idle.

I don't know waht is causing it but I think it should be addressed for the newer releases.

[Henrik K]

Can you clarify if this is anything like my ticket 751?

Is it host load related for you? Can you clearly state what happens when both host and guests are idle, and you start doing something heavy on host? Can you see some idle virtualbox process taking more CPU only then?

[Kerem ERSOY]

Hi Hege,

This is a bit different in that I observe high CPU utilization when both host and guest are idle. There's a high CPU overhead and since both host and guest are near idle this extra %40 - %60 is consumed by VB. This happens only in Vista and Windows 2003 though. However with XP asa a host OS when both are idle VB is also idle. I have no information on XP guest.

[Klaus Espenlaub]

Seems someone needs to bring some facts into this discussion. The initial report and most comments refer to CentOS (or closely related) guests. The problem with that distribution (which it shares with RHEL) is that this kernel uses a 1kHz timer. Handling this amounts of timer interrupts in a VM alone needs a relatively large amount of CPU time. Even if the guest is idle. This applies to all VM products out there, not just VirtualBox.

Thus for CentOS there are kernels available with a lower timer frequency, at ​http://vmware.xaox.net/centos/

Installing such a kernel in the guest should reduce the CPU load on the host greatly.

[Kerem ERSOY]

Thanks for the explanation Klaus. But do you have an explanation how the same guest run in 2003 and Vista has high CPU utiization while over the XP there's no such a condition ? If you read my comment throughly you will also notice that I've also tried with fedora core 7 too. Not just RHEL. Let me add that this character was still same the for an ubuntu 7.0.4 server and as you would have guessed I've read through the forum before posting and tried to start my kernel with 1 Hz too

[Klaus Espenlaub]

Replying to KeremE:

Don't know why you see such differences between different windows version being used as host OS.

[Kerem ERSOY]

Because this is my point Klaus. If I copy the virtual box file over a Vista Ultimate or Windows 2003 server when both host and guest sits idle I observe more than 50% of CPU time is occupied by the VirtualBox. But if I move the same guest to XP then there's no %50 overhead, only 5-10% is occupied by VirtualBox when both are idle.

So it is obvious that there's a difference between 2003/Vista and XP as a host. Why are you trying to convince me that there's nothing different by Windows hosts? Are you sure that you've read my inital post completetly?

[Klaus Espenlaub]

In the first place I started with answering the issue the original reporter had. And that was on Linux host, which is a lot more sensitive to timer issues.

From that point of view it should be rather easy to understand that my "don't know" simply means that I don't know why you see differences. I'm not trying to convinvce you that your observation is wrong. If you read this between the lines I certainly didn't mean it.

[Paul Millar]

Replying to klaus:

Seems someone needs to bring some facts into this discussion.

[a very minor point: you certainly bring some much needed information. We have some observations, but (I, certainly) lacked the knowledge to say what they mean. "Facts" tend to cover both concepts]

The initial report and most comments refer to CentOS[-like OS that uses a kernel with] a 1kHz timer.

Ah, I was unaware of this.

Handling this amounts of timer interrupts in a VM alone needs a relatively large amount of CPU time. Even if the guest is idle...

This certainly fits the observations.

Thus for CentOS there are kernels available with a lower timer frequency, at ​http://vmware.xaox.net/centos/

Installing such a kernel in the guest should reduce the CPU load on the host greatly.

Thanks Klaus, You're absolutely right.

I downloaded a low-timer-resolution kernel (from above URL) and installed it. The new image ran with much lower overhead. I now see CPU usage (on host system, when guest is idle) of ~8%. This overhead does sometimes drop as low as 3%, so perhaps the guest can be optimised further by stopping unnecessary processes.

As you say, the interrupt handler is called frequently. The fix is to reduce the timer frequency. However, I'd say, it would be better to reducing the CPU work required per interrupt. Is it known what dominates this load? Perhaps the gettimeofday() call?

[as an aside, I tried to run VirtualBox with a number of profilers, but was unable to get it to work. Is there any method of profiling VBox's CPU usage?]

Cheers,

Paul.

[Klaus Espenlaub]

Running VirtualBox in a default app profiler won't work. It's simply too complicated, and there are a large number of mines the profiler can step on. As VirtualBox (from an outside point of view) runs guest code, it has a rather complicated control flow, including switching from/to different processor contexts. VirtualBox can be compiled to collect profiling information, and if the information it already produces doesn't suffice, just add further timing/frequency measurements. A small amount of profiling code is included in a release build, but this probably doesn't cover information needed for optimizing the cost of guest interrupt handling.

But as there have been some changes to the timer code in the current SVN version (not in the 1.5.x series), performance might already have been improved.

[Paul Millar]

Replying to klaus:

Running VirtualBox in a default app profiler won't work. It's simply too complicated, [...]

[I can well believe this!]

VirtualBox can be compiled to collect profiling information, and if the information it already produces doesn't suffice, just add further timing/frequency measurements A small amount of profiling code is included in a release build, but this probably doesn't cover information needed for optimizing the cost of guest interrupt handling.

This sounds promising. Could you give a few more hints? The FAQ and Tech. docs didn't seem to mention profiling (or I may have simply missed it).

But as there have been some changes to the timer code in the current SVN version (not in the 1.5.x series), performance might already have been improved.

Are these improvements in SVN Trunk? I compiling from SVN a few days ago (a naive attempt to use "-pg"). I was using the stock (1kHz) RHEL-4 kernel. If there was a performance improvement, it wasn't as dramatic as switching to the CentOS "Vmware" kernel.

Cheers,

Paul.

[Kerem ERSOY]

Hi,

Now I can confirm that vmware kernel solved my Vista nad Windows 2003 problem. Previously I'd tried to add nohz switch without luck.

But since everything seems ok with XP host should I try to use vmware kernel too? Furthermore should we think that XP is more immune to this timer issue? What might cause this behaviousr? Because as far as I understand if there's a problem with this issue we can fix it using a fixed kernel also it is possible to fix it at the host os side too.

[Carl Thompson]

I have the same problem on 1.5.0 and 1.5.2 with a Linux host (2.6.23.8 kernel, 2GB, Intel T1060 @1.6Ghz) running both Windows XP and Linux guests. I have tried the "nohz=off" kernel option with no success. When the XP guest is idle the host consumes 25%-35% CPU. It's similar for the Linux guest. By Contrast when running VMWare Server with an XP guest with the exact same software the host idles at about 4%-5% CPU.

Thanks, Carl

[Kerem ERSOY]

Hi Carl,

I had tried to with "nohz=off" also with no avail. But the kernel compiled with lower Hz value solved each of our high CPU overhead problem. Please follow the link in klausm's post. Grab your kernel from there (please selecet the plain kernel olny because currently VBox does not support PAE)

Cheers, K.

[Kerem ERSOY]

Replying to klaus:

In the first place I started with answering the issue the original reporter had. And that was on Linux host, which is a lot more sensitive to timer issues.

From that point of view it should be rather easy to understand that my "don't know" simply means that I don't know why you see differences. I'm not trying to convinvce you that your observation is wrong. If you read this between the lines I certainly didn't mean it.

Sorry Klaus, when you told "see" I thought what you mean was "why I see things that way?" while you've really meant "you have no idea why I observe such a difference between different Windows platforms". This caused the flame. But now I got the kernel (installed the vmware repo in addition for further releases) and it works like charm on both 2003 and Vista. But from what I observe, the fix can be from both host side and guest side (and definitely makes things more complicated) and somehow XP is far less sensitive to these timer interrupt issues than Win 2003 and Vista..

Thanks again.

Cheers, K.

[Carl Thompson]

After recompiling my kernel with the timer freq set all the way down at 100Hz (it was already at 300Hz) things improve to where the host's processor is between 10%-20% busy when the guest is completely idle but that still is unacceptable because: 1) That's still too much CPU wasted. 2) It's unreasonable to expect me to lower my timer freq to 100Hz because of the the negative effect that has on interactive performace. I would also like to be able to enable the tickless option again because it greatly increases battery life.

I don't want to have to cripple my system to use VirtualBox!

Thanks, Carl

Replying to KeremE:

Hi Carl,

I had tried to with "nohz=off" also with no avail. But the kernel compiled with lower Hz value solved each of our high CPU overhead problem. Please follow the link in klausm's post. Grab your kernel from there (please selecet the plain kernel olny because currently VBox does not support PAE)

Cheers, K.

[Kerem ERSOY]

Replying to clipcarl:

After recompiling my kernel with the timer freq set all the way down at 100Hz (it was already at 300Hz) things improve to where the host's processor is between 10%-20% busy when the guest is completely idle but that still is unacceptable because: 1) That's still too much CPU wasted. 2) It's unreasonable to expect me to lower my timer freq to 100Hz because of the the negative effect that has on interactive performace. I would also like to be able to enable the tickless option again because it greatly increases battery life.

I don't want to have to cripple my system to use VirtualBox!

Thanks, Carl

Karl,

I really did not understand how would you possibly cripple your system. The Hz solution we're talking about here is for the guest OS only not for the Host itself. All you could possbly cripple is your guest.

Cheers K.

Replying to KeremE:

Hi Carl,

I had tried to with "nohz=off" also with no avail. But the kernel compiled with lower Hz value solved each of our high CPU overhead problem. Please follow the link in klausm's post. Grab your kernel from there (please selecet the plain kernel olny because currently VBox does not support PAE)

Cheers, K.

[Carl Thompson]

Replying to KeremE:

I don't want to have to cripple my system to use VirtualBox!

I really did not understand how would you possibly cripple your system. The Hz solution we're talking about here is for the guest OS only not for the Host itself. All you could possbly cripple is your guest.

Oh, that wasn't clear. I thought the Hz solution was for the _host_ not for the _guest_. And changing it on the _host_ does appear to have helped a little (or maybe it was compiling without tickless that helped... although I was use the "nohz=off" kernel boot option before which should be the same, right?).

But on the guest side, the primary guest I want to run is Windows XP. I have no idea how I would change the timer resolution of that. Also the Linux guest I'm trying to run is a Linux Media Center live CD so I can't change the timer resolution there either (and wouldn't want to for multimedia performance).

Thanks, Carl

[Carl Thompson]

BTW, the CPU usage percentages I'm giving are of the total CPU on my dual-core system. So when I say a single idle guest is using 10%-35% of the host's CPU, I mean it's using 20%-70% of a single processor core.

[Kerem ERSOY]

Replying to clipcarl:

Replying to KeremE:

I don't want to have to cripple my system to use VirtualBox!

I really did not understand how would you possibly cripple your system. The Hz solution we're talking about here is for the guest OS only not for the Host itself. All you could possbly cripple is your guest.

Oh, that wasn't clear. I thought the Hz solution was for the _host_ not for the _guest_. And changing it on the _host_ does appear to have helped a little (or maybe it was compiling without tickless that helped... although I was use the "nohz=off" kernel boot option before which should be the same, right?).

Right all the settings and kernels are for the guest. One more thing about the lower Hz setting, 100 Hz was the defult for 2.4.x kernels and it was doing a nice job with that. The transition to the 1000 Hz default kernel was made with 2.6.x series kernels and mostly done to ease time calculations easier since 1 ms is the period of 1 Khz clock. (This was told by Linus Torvalds himself.) I don't think it will have harm to work with a 100 Hz kernel apart from reduced timer sensitivity if you realy need it.

But on the guest side, the primary guest I want to run is Windows XP. I have no idea how I would change the timer resolution of that. Also the Linux guest I'm trying to run is a Linux Media Center live CD so I can't change the timer resolution there either (and wouldn't want to for multimedia performance).

Yeah you need to recompile your kenel to achieve that for Linux. There's nothing to do for XP. I am not sure if it is an issue for XP too. BTW 1/100 is even better than default XP timer hich is currently set around 60 - 65 Hz. AFAIK :)

Thanks, Carl

[Carl Thompson]

BTW, I tested with a FreeBSD guest as well as on a different host and get the same problem... So since this problem is evident with FreeBSD guests, Linux guests and Windows guests it seems clear that this is indeed a VirtualBox bug and not a guest configuration issue. Especially since VMWare doesn't exhibit the same behavior.

Since modifying the guest OS isn't reasonable (unless you're running Xen!) I guess there's nothing to be done then except wait for a fix for this bug from the VirtualBox team. I hope it's in the next version because I really would like to migrate from VMWare to VirtualBox!

[Kerem ERSOY]

Replying to clipcarl:

BTW, I tested with a FreeBSD guest as well as on a different host and get the same problem... So since this problem is evident with FreeBSD guests, Linux guests and Windows guests it seems clear that this is indeed a VirtualBox bug and not a guest configuration issue. Especially since VMWare doesn't exhibit the same behavior.

Since modifying the guest OS isn't reasonable (unless you're running Xen!) I guess there's nothing to be done then except wait for a fix for this bug from the VirtualBox team. I hope it's in the next version because I really would like to migrate from VMWare to VirtualBox!

Please don't forget that the kernels we discus here are built by VMWare ! and for the VMWare hosts and we're benefiting from them like VMWare users. Will you please test with one fot these kernels built by VMware for VMware ?

[Carl Thompson]

Replying to KeremE:

Please don't forget that the kernels we discus here are built by VMWare ! and for the VMWare hosts and we're benefiting from them like VMWare users. Will you please test with one fot these kernels built by VMware for VMware ?

I don't have have any VMs I need/want to run where I could use a custom Linux kernel. The VMs I run are Windows, FreeBSD, Solaris and occasionally a live CD. Running one of those modified guests would serve no purpose for me; I have no workload that requires that. (All of my _real_ computers are Linux.) So even if VirtualBox can run those particular modified Linux guests without the high processor usage when idle it has no benefit for me. You've already said they work for you and I trust you. I just can't use them myself.

[Kerem ERSOY]

I get it. But I am afraid you are in a wrong tread. The original asker had told he was running Linux guests so did I. All the discussion in the tread belongs to this subject. Please keep browsing the forum.

[Carl Thompson]

It's all one bug. You shouldn't have to run a special kernel for Linux guests either! Since we are all talking about the same bug (high CPU use in host when guests are idle) it seems appropriate to post here. I see know reason we need to create separate bugs / discussion threads for each host / guest combination when we are all talking about one particular bug!

Replying to KeremE:

I get it. But I am afraid you are in a wrong tread. The original asker had told he was running Linux guests so did I. All the discussion in the tread belongs to this subject. Please keep browsing the forum.

[the4thchild]

I found a user discussion thread that might be relevant: ​http://article.gmane.org/gmane.comp.emulators.virtualbox.general/2374

[Robert Davies]

Under Host OS OpenSuSE 10.3, running on AMD X2 5600+ box.

Forget CONFIG_HZ, it's something else that's causing the problem.

OS 10.3 - 2.6.22.13-03.3 default distro kernel as guest, NO CPU waste when idle, CONFIG_HZ_250=y Debian stable 2.6.18, kernel, Heavy wastage when idle, 2.6.18 CONFIG_HZ_100=y

I've tried kernel with/without space optimisation on Debian, and tried to make minimal kernel to fix this, annoying issue.

I'm going to try, upgrading to testing and see if a more recent Debian kernel, solves it.

SuSE 10.3 - text mode install, just slightly larger than Debian minimal install zypper has interface similar to apt-get.

Logged activity with vmstat :

rob@fir:~> zcat /proc/config.gz | grep HZ CONFIG_NO_HZ=y # CONFIG_HZ_100 is not set CONFIG_HZ_250=y # CONFIG_HZ_300 is not set # CONFIG_HZ_1000 is not set CONFIG_HZ=250 CONFIG_MACHZ_WDT=m

rob@fir:~> vmstat 60 procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----

r b swpd free buff cache si so bi bo in cs us sy id wa 2 0 0 2416204 66412 462672 0 0 670 139 305 770 7 3 80 10 0 0 0 2415456 67112 462676 0 0 0 23 242 1087 1 1 98 1 0 0 0 2415828 67464 462676 0 0 0 13 235 974 0 0 99 0 0 0 0 2416076 67916 462676 0 0 0 13 242 1038 0 1 98 1 0 0 0 2415968 68036 462676 0 0 0 7 234 916 0 0 99 0 0 0 0 2415524 68404 462680 0 0 0 8 240 1040 0 0 98 1 0 0 0 2415448 69420 463032 0 0 14 16 246 991 1 1 97 1 2 0 0 2415156 69860 463132 0 0 2 21 254 959 0 0 98 1 0 0 0 2414940 70120 463408 0 0 0 12 247 902 0 0 98 1 0 0 0 2414816 70256 463460 0 0 0 5 238 832 0 0 99 0 0 0 0 2414260 70456 463764 0 0 1 11 238 841 0 1 98 1 0 0 0 2411904 70816 463952 0 0 0 16 253 989 3 1 95 1 0 0 0 2411724 71016 464144 0 0 0 9 250 952 1 1 98 1 1 0 0 2412556 71528 464400 0 0 4 19 252 940 0 1 98 1 0 0 0 2412260 71764 464400 0 0 0 11 252 1022 2 0 97 1 0 0 0 2411812 72232 464396 0 0 0 18 249 927 2 1 96 1 0 0 0 2411100 72604 464728 0 0 1 17 253 902 0 1 98 1

Switch to Debian Guest 2.6.18 CONFIG_HZ_100=y 0 0 0 2717396 73224 575436 0 0 249 1842 274 1336 14 8 75 3

0 0 0 2716800 73508 575732 0 0 2 10 260 847 8 11 80 1 1 0 0 2716576 73752 575732 0 0 0 7 270 899 7 10 82 1 0 0 0 2713452 74244 575740 0 0 0 18 266 1620 9 11 80 1

procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----

r b swpd free buff cache si so bi bo in cs us sy id wa 0 0 0 2631100 74652 612576 0 0 602 14 272 1001 8 13 75 3 0 0 0 2630760 74792 612724 0 0 0 8 237 710 8 12 80 1 0 0 0 2611620 75044 619100 0 0 71 40 242 745 8 13 77 2 0 0 0 2609540 75328 620476 0 0 15 20 243 762 8 12 79 1 0 0 0 2609260 75472 620484 0 0 0 9 263 861 9 12 79 0

[Robert Davies]

Apologies, too used to text mode!

SuSE 10.3

rob@fir:~> zcat /proc/config.gz | grep HZ
CONFIG_NO_HZ=y
# CONFIG_HZ_100 is not set
CONFIG_HZ_250=y
# CONFIG_HZ_300 is not set
# CONFIG_HZ_1000 is not set
CONFIG_HZ=250
CONFIG_MACHZ_WDT=m

rob@fir:~> vmstat 60
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 2  0      0 2416204  66412 462672    0    0   670   139  305  770  7  3 80 10
 0  0      0 2415456  67112 462676    0    0     0    23  242 1087  1  1 98  1
 0  0      0 2415828  67464 462676    0    0     0    13  235  974  0  0 99  0
 0  0      0 2416076  67916 462676    0    0     0    13  242 1038  0  1 98  1
 0  0      0 2415968  68036 462676    0    0     0     7  234  916  0  0 99  0
 0  0      0 2415524  68404 462680    0    0     0     8  240 1040  0  0 98  1
 0  0      0 2415448  69420 463032    0    0    14    16  246  991  1  1 97  1
 2  0      0 2415156  69860 463132    0    0     2    21  254  959  0  0 98  1
 0  0      0 2414940  70120 463408    0    0     0    12  247  902  0  0 98  1
 0  0      0 2414816  70256 463460    0    0     0     5  238  832  0  0 99  0
 0  0      0 2414260  70456 463764    0    0     1    11  238  841  0  1 98  1
 0  0      0 2411904  70816 463952    0    0     0    16  253  989  3  1 95  1
 0  0      0 2411724  71016 464144    0    0     0     9  250  952  1  1 98  1
 1  0      0 2412556  71528 464400    0    0     4    19  252  940  0  1 98  1
 0  0      0 2412260  71764 464400    0    0     0    11  252 1022  2  0 97  1
 0  0      0 2411812  72232 464396    0    0     0    18  249  927  2  1 96  1
 0  0      0 2411100  72604 464728    0    0     1    17  253  902  0  1 98  1


Switch to Debian Guest  2.6.18 CONFIG_HZ_100=y
 
0  0      0 2717396  73224 575436    0    0   249  1842  274 1336 14  8 75  3
 0  0      0 2716800  73508 575732    0    0     2    10  260  847  8 11 80  1
 1  0      0 2716576  73752 575732    0    0     0     7  270  899  7 10 82  1
 0  0      0 2713452  74244 575740    0    0     0    18  266 1620  9 11 80  1
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 0  0      0 2631100  74652 612576    0    0   602    14  272 1001  8 13 75  3
 0  0      0 2630760  74792 612724    0    0     0     8  237  710  8 12 80  1
 0  0      0 2611620  75044 619100    0    0    71    40  242  745  8 13 77  2
 0  0      0 2609540  75328 620476    0    0    15    20  243  762  8 12 79  1
 0  0      0 2609260  75472 620484    0    0     0     9  263  861  9 12 79  0

[Robert Davies]

Looks like this one was some kind of kernel bug. I tested with 1.5.2_OSE kernel upgrade, makes things very acceptable.

Done Debian upgrade to testing, installed my minimal vbox kernel - 2.6.22-vbox :

rob@fir:~> vmstat 5
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 1  0    168 411324 152480 2577048    0    0   160    97  315 1087 12 16 70  3
 7  0    168 410976 152504 2577308    0    0    51    42  394 1535  1  2 92  5
 2  0    168 410196 152536 2578080    0    0   157    26  327 1569  2  2 92  4
 4  0    168 408640 152536 2579232    0    0   230    15  343 1749  3  1 94  2
 3  0    168 408336 152556 2579652    0    0    82    45  367 1584  2  2 93  3
 1  0    168 408416 152592 2579652    0    0     0    35  279 1386  2  1 95  2

# Stop my bit torrent session 

 3  0    168 408448 152592 2579660    0    0     0    28  280 1245  1  1 98  1
 2  0    168 408400 152592 2579660    0    0     0    26  264 1325  2  1 96  2
 2  0    168 408400 152592 2579660    0    0     0     4  300 1460  2  1 97  1
 2  0    168 413480 152596 2579660    0    0     0    23  248 1198  2  0 97  0
 2  0    168 413528 152596 2579660    0    0     0    16  304 1401  0  1 98  1
 3  0    168 416016 152596 2579660    0    0     0     6  300 1514  2  1 96  1
 2  0    168 416048 152596 2579660    0    0     0    16  279 1306  0  0 98  2

One extra change was removing SMP (sorry but kernel build takes too long not too apply more changes), but the SuSE kernel wasn't single CPU so I don't think that effects things.

[Jozef Kutej]

I'm running VBox 1.5.4 on Debian Etch.

I setup my the virtual machine to boot from the cdrom. It has ACPI enabled, IO APIC disabled, VT-x enabled. I change no configuration just the cd-s in the drive. All are i386. I touch nothing I just let the os boot.

Ubuntu 7.10 install cd is taking 0-1% of cpu. Ubuntu 6.10 install cd is taking 30-40% of cpu.

(Debian install cd-s starts with choose language dialog) Debian Etch install cd first minute 130% (as i have two cores) and then 30-40% of cpu. Debian testing install cd first minute 100% and then 0-1% of cpu.

DSL 4.2.1 live cd 30-40% of cpu.

Any ideas how to "fix" the cpu usage?

[Robert Davies]

The guest kernel upgrade solved it for me, perhaps the 2.6 introduction of working tickless mode, reduces the frequency of wake ups, or some spin lock is not spinning, no more. I had good results with 250Hz OpenSuSE 2.6.22 kernels, and upgrading from 2.6.18 to 2.6.22 Deboam kernel had similar effect. Not a significant difference between 250Hz and 100Hz from my results.

[Jozef Kutej]

OK when I put 2.6.22-3-686 from Debian Lenny to my Debian Etch machine it's not using 30-40% of cpu in idle mode. It's using 100%... But for a first minute or so, after boot. Then it's ok - 0%. Quite strange I would say.

SO it works, but it's a hack or workaround but not a solution. Why in VBox 1.4 it worked fine and in the versions after no more?

[Frank Mehnert]

jk, do you have the clipboard enabled? If so, please could you disable the clipboard and check if that makes any difference?

[Jozef Kutej]

I disabled the clipboard. At the first I thought it helped but no... I was doing tests using GUI. First I booted up (choosed in the grub) 2.6.18-5-686 and it took 22s to boot, for next 60s 130% cpu (I have dual core), then 30% in idle. Then I logged in and rebooted the machine with 'reboot' command, choosed 2.6.22-3-686 and after the boot 0% cpu in idle right away. So it seem to be working. But afterwards I shutdown the machine with 'halt' and started the machine with Start button in the GUI again and the one minute with 100% cpu usage was back there.

When I start the machines with "VBoxManage startvm Debian -type vrdp", the one minute 100% cpu penalty is not there.

Never the less it's not a big problem as it's just one minute, but the 30% cpu usage with old kernels is not nice. That one is there all the time.

[Technologov]

This looks like a duplicate of bug #824.

-Technologov

[Michael Thayer]

Could you please this in VirtualBox 1.5.6 and reopen if it is still reproducible?

[Carl Thompson]

This problem still exists in VirtualBox 1.5.6.

[the4thchild]

I also see this problem with a WinXP guest on a Linux host in VirtualBox 1.5.6. I think it's also related to newly opened ticket #1233.

[bvrom]

I just installed VirtualBox 1.5.6 (1.5.6r28266 according to VBoxManage -version) on a fully updated Ubuntu Gutsy host. As my guest machine I installed Debian etch on a VM with 32Mb memory and 2GB diskspace. The installation process went fine, but I am experiencing the same problem as some of the other people in this thread. While my guest is completely idle (0.3% or 1.0%CPU usage according to top) the VirtualBox process on the host uses between 37%-45% CPU continuously. While the host is not a modern system (1333 MHz Intel Celeron CPU) I wonder if this is normal. I haven't (yet) tried any of the suggestions (e.g. lowering the Hz setting in the kernel). I would prefer to be able to use the stock Debian Etch kernel. Bram

[Kerem ERSOY]

Hi,

Please try to install one of these low Hz. kernels. I did it this way and I have no problem on my Linux guests anymore. I use VMware kernels over Centos 5 distro. I have 3 production servers with no obvious problems.

[bvrom]

I recompiled the standard Debian Etch kernel (2.6.18....) on the guest with CONFIG_HZ_100=y. CPU usage of the VirtualBox process on the host is now down to 18%-28% with an idle guest. While this is an improvement, I still think this is a lot. Can I improve this further? Perhaps by installing the guest extensions?
Bram

[Robert Davies]

Yes, upgrade to a newer version kernel using the Debian kernel tools. Your figures are very similar to mine reported "12/10/07 06:38:43" in this thread.

[Rian Quinn]

I have installed Ubuntu Gutsy 7.10 will all of the updates. The host is unmodified and the hardware is relatively up-to-date (Intel Core 2 Duo with V-Tx).

I am running a Windows XP Guest to be able to run Office 2007. When VirtualBox is not running the machine runs amazing. When I start up VirtualBox things go wrong.

If I let the host and the guest go idle Virtual Box consumes a large portion of CPU in intervals (meaning every once and a while the CPU usage jumps from 10% to 100%). This of course freezes the entire machine rendering the host unusable until VirtualBox relaxes, thus rendering the host usable again. What I have noticed is if I take the mouse and keep moving it everything is ok. However once I stop moving the mouse (host or guest) VirtualBox does its thing. This of course is really frustrating.

For example, as I write this, every time I stop moving my mouse everything freezes for a second or two and then goes back to normal.

Rian

[Rian Quinn]

Replying to rianquinn:

I have installed Ubuntu Gutsy 7.10 will all of the updates. The host is unmodified and the hardware is relatively up-to-date (Intel Core 2 Duo with V-Tx).

I am running a Windows XP Guest to be able to run Office 2007. When VirtualBox is not running the machine runs amazing. When I start up VirtualBox things go wrong.

If I let the host and the guest go idle Virtual Box consumes a large portion of CPU in intervals (meaning every once and a while the CPU usage jumps from 10% to 100%). This of course freezes the entire machine rendering the host unusable until VirtualBox relaxes, thus rendering the host usable again. What I have noticed is if I take the mouse and keep moving it everything is ok. However once I stop moving the mouse (host or guest) VirtualBox does its thing. This of course is really frustrating.

For example, as I write this, every time I stop moving my mouse everything freezes for a second or two and then goes back to normal.

Rian

Well I might have solved my own problem. Once I ejected the CD, and turned off the mount, everything seemed to go back to normal. I have been working now for a few hours with not a single glitch.

So that makes me ask the question, why would the CD being mounted to the CD-ROM device cause the system to periodically (like once a minute) lock-up even if I'm not using it.

Rian

P.S. Using VirtualBox version 1.5.6

[bvrom]

I have installed the 2.6.22 kernel in Etch now using the backports. CPU usage of the VirtualBox process with an idle guest is now down to 3%-4%, which I assume is what it should be. This is definitely acceptable for me.
Thank you RobVBox for the suggestion.

Bram

[Carl Thompson]

Of course that doesn't help those of us using non-Linux guests or Linux guests where the kernel can't be recompiled.

[Robert Davies]

Are you turning off tickless mode in the host Linux kernel on boot (parameter nohz=off as suggested by the FAQ)?

My /proc/cmdline : root=/dev/sda5 vga=0x317 resume=/dev/disk/by-label/SWAP splash=silent showopts CPUFREQ=no nohz=off

Possibly someone realising why it doesn't happen with 2.6.22 guest but does with 2.6.18; may point to a real solution in Vbox. Also 2.6.18 is almost an "ancient" kernel now, so fewer and fewer will be affected.

[toaste]

I have a similar issue in VirtualBox 1.5.6

Host is Ubuntu 7.10 (2.6.22-14-generic) running SMP on a dual core processor. When booted, VirtualBox consumes an entire core even when guest is idle (pegged at 50%).

Guest is a migrated Windows XP Pro install, and requires IO ACPI be enabled to boot in VirtualBox.

Guest OS was migrated to another virtual machine (WMware Server 1.0.4). Guest does not cause high host cpu use in VMWare.

[Carl Thompson]

I have tried turning off the "nohz" option on the host machine in the past with no success. I believe the success some have had with this option is when it's used on the guest machine not the host. And of course having to set the "nohz" option off on the host is contrary to the point of virtualization which is to save resources. The "nohz" options saves both processor and power so it is sub-optimal to have to turn it off.

Also I've noticed that VirtualBox still uses an unacceptable amount of processor even when the guest OS is paused (using VirtualBoxes' "pause" feature). So there is definitely something wrong with VirtualBox.

I believe all these posts about running special guest kernels or disabling useful host kernel features are counter-productive and detract from the real problem in VirtualBox. VirtualBox is supposed to be a general-purpose virtualization solution like VMWare and not require modified OS's. If we wanted to have to run modified kernels then we'd use Xen!

[Kerem ERSOY]

Dear Clipcarl,

I don't know how familiar are you with developing this type of "general purpose virtualiztion code". But you should know that the "Virtualization" world is not a perfect world. So there are several problems and drawbacks du to how CPU's and OS kernels and BIOS' es are implemented. The point you are missing is this is not just a problem of VirtualBox, but you can see the same problem with lots of other virtualizers. This is why, one of the soundest advices in this tread leads you to a kernel veleopped by and for VMWare (another virtualization server) and fixes the problem. But you should espect these kind of problems when you are urnning two differen OS'es designed to be exclusive owner of the hardware. The point is to pinpoint the problems causing the consumation of all process power and eliminating them with a minimum impact on both host and guest. when it comes to this point calling a timer interrupt 100 times a second rather than 1000 seconds will not cause you a problem if youa re no trying to process 1/100th second intervals. So answers on this tread you see are very fit and solved the problems for most people. Please try to see matters a bitt like that. VirtualBox is just a virtualizer. what the host and guest does on theri timeshare is another problem. BTW using Xen kernels have problems of their own which is no different that this type of unacceptatble CPU busy situations etc.

[wphilips1]

Dear KeremE,

I am suffering from a similar problem, but with a windows XP guest and a linux host. So have other people (see the beginning of this thread).

With a windows guest it is not possible to change the kernel (I think). Even if it were possible it would probably involve reinstalling the whole machine (in contrast to a linux guest where the client upgrade would be much simpler).

Many people use virtualbox to run windows on linux, so as many other posters, I hope this issue can be resolved.

PS. I have run the same (or very similar) virtual machine on three different systems:

-acer travelmate 6590 series with fedora 8 (dual core)

-acer travelmate 803 with fedora 8 9single core (single core centrino)

-AMD Athlon(TM) XP 2600+ desktop running fedora 8

The problem persists on all machines. Adding the nohz=off option to the kernel command line does not make any difference. The cpu time consumed by virtual box on the host while the client is idle (as verified in windows task manager) is consistently around 15-20%

[Robert Davies]

Replying to clipcarl:

I have tried turning off the "nohz" option on the host machine in the past with no success. I believe the success some have had with this option is when it's used on the guest machine not the host.

No, and counter to information provided in FAQ. For instance I've found recent Linux guests seem to work happily with this option permitted, with it disabled in the host. You want it, because it reduces the number of wake ups that the guest OS will cause. It'd be nice if it's no longer necessary in host, but if I've a CPU wastage problem, then I definitely need to turn it off before adding to bug reports.

The FAQ may be out of date, and with recent Linux kernels, perhaps the host can have this option to, but if you've found a bug like excess CPU and you don't turn on this option, or clearly mention that it makes no difference then your report is less useful.

And of course having to set the "nohz" option off on the host is contrary to the point of virtualization which is to save resources. The "nohz" options saves both processor and power so it is sub-optimal to have to turn it off.

True, but tickless kernels were recently introduced and problems were found running the host tickless. The Virtualisation software may make assumptions about interrupt processing so it's hardly surprising if such a radical change in the host, would require modifications in Vbox. Those interrupts are where most access to the 'iron' is taking place, and VBox is a component of the Linux kernel on the host. Intimacy implies more knowledge, and that then tends to require updates due to the tighter coupling.

Also I've noticed that VirtualBox still uses an unacceptable amount of processor even when the guest OS is paused (using VirtualBoxes' "pause" feature). So there is definitely something wrong with VirtualBox.

Are you running a configuration that's taking account of known bugs as per FAQ?

My data suggests that 2.6.22 host + guest works better, and I did not see CPU wastage issues with Win98SE (once Rain installed) nor with Vista.

I believe all these posts about running special guest kernels or disabling useful host kernel features are counter-productive and detract from the real problem in VirtualBox

Actually folk are providing information, actually 'clues' which may help a developer find the underlying problem. It is those just making a general complaint, and not providing specific information that confuse things for other users and are not helping the developers, find a bug.

Those encountering problem with Linux guest OS and 2.6.18 have a definite route to upgrade kernel inorder to remove the issue. It could very well be a kernel bug in 2.6.18 rather than a VBox problem, the expectation that virtualisation software could solve all guest OS issues is unrealistic. For instance Win98SE requires rain to be installed, so that the Win guest idles the CPU rather than burns it up.

Nobodies saying that it's nice not to take advantage of "tickless" kernel, enusre that HZ is not too fast and suchlike, when the cause of problems is discovered, then restrictions are likely to become uncessary.

[the4thchild]

I used to encounter this problem, but it seems to have resolved itself over the past few weeks. Previously, my WinXP guest on Fedora 8 host used 30-40% of the CPU while idle, but now I see around 5% usage. I don't recall a dramatic improvement with the upgrade to VirtualBox 1.5.6 alone, but perhaps the combination of 1.5.6 plus the upgrade to kernel 2.6.24 made the difference. Is anyone else seeing this improvement?

[wphilips1]

I am currently running

2.6.24.3 #2 SMP Fri Mar 28 14:42:12 CET 2008 x86_64 x86_64 x86_64 GNU/Linux

with VirtualBox 1.5.6

VirtualBox is still consuming 20-30% cpu time when the windows-xp client is idle. So, unlike the4thchild, I do not see any improvement because of a more recet kernel.

[dirkgently]

To add my two cents worth. I'm running a CentOS 4.6 guest with no package updates on a WindowsXP SP2 host running Virtual Box 1.5.6. I have a dual core Intel Core 2 6600 CPU. The kernel versions I'm running are:

• CentOS 4.6 SMP kernel is 2.6.9-67.EL
• CentOS 4.6 non-SMP kernel is 2.6.9-67.EL
• vmware.xaox.net non-SMP kernel is 2.6.9-67.0.7.plus.c4.VMware

I've found the following things:

1. CentOS 4.6 SMP kernel gets stuck on boot and never gets past "Ok, booting the kernel."

2. CentOS 4.6 non-SMP kernel boots and consumes all of one core while idling in X windows. top within the CentOS guest shows it is idle as far as it's concerned.

3. Using the CentOS 4.6 non-SMP kernel and setting the affinity of the VirtualBox process in Windows Task Manager to CPU0 changes the situation dramatically. CPU usage goes to ~25% of one core.

4. Using the CentOS 4.6 non-SMP kernel from ​http://vmware.xaox.net/centos/4.6/i386/ AND setting affinity to CPU0 changes the situation dramatically again. There is now virtually no CPU usage on the Windows host when the CentOS guest is idling.

5. Using the CentOS 4.6 non-SMP kernel from vmware.xaox.net and NOT setting affinity, CPU usage goes up again to consume all of one core.

The optimal solution for me has been setting affinity to CPU0 and using the latest non-SMP kernel from vmware.xaox.net. I've not yet tried an SMP kernel from vmware.xaox.net. It may have a similar problem to the CentOS 4.6 SMP kernel.

I should also note all of this testing has been with VT-x turned off in the virtual machine settings. I haven't gone through any of the above iterations with VT-x turned on.

My problems at least seem to stem from the combination of running Virtual Box on a dual core system and using a high hertz kernel. Using a low hertz kernel and setting the process affinity to one core or the other dramatically improves things.

See: ​http://forums.virtualbox.org/viewtopic.php?t=5511

This seems similar to ticket #1233

[fzzzt]

I'm seeing this problem on 1.6.4 with FreeBSD 7.0-RELEASE with the GENERIC kernel (not ULE, though I don't know if that alters timing). I've changed kern.hz from 1000 to 100 and even to 50. Regardless of that setting my host CPU usage is a solid 50% when idle (saturating one of my two cores). I really mean solid, too; it probably varies <5% in either direction.

My host system (Vista 64-bit) isn't doing anything except running VirtualBox. I'm running benchmarks comparing various VM apps. When the guest is processing, the host stays at 50%. When the guest is idle, the host still stays at 50%. I haven't tried higher than 1000 or anything else, like device polling, but I may just to see what happens.

I haven't tested VirtualPC 2007 yet but will do that next, once I get my VB numbers. I wonder if I'll see a similar problem with that...

[wphilips1]

In the mean time I discovered something very strange: instead of using a earlier virtual machine converted from vmware, I create a new one and installed a fersh copy of windows, office etc. (I did this very reluctantly because it takes a great deal of time).

The problem did not appear in this new virtual machine.

Now here is the strange thing: when I run the old machine (converted from vmware) and the new one simultaneously, the cpu time is very low when both guests are idle. If I stop the new machine, the cpu time of the old one rises to a large value.

In other words: both machines together consume less cpu time than the old one alone.

I think this may shed some light on the problem.

[Frank Mehnert]

Your observation is explainable. With two concurrent VMs VirtualBox uses different scheduling parameters. Future releases will allow to tune these settings.

[fzzzt]

Replying to wphilips1:

The problem did not appear in this new virtual machine.

Now here is the strange thing: when I run the old machine (converted from vmware) and the new one simultaneously, the cpu time is very low when both guests are idle. If I stop the new machine, the cpu time of the old one rises to a large value.

So initially the new VM wasn't high CPU, then after you started and stopped the second one, this new VM was high CPU? That seems odd.

[fzzzt]

I just discovered the latest snapshot of FreeBSD 7.0-STABLE (200807) has very low idle CPU usage. I tried it because I've been testing Hyper-V at work, and you need a recent snapshot for it to boot correctly, so I figured maybe those changes would affect VirtualBox also. One difference between this snapshot and 7.0-RELEASE is that it's default GENERIC kernel uses the SCHED_ULE scheduler rather than the SCHED_4BSD scheduler in 7.0-RELEASE. I recompiled with ULE but it didn't make a difference--still high CPU.

[Michael]

I just installed CentOS5 on two different boxes (without using the custom kernel mentioned previously) on Version 1.6.4.

Box 1 runs Windows Vista - Intel Core 2 Duo(T7250) 2GB RAM

• CentOS runs just fine. CPU usage is ~ 5%, nothing to complain here.

The same install source on Windows XP 3,5GB RAM, on a Pentium(R) 4 CPU 3.06Ghz (HT) gives me a 100% CPU usage at login prompt (Runlevel 3).

Other OS-Guest like WinXP, Ubuntu or Solaris are running just fine on this (older) box.

I haven't played around with CPU affinities yet, but patching my kernel with the kernel found at: ​http://vmware.xaox.net/centos/5.2/i386/kernel-2.6.18-92.1.10.el5.centos.plus.VMware.i686.rpm

solved my problem (CPU usage down to ~5%). However: I still don't understand why my Vista-Box doesn't have this problem!

Any updates on this (The bug report is for 1.5.2 - I used 1.6.4) ?

[Gabor Nagy]

I had one of my cores spinning with full speed continuously after I have converted my XP boxes from VMWare.

The solution was to boot the guest and go to Control Panel/System/Hardware/Device Manager/Computer, and change the system type to 'Advanced Configuration and Power Interface (ACPI) PC'.

The guest seemed to be much faster afterwords, and not using one core 100% all times.

As a side effect, I am able to disable IO APIC now.

[chandan thakur]

Hi All,

I am new to using virtualbox. I was experiencing the same problems of 30% to 40% CPU load. I have centos 5.0 as guest on windows XP machine. After reading all the comments above I could reduce finally the CPU load on my host machine(windows xp in this case to minimum). I did following things to do the same.

1. Must use kernel with 100Hzs timer rather than 1000Hz ticks. For this you can compile

"your kernel again or can take pre-compiled kernels as mentioned above : at ​http://vmware.xaox.net/centos/"

2. Install guest additions

3. Disable USB 2.0 controller (It might help in some cases)(do not ask me why).

4. Most important : increase your graphics memory to atleast 32 MB in settings for guest operating system. This helps much. Now I have less than 5% CPU utilization on centos 5.0 guest, winxp host + vb 2.02

I have not tried it on many machine. Bt this could solve problems for few.

Chandan Thakur

[Frank Mehnert]

Please reopen if still applicable with VirtualBox 2.2.4 or 3.0.0. Note that it is not necessary to disable the USB controller anymore with recent versions of VirtualBox.

[simoncpu]

Replying to frank:

Please reopen if still applicable with VirtualBox 2.2.4 or 3.0.0. Note that it is not necessary to disable the USB controller anymore with recent versions of VirtualBox.

Good day!

I'm currently using version 2.2.4 r47978. The uname for my host is:

Linux soulfury 2.6.28-14-generic #47-Ubuntu SMP Sat Jul 25 01:19:55 UTC 2009 x86_64 GNU/Linux

The uname for my guest is:

FreeBSD inara.simoncpu.com 7.1-RELEASE-p4 FreeBSD 7.1-RELEASE-p4 #0: Thu May 7 12:19:03 PHT 2009 admin@inara:/usr/obj/usr/src/sys/SOLIDWALL i386

top gives the following output even though both host and guest OSes are idle:

7053 simoncpu 20 0 575m 261m 27m R 32.2 13.1 6024:17 VirtualBox

VirtualBox is consuming ~30-35% CPU. I'm reopening this because this problem still exist.

Thanks!

[ simon.cpu ]

[Michael Thayer]

simoncpu: it is enough to comment on one ticket. I have pasted your comment onto #1946, as I have already added a comment of my own there.