High CPU load with MP Linux 4.10 guest

[Urmas]

When a Linux 32-bit guest is configured with 2 processors and uses the 4.10 kernel, it loads 100% a host processor core. It doesn't happen with kernel 4.9 and before. It doesn't happen with a single guest processor.

During boot the message smpboot: do_boot_cpu failed(-1) to wakeup CPU#1 appears.

[Urmas]

VM log

[Frank Mehnert]

And you are 100% sure that the guest is completely idle during this time?

[Urmas]

Yes. When the guest is active, it loads host up to 200%.

[Frank Mehnert]

Sorry, that was not my question. Your host has 2 physical CPU cores. So if the guest is busy then the host will be busy as well. You always need to consider a certain virtualization overhead, so if your guest is "quite of busy" then your host will be "very busy". Of course this effect is even worse if you run any application next to the VM.

You say that a Linux 4.9 guest was fine but after upgrading to 4.10 the host load is much higher. Did you only change the guest kernel or something else? Could you double check with the 4.9 kernel that the exact same guest configuration induces a notably smaller load on the host than the 4.10 kernel? In particular I would be interested in some numbers: Run 'top' in the guest for some time and check how idle the guest is (third line, the percent number before 'id'. That's the best indicator because it's the difference 100 - (summarized load of all processes).

Also I would like to see a guest kernel.log (please attach to this ticket) when the guest induces a host load which you consider as too high.

[Urmas]

For both kernels, idle is 99.3. Host load with 4.9 kernel is 0-1%, with 4.10 25-26%

[Frank Mehnert]

Thanks. Now I would also like to see a guest kernel log file for 4.10.

[Urmas]

kernel log

[Urmas]

So?

[Frank Mehnert]

Thanks again. Unfortunately I cannot spot anything unusual in the log files. We need to reproduce the problem, ATM we cannot...

It would be interesting to know which thread of the VirtualBox process (on the host) is responsible for the high CPU load. On a Linux host I would advise you to use the 'top' command and enable 'H' (thread view) together with 'V' (forest view). I don't know if a similar utility exists on a Windows host as well, perhaps the process explorer...

[Urmas]

I managed to fix it by removing the following snippet from the *.vbox file:

      <USB>
        <Controllers>
          <Controller name="OHCI" type="OHCI"/>
        </Controllers>
      </USB>

[Frank Mehnert]

So in other words if you disable the USB controller in the VM settings (no need to manually edit the .vbox file) then you observe a reduced CPU load with a 4.10 kernel. Interesting observation. We will try to reproduce the problem with this VM setting.

[Frank Mehnert]

Did some more tests, still not able to reproduce. I used two different guests, Ubuntu 16.10 with a 4.10 kernel installed from the kernel ppa and Ubuntu 17.04 which comes with Linux 4.10 by default. For boths guests the idle CPU load is about 5% with and without OHCI enabled.

However, I see that you are on a Windows 7 host. An increased CPU load could be very well due to this host operating system if the guest uses more than 1 VCPU. On Windows we use different hacks to get SMP working on certain systems. Maybe Linux 4.10 has changed some scheduling causing more context switches and this could indeed increase the VirtualBox CPU load on such a system.

[Frank Mehnert]

Still not being able to reproduce. Could attach the outpuf of

cat /proc/interrupts

from your guest running Linux 4.10 to this ticket?

Also I would still be interested in the information which thread(s) is/are responsible for the CPU load. So while the guest with Linux 4.10 is running and you observe the high CPU load on the host, start

top -d 10 -b -n 2 -H -p PID > top.log

on the host. PID is the PID of the current running VirtualBox VM process. Please also make sure to close the VM selector while your VM is running. The 'top' command will restrict the output only to the VM process and it will display all threads. It will run for about 10 seconds. Please attach the resulting top.log file to this ticket as well.

[Urmas]

Still present with vbox 5.2.16 and kernel 4.17.14

I don't have the "top" command on my system.

[Urmas]

/proc/interrupts

[Urmas]

The process hugging the host CPU is:

VirtualBox.exe 60eaff78-4bdd-042d-2e72-669728efd737-suplib-3rdchild --comment "..." --startvm 18821ac4-418f-4af8-8c0b-83026cc27851 --no-startvm-errormsgbox "--sup-hardening-log=...\Logs\VBoxHardening.log"