Solaris SMP problems

[Kevin Corby]

Hello all,

I am running VirtualBox 4.1.0 on a Solaris 10 host with the latest set of patches. The guest OS is Redhat 5.1. The hardware is a sunfire x4600 server with 8 pairs of CPUs for a total of 16. Hardware virtualization is available, enabled, and exclusive as far as I can tell, and all of the relevant settings seem to be correct (I/O APIC is enabled, cpuexecutioncap = 100).

I want to allow the guest OS (Redhat) to use 8 CPUs. I am able to configure the VM that way, and inside the guest OS it clearly believes 8 CPUs are available, and if I run 8 CPU-bound processes they will spread across them. However, this runs very slowly. If I look on the Solaris host machine, I can see that only a max of 12% of the CPU is ever getting used by the virtualbox process that is running the VM, even during its peak. This corresponds to 2 out of the 16 CPUs. The VM seems to run in one single process in Solaris, but that process has 57 threads (LWPs in Solaris-speak).

If I look at the CPU usage across the 16 processors in Solaris, I find that all 16 are 70%+ idle, which makes me think that the VirtualBox load seems to be getting shared across all processors, but it's clear that even with many more active threads than two I can only ever get up to 12% usage at a time. When I look at the CPU load caused by each LWP I can see a number of them that are active but stuck around 1.2% of CPU usage because of contention with each other.

Is there a reason the VM is not using more than 12% of the total CPU, like some kind of Solaris-imposed cap? Is there a way to remedy this?

Thanks, Kevin