[vbox-dev] Linux Host VM Cap?

Mon May 2 17:01:21 GMT 2011

On 01.05.2011 17:56, Knut St. Osmundsen wrote:
> Hi Zak!
>
> The VMM imposes a limit of 1023 VMs on 64-bit hosts and 127 VMs on
> 32-bit hosts. See:
> http://www.virtualbox.org/browser/trunk/src/VBox/VMM/VMMR0/GVMMR0.cpp#L121

That means your plan to have 1800 VMs will not work without bumping the 
hardcoded limits, and in this area it needs much more thinking than just 
changing a #define.

Anyway, let's get you to that limit first...

If you run "ipcs -ls" you'll get a "max number of arrays = 128" line, 
which is what you're most likely bumping into.

You can read the kernel defaults with "sysctl kernel.sem" as well, and 
the last number needs to be increased to a bit more than 1023, just to 
be on the safe side if some other app also needs a few semaphores.

"sysctl -w kernel.sem=250 32000 32 1200" should eliminate this limit. 
You can also put this in /etc/sysctl.conf, so that it is set straight on 
system boot, as otherwise these settings are lost on a reboot.

So... how many VMs can you start now?

Klaus

>
> Seeing you have 2TB of memory and 128 cores, I would assume you're
> running 64-bit Ubuntu. So, it is probably some other limit that you're
> hitting. It could be SysV semaphores, it could be memory available below
> 4GB, ... Could you please provide some more details regarding what
> happens when you are unable to launch more VMs? For instance, does a new
> VBox.log file get created for the VM? If it does, please zip it up and
> send it to me (or Klaus) per private mail and we'll work it out.

I think the 64bit host OS assumption is safe... I would seriously ask 
people to have their brain inspected if they'd use a 32bit OS for this.

>
>
> On Apr 29, 2011, at 7:43 PM, Zak Perschon wrote:
>
>> Ah, sorry forgot to include the original support thread. The limit I
>> hit was 128 active VMs at once. Once I hit that number, I was unable
>> to launch any more unless I stopped one that was already running. My
>> original thread can be found here (though there is little information
>> contained in
>> it):http://forums.virtualbox.org/viewtopic.php?f=7&t=40955
>> <http://forums.virtualbox.org/viewtopic.php?f=7&t=40955>
>>
>> -Zak
>>
>> > Date: Fri, 29 Apr 2011 16:25:44 +0200
>> > From:klaus.espenlaub at oracle.com <mailto:klaus.espenlaub at oracle.com>
>> > To:vbox-dev at virtualbox.org <mailto:vbox-dev at virtualbox.org>
>> > Subject: Re: [vbox-dev] Linux Host VM Cap?
>> >
>> > On 25.04.2011 18:00, Zak Perschon wrote:
>> > >
>> > > I've come across what is seemingly a hard cap for the amount of
>> > > concurrent VMs I can run on a Linux based machine (after posting
>> on the
>> > > support forums, I was directed here). Apparently this same cap was
>> > > encountered on a solaris host and a fix was deployed to increase the
>> > > amount of VMs that were allowed to run at the same time. Was this fix
>> > > supposed to be included in other OS releases? Any information on this
>> > > would be greatly appreciated.
>> >
>> > That's a rather vague statement, and if you could tell what number of
>> > concurrently running VMs you can achieve I might be able to infer what
>> > limit is causing trouble for you.
>> >
>> > It could be the SysV semaphore count, the number of open files per
>> > process, and so on. All these defaults depend on the kernel
>> > configuration and/or distribution, so it's hard to generalize.
>> >
>> > Solaris is easy compared to that, as it is very uniform compared to
>> Linux.
>> >
>> > Klaus
>> >
>> > >
>> > > Here is the information on the hardware/VM profiles I am using
>> > >
>> > > Host Machine:
>> > > 128 Processor Cores (64 dual core Xenon processors running at 2.27 GHz)
>> > > 2.0 TB Memory (1.8~ Usable)
>> > > 1.8 TB RAID 0 (6 x 300GB HDD running at 10k RPM)
>> > > 16TB SAS
>> > >
>> > > VM Profile
>> > > Compiled TinyCore Linux 2.6
>> > > Required Memory: 150~MB
>> > > Required HDD Space: 50MB
>> > >
>> > > -Zak
>> >
>> > _______________________________________________
>> > vbox-dev mailing list
>> >vbox-dev at virtualbox.org <mailto:vbox-dev at virtualbox.org>
>> >http://vbox.innotek.de/mailman/listinfo/vbox-dev
>> _______________________________________________
>> vbox-dev mailing list
>> vbox-dev at virtualbox.org <mailto:vbox-dev at virtualbox.org>
>> http://vbox.innotek.de/mailman/listinfo/vbox-dev
>
> --
>
> Kind regards / Mit freundlichen Gruessen / Vennlig hilsen,
> bird
>
> --
>
> ORACLE Deutschland B.V. & Co. KG Knut St. Osmundsen
> Werkstrasse 24 Senior Staff Engineer, VirtualBox
> 71384 Weinstadt, Germany mailto:bird at sun.com
>
> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen
> Registergericht: Amtsgericht Muenchen, HRA 95603
>
> Komplementaerin: ORACLE Deutschland Verwaltung B.V.
> Rijnzathe 6, 3454PV De Meern, Niederlande
> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697
> Geschaeftsfuehrer: J. Kunz, M. van de Molen, A. van der Ven
>
>
>
> _______________________________________________
> vbox-dev mailing list
> vbox-dev at virtualbox.org
> http://vbox.innotek.de/mailman/listinfo/vbox-dev

-- 
Oracle <http://www.oracle.com>
Dr. Klaus Espenlaub | Snr. Manager Software Development Desktop
Virtualization
Phone: +49 7151 60405 205 <tel:+49715160405205>
Oracle VM VirtualBox

ORACLE Deutschland B.V. & Co. KG | Werkstr. 24 | 71384 Weinstadt

ORACLE Deutschland B.V. & Co. KG
Hauptverwaltung: Riesstr. 25, D-80992 München
Registergericht: Amtsgericht München, HRA 95603

Komplementärin: ORACLE Deutschland Verwaltung B.V.
Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697
Geschäftsführer: Jürgen Kunz, Marcel van de Molen, Alexander van der Ven

Green Oracle <http://www.oracle.com/commitment> 	Oracle is committed to
developing practices and products that help protect the environment