[vbox-dev] Linux Host VM Cap?
klaus.espenlaub at oracle.com
Mon May 2 10:01:21 PDT 2011
On 01.05.2011 17:56, Knut St. Osmundsen wrote:
> Hi Zak!
> The VMM imposes a limit of 1023 VMs on 64-bit hosts and 127 VMs on
> 32-bit hosts. See:
That means your plan to have 1800 VMs will not work without bumping the
hardcoded limits, and in this area it needs much more thinking than just
changing a #define.
Anyway, let's get you to that limit first...
If you run "ipcs -ls" you'll get a "max number of arrays = 128" line,
which is what you're most likely bumping into.
You can read the kernel defaults with "sysctl kernel.sem" as well, and
the last number needs to be increased to a bit more than 1023, just to
be on the safe side if some other app also needs a few semaphores.
"sysctl -w kernel.sem=250 32000 32 1200" should eliminate this limit.
You can also put this in /etc/sysctl.conf, so that it is set straight on
system boot, as otherwise these settings are lost on a reboot.
So... how many VMs can you start now?
> Seeing you have 2TB of memory and 128 cores, I would assume you're
> running 64-bit Ubuntu. So, it is probably some other limit that you're
> hitting. It could be SysV semaphores, it could be memory available below
> 4GB, ... Could you please provide some more details regarding what
> happens when you are unable to launch more VMs? For instance, does a new
> VBox.log file get created for the VM? If it does, please zip it up and
> send it to me (or Klaus) per private mail and we'll work it out.
I think the 64bit host OS assumption is safe... I would seriously ask
people to have their brain inspected if they'd use a 32bit OS for this.
> On Apr 29, 2011, at 7:43 PM, Zak Perschon wrote:
>> Ah, sorry forgot to include the original support thread. The limit I
>> hit was 128 active VMs at once. Once I hit that number, I was unable
>> to launch any more unless I stopped one that was already running. My
>> original thread can be found here (though there is little information
>> contained in
>> > Date: Fri, 29 Apr 2011 16:25:44 +0200
>> > From:klaus.espenlaub at oracle.com <mailto:klaus.espenlaub at oracle.com>
>> > To:vbox-dev at virtualbox.org <mailto:vbox-dev at virtualbox.org>
>> > Subject: Re: [vbox-dev] Linux Host VM Cap?
>> > On 25.04.2011 18:00, Zak Perschon wrote:
>> > >
>> > > I've come across what is seemingly a hard cap for the amount of
>> > > concurrent VMs I can run on a Linux based machine (after posting
>> on the
>> > > support forums, I was directed here). Apparently this same cap was
>> > > encountered on a solaris host and a fix was deployed to increase the
>> > > amount of VMs that were allowed to run at the same time. Was this fix
>> > > supposed to be included in other OS releases? Any information on this
>> > > would be greatly appreciated.
>> > That's a rather vague statement, and if you could tell what number of
>> > concurrently running VMs you can achieve I might be able to infer what
>> > limit is causing trouble for you.
>> > It could be the SysV semaphore count, the number of open files per
>> > process, and so on. All these defaults depend on the kernel
>> > configuration and/or distribution, so it's hard to generalize.
>> > Solaris is easy compared to that, as it is very uniform compared to
>> > Klaus
>> > >
>> > > Here is the information on the hardware/VM profiles I am using
>> > >
>> > > Host Machine:
>> > > 128 Processor Cores (64 dual core Xenon processors running at 2.27 GHz)
>> > > 2.0 TB Memory (1.8~ Usable)
>> > > 1.8 TB RAID 0 (6 x 300GB HDD running at 10k RPM)
>> > > 16TB SAS
>> > >
>> > > VM Profile
>> > > Compiled TinyCore Linux 2.6
>> > > Required Memory: 150~MB
>> > > Required HDD Space: 50MB
>> > >
>> > > -Zak
>> > _______________________________________________
>> > vbox-dev mailing list
>> >vbox-dev at virtualbox.org <mailto:vbox-dev at virtualbox.org>
>> vbox-dev mailing list
>> vbox-dev at virtualbox.org <mailto:vbox-dev at virtualbox.org>
> Kind regards / Mit freundlichen Gruessen / Vennlig hilsen,
> ORACLE Deutschland B.V. & Co. KG Knut St. Osmundsen
> Werkstrasse 24 Senior Staff Engineer, VirtualBox
> 71384 Weinstadt, Germany mailto:bird at sun.com
> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen
> Registergericht: Amtsgericht Muenchen, HRA 95603
> Komplementaerin: ORACLE Deutschland Verwaltung B.V.
> Rijnzathe 6, 3454PV De Meern, Niederlande
> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697
> Geschaeftsfuehrer: J. Kunz, M. van de Molen, A. van der Ven
> vbox-dev mailing list
> vbox-dev at virtualbox.org
Dr. Klaus Espenlaub | Snr. Manager Software Development Desktop
Phone: +49 7151 60405 205 <tel:+49715160405205>
Oracle VM VirtualBox
ORACLE Deutschland B.V. & Co. KG | Werkstr. 24 | 71384 Weinstadt
ORACLE Deutschland B.V. & Co. KG
Hauptverwaltung: Riesstr. 25, D-80992 München
Registergericht: Amtsgericht München, HRA 95603
Komplementärin: ORACLE Deutschland Verwaltung B.V.
Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697
Geschäftsführer: Jürgen Kunz, Marcel van de Molen, Alexander van der Ven
Green Oracle <http://www.oracle.com/commitment> Oracle is committed to
developing practices and products that help protect the environment
More information about the vbox-dev