vboxdrv runs kernel out of memory

[daryll]

It appears that performing a lot of network traffic with the host computer while a VirtualBox image is running can cause the kernel to run out of memory.

Host:
Running latest Fedora 12
kernel 2.6.32.10-90.fc12.x86_64

Guest:
Running Fedora 11
bridged networking

• Export a file system from the host computer
• Mount the file system on a non-guest computer
• Write a bunch of files to the file system from the non-guest

Eventually the host computer starts to crash as the slab fails to allocate memory. Repeated the test three times and reproduced the problem each time. Stopped running the virtual machine and ran the same test and it worked fine.

I'm attaching one of the oops' when this happened.

[daryll]

Kernel oops

[frank]

Any word about the traffic you generated between guest and host?

[frank]

I mean, is this TCP or UDP, any hint to reproduce? How long does it take for you to reproduce this host oops?

[daryll]

Replying to frank:

I mean, is this TCP or UDP, any hint to reproduce? How long does it take for you to reproduce this host oops?

Sorry I didn't see the follow up until now. I was writing files to the server file system using NFSv3. By default that's UDP as I recall.

[daryll]

Replying to daryll:

Replying to frank:

I mean, is this TCP or UDP, any hint to reproduce? How long does it take for you to reproduce this host oops?

Sorry I didn't see the follow up until now. I was writing files to the server file system using NFSv3. By default that's UDP as I recall.

Oh. And I can reproduce it pretty quickly (a few minutes), but I was writing a sequence of 12MB files, so it around a GB total.

[frank]

Do I understand you correct that the guest computer is not directly involved into the network traffic? That is, a remote host has a directory of the VBox host mounted and the remote host writes to that mounted directory and after some time, the VBox host crashes? And I assume this does not happen if you don't have a VM running at the VBox host, correct?

[daryll]

Replying to frank:

Do I understand you correct that the guest computer is not directly involved into the network traffic? That is, a remote host has a directory of the VBox host mounted and the remote host writes to that mounted directory and after some time, the VBox host crashes? And I assume this does not happen if you don't have a VM running at the VBox host, correct?

That is correct.

I boot the host. I fire up virtualbox on the host, copy the files between a 3rd system and the host, the host computer crashes. I reboot the system, I don't run virtualbox, copy the files between a 3rd system and host, and the host computer is fine.

[frank]

Cannot reproduce this problem so far, though using a Karmic host with 2.6.31. Which network card did you select for the VM (please attach the VBox.log file) and which physical NIC does your host have?

[daryll]

The server is running forcedeth. nVidia gigabit driver. VirtualBox is set to paravirtualized bridged eth0.

[yareckon]

I ran into this issue on Vbox OSE 3.2.0 r29652.

How to reproduce.

• Make a windows XP VM with 512MB guest memory.
• Share a folder with the virtual system.
• Create a test folder with 100,000 small text or image files using a random generator tool, or crawl the new york times website or something to get a lot of small files.
• Copy the folder with the 100,000 small files from guest system to the host system.
• Your host machine will extremely rapidly run out of memory. In a copy over the virtual network interface, after a few thousand files, my 4GB of RAM was full, and after a few thousand more, my 4GB of swap fills. The computer then crashes if I don't do anything.
• Pausing the execution of the VM during the copy, quitting virtualbox immediately releases the memory. Resuming the VM does not immediately grab the leaked memory back, instead as the copy resumes, the next couple thousand files incrementally eat up the host's memory.

This is on amd64.

[yareckon]

also, is this the same as #6918 ?

[yareckon]

OK, here is an update and some more clues.

I found the virtualbox PPA for ubuntu and installed 3.2.4 OSE. Rebooted host. Went to my virtual machine and installed 3.2.4 guest additions. Rebooted guest. Started file copy from scratch.

Here is a point I hadn't noticed before. For the first third of the copy, no problems, thousands of files go by, no change in memory allocation on the host system. Then, after about 10K files, the memory graph takes off like a rocket and 1000 files or so later my host's memory is exhausted. 1000 files after that, my swap is too.

The freeing of memory in the midst of the copy by saving state, closing virtualbox and resuming state works the same as described above under 3.2.0 OSE.

I'm gonna have to mount the virtual drive as a device on the host to get these files off, or moving them is gonna take 3 hours of manual nursing.

Again this is ubuntu linux amd64 guest.

[frank]

yareckon, please could you also check VBox 3.2.6 which is the recent release? This release fixed some memory leaks.

Furthermore, your problem is most like not related to this defect but rather to #6918. Anyway, please test VBox 3.2.6.

[faulebutter]

I experience the same problem with VB 3.2.8 PUEL on F13 host with heavy network load (VMDK on an NFS share and VRDP session with video running). The VM gets killed by kernel due to memory overallocation. (I've also posted this on forums:  http://forums.virtualbox.org/viewtopic.php?f=7&t=34702&p=155557#p155557).

[frank]

VBox 3.2.10 fixed some VRDP memory leaks, please retry. But this problem is also different from the original problem.

[frank]

Still relevant with VBox 4.0.4?

[lpleao]

I'm experiencing a similar problem with version 4.1.2. My host is running 64bit Ubuntu Lucid. This host has two NICs that are bridged together. One NIC connects to the LAN, the other one connects to an ESXi box. ESXi is using the host as a NFS based datastore. When I have a VM running (bridge mode) and generate some heavy NFS traffic on the ESXi box (installing Windows in a VM, for example), the host just freezes almost immediately. If I stop the VMs, I have no problem running the host as a NFS server. If I don't generate any NFS traffic, the host will run the VMs without any perceived issue (apart from the fact the windows VMs won't reboot for some reason...)

[lpleao]

After investigating this further, I found that the problem is the RTL-8111 card that was part of the bridge. System is stable now. It's not a VBox bug... time to investigate why VMs aren't rebooting...

Replying to lpleao:

[frank]

So let's close this bug. Create another ticket for different problems.