VirtualBox

Ticket #11863 (closed defect: fixed)

Opened 10 months ago

Last modified 6 months ago

How to lock up the host machine by a guest using tracepath over virtio-net network interface

Reported by: Thomas Dreibholz Owned by:
Priority: blocker Component: network
Version: VirtualBox 4.2.12 Keywords: Denial of Service, Host DoS, kernel lock-up
Cc: Simone Ferlin-Oliveira <ferlin@…>, Amund Kvalbein <amundk@…> Guest type: Linux
Host type: Linux

Description

Hi,

I have discovered a problem with virtio-net that leads to a lockup of the host machine's kernel and the need for a hard reset to make it working again. It can be reproduced easily as follows:

  • The host system is a 64-bit Linux (tested with Ubuntu 12.04 LTS and Kubuntu 13.04). Did not try 32 bit.
  • VirtualBox is the latest version 4.2.12 (using Oracle's Ubuntu repository).
  • Create a new VM, use e.g. Kubuntu live CD image (32 or 64 bit, makes no difference). No disk needed.
  • Network adapter is: Bridged, Adapter Type: virtio-net.
  • Boot the system, ensure that network is working.
  • tracepath 8.8.8.8
  • Now, the virtual machine locks up and the host machine's kernel seems to have at least one core blocked. See the attached picture of the host machine's console output. The message is "BUG: soft lockup - CPU #2 stuck for 22s ...". Also, the network on the host machine does not work any more. For example, "ifconfig" just hangs.
  • To recover the host machine, it needs a hard reset. "sudo reboot", etc. will not work, since the kernel seems to hang.

This bug is critical, since it makes the host machine's network unusable (particularly, if the host system is at a remote location), and it is very easy to trigger with just a simple, standard "tracepath" call inside a virtual machine. It is therefore trivial for a normal user in such a machine to trigger a denial of service. I did no further investigation of the problem yet, but if it is related to the path MTU discovery by tracepath, it might be possible to trigger it by a lot of other software as well.

Best regards,

Thomas

Attachments

kernel-lockup.jpeg Download (476.1 KB) - added by Thomas Dreibholz 10 months ago.
A screenshot of the host's console output after triggering the bug
diff_intnet_4_2 Download (26.4 KB) - added by frank 6 months ago.
Patch for VBox 4.2.14 (included in VBox 4.2.16)
diff_nat_4_2 Download (2.9 KB) - added by frank 6 months ago.
Patch for VBox 4.2.16 (included in VBox 4.2.18)
diff_intnet_4_1 Download (25.5 KB) - added by frank 6 months ago.
Patch for VBox 4.1.26 (included in VBox 4.1.28)
diff_nat_4_1 Download (2.9 KB) - added by frank 6 months ago.
Patch for VBox 4.1.26 (included in VBox 4.1.28)

Change History

Changed 10 months ago by Thomas Dreibholz

A screenshot of the host's console output after triggering the bug

comment:1 Changed 10 months ago by frank

Thank you for this report. VBox 4.2.14 contains a workaround which prevents this problem. The real fix will be included in the next maintenance release.

comment:2 Changed 10 months ago by frank

  • Status changed from new to closed
  • Resolution set to fixed

Correct fix is included in 4.2.16 (released today).

comment:3 Changed 6 months ago by debfx

 http://www.oracle.com/technetwork/topics/security/cpuoct2013-1899837.html says versions prior to 4.2.18 are affected which contradicts what you've written here. Which information is correct?

comment:4 Changed 6 months ago by debfx

Version 4.1.18 has VNET_F_HOST_UFO disabled in vnetGetHostFeatures() which seems to be the workaround from 4.2.14. So I wonder if that version is affected.

comment:5 Changed 6 months ago by frank

Sorry for the confusion and the sparse documentation. We have to follow the Oracle policies and must not talk about security problems before they are fixed. And even then we must be careful.

VBox 4.2.16 contains the actual fix for virtio but if you compare 4.2.14 and 4.2.16 you will observe more differences. I will attach the diffs for 4.2 and 4.1 to this ticket.

VBox 4.2.18 contains another fix for the NAT backend. It turned out that the fix for NAT is correct but we were not able to trigger a situation where it was really required.

VBox 4.1.28 contains all these changes (all done after 4.1.26).

Changed 6 months ago by frank

Patch for VBox 4.2.14 (included in VBox 4.2.16)

Changed 6 months ago by frank

Patch for VBox 4.2.16 (included in VBox 4.2.18)

Changed 6 months ago by frank

Patch for VBox 4.1.26 (included in VBox 4.1.28)

Changed 6 months ago by frank

Patch for VBox 4.1.26 (included in VBox 4.1.28)

Note: See TracTickets for help on using tickets.

www.oracle.com
ContactPrivacy policyTerms of Use