[vbox-dev] Any plan to support SR-IOV?

[Brian Johnson]

Fred Liu <Fred_Liu at ...> writes:

> 
> Thanks for the clarification.
> Intel 82576 nic has been released for more one year, it will be easier and 
easier to find.
> I/O performance bottleneck is really the barrier to deploy virtualization  
confidently.
> KVM and XEN is sort of ahead of VBOX in this aspect.
> 
> Thanks.
> 
> Fred
> 
> -----Original Message-----
> From: Alexey Eromenko [mailto:al4321 at ...] 
> Sent: Thursday, March 03, 2011 10:54 PM
> To: Fred Liu
> Cc: vbox-dev at ...
> Subject: Re: [vbox-dev] Any plan to support SR-IOV?
> 
> On Thu, Mar 3, 2011 at 4:42 PM, Fred Liu <Fred_Liu at ...> wrote:
> >
> >
> > -----Original Message-----
> > From: Fred Liu
> > Sent: Thursday, March 03, 2011 1:53 PM
> > To: vbox-dev at ...
> > Subject: Any plan to support SR-IOV?
> >
> 
> SR-IOV is a host technology, not a hypervisor one.
> 
> What hypervisor should support is VT-d, then SR-IOV gets supported
> automatically.
> 
> There were few commits in the VBox source tree with regards to VT-d
> support, but it looks like very early status.
> In Addition, Hardware is too scarse to test it for community members to help.
> 

Fred,

I should start off by stating that I work at Intel in the LAN Access Division. 
I am responsible for 10Gb Ethernet and Virtualization technologies like SR-IOV 
on Intel(R) Ethernet Controllers. 

SR-IOV-capable devices provide configurable numbers of independent VFs, each 
with its own PCI Configuration space. The VMM assigns one or more VF to a 
virtual machine. Memory Translation technologies such as those in Intel® VT-x 
and Intel® VT-d provide hardware assisted techniques to allow direct DMA 
transfers to and from the VM, thus bypassing the software switch in the VMM.

The Intel(R) 82599 Ethernet Controllers and Intel(R) 82576 Ethernet 
Controllers support SR-IOV and can be found on a wide selection of adapters 
and as LOMs on many server boards. The 82576 is our 1Gb part and provides up 
to 8 VFs and 82599 is our 10Gb part with up to 64 VFs.

To use SR-IOV you need to have VT-d enabled in the server BIOS like the 
previous post stated BUT you also need to have an SR-IOV enabled BIOS. Some 
server support VT-d but do not support SR-IOV because the BIOS does not setup 
the PCI config space for the VFs. Several server OEMs offer systems that have 
SR-IOV enabled, such as the Dell R710 and R910 and most of the current Intel 
Server Boards and Systems. I know that other systems have been enabled with 
the SR-IOV BOIS so please check your server model and BIOS version before 
trying to setup SR-IOV.

Ecosystem Requirements taken from Patrick Kutch's PCI-SIG SR-IOV Primer 
Revision 2.5 -- http://download.intel.com/design/network/applnots/321211.pdf 

BIOS
The BIOS performs a role in partitioning Memory Mapped I/O and PCI Express Bus 
numbers between host bridges.
In many systems, mechanisms for allocating PCI resources to the host bridges 
are not standardized and software relies on the BIOS to configure these 
devices with sufficient memory space and bus ranges to support I/O devices in 
the hierarchy beneath each host bridge.
BIOS enumeration code needs to be enhanced to recognize SR-IOV devices so that 
enough MMIO (Memory Mapped IO) space is allocated to encompass the 
requirements of VFs. Refer to the PCI-SIG SR-IOV specification for details on 
how to parse PCI Config space and calculate the maximum amount of VF MMIO 
space required.

Here are some other links that a Patrick and I have put together.

Here is a link to a Video that Patrick Kutch did explaining how SR-IOV works – 
http://www.youtube.com/watch?v=hRHsk8Nycdg

Here is a link to more technical resources that Patrick has regarding SR-IOV.
http://communities.intel.com/people/Patrick_Kutch?view=overview  

Thanks,
Brian Johnson
Intel Corp