VirtualBox

Ticket #10600 (closed defect: fixed)

Opened 2 years ago

Last modified 20 months ago

vbi_internal_alloc() should not call contig_alloc() for large allocations => Fixed in SVN

Reported by: blakej Owned by:
Priority: major Component: other
Version: VirtualBox 4.1.16 Keywords:
Cc: artem.kachitchkine@… Guest type: all
Host type: Solaris

Description

A user of VirtualBox 4.1.16, with Solaris s11u1_16 as the host, commented that he was seeing "virtualbox's guru-meditation-screen-of-death due to not being able to allocate and lock memory [...] even though plenty of memory is available." The relevant comment from his logfile was:

PGMR3PhysAllocateLargePage: allocating large pages takes too long

(last attempt 8462 ms; nr of timeouts 1); DISABLE

I asked him to look into what VirtualBox was doing when it generated that error message. He found that it was in the vbox driver, making a call from vbi_internal_alloc() -> contig_alloc() -> page_create_io() -> delay().

Further investigation showed that vbi_internal_alloc() was being called with a size of 2MB, an alignment of 2MB, a high-address limitation of 264 - 1 (i.e. no limitation), and a request for contiguity. In other words, it just wanted a single large page.

Looking at the VBox 4.1.16 source, vbi_internal_alloc() calls contig_alloc() to get this page. As it turns out, contig_alloc() is just about the slowest imaginable way to allocate a single large page -- it assumes that it needs to respect the rest of the ddi_dma_attr_t, and it doesn't assume that it's nicely page-aligned, so it may end up trawling through huge amounts of the system's memory trying to stitch together a valid region for the caller.

If you would like information on a better approach for doing this, please contact me directly -- my email is blake.a.jones at oracle.com. (I am the technical lead for the VM2 project in Solaris.) I expect that a different approach might go 100x faster.

Attachments

vbi.c Download (2.0 KB) - added by blakej 2 years ago.
draft implementation of large page allocation/free support for Solaris

Change History

comment:1 Changed 2 years ago by jmcp

Does this path still go through vbi_pages_alloc(), calling in to page_resv() with KM_NOSLEEP?

Changed 2 years ago by blakej

draft implementation of large page allocation/free support for Solaris

comment:2 Changed 2 years ago by blakej

I've attached a sample implementation of how to allocate and free large pages using proper Solaris VM interfaces. I haven't tried compiling it, much less running it, but you're welcome to give it a try. Please feel free to contact me if you have problems or questions.

comment:3 Changed 2 years ago by blakej

For those in the Solaris group looking at this, I believe my fix should work both before and after the VM2 putback.

comment:4 Changed 2 years ago by ramshankar

I will have a look at this, thanks for the patch.

comment:5 Changed 2 years ago by ramshankar

Calling contig_alloc() for the large page (2M contiguous allocation via PGM/GMM) is a bug (see #if 0 in memobj-r0drv-solaris). We should call into vbi_large_page_alloc() and not vbi_phys_alloc()->contig_alloc().

We shouldn't have any code that requires > 2M contiguous physical memory. We have a few places where we need 12K below 4G and ~128K below 4G (per VM with single VCPU + 20K per extra VCPU) and currently have to resort using contig_alloc for those but for this case we shouldn't be going via contig_alloc.

Version 0, edited 2 years ago by ramshankar (next)

comment:6 Changed 2 years ago by ramshankar

  • Summary changed from vbi_internal_alloc() should not call contig_alloc() to vbi_internal_alloc() should not call contig_alloc() for large allocations

comment:7 Changed 22 months ago by ramshankar

Fixed in SVN and backported to 4.1. Fix should be available in 4.1.20.

comment:8 Changed 22 months ago by ramshankar

  • Summary changed from vbi_internal_alloc() should not call contig_alloc() for large allocations to vbi_internal_alloc() should not call contig_alloc() for large allocations => Fixed in SVN

comment:9 Changed 20 months ago by frank

  • Status changed from new to closed
  • Resolution set to fixed

4.1.20 was just released.

Note: See TracTickets for help on using tickets.

www.oracle.com
ContactPrivacy policyTerms of Use