Ticket #10600 (closed defect: fixed)
vbi_internal_alloc() should not call contig_alloc() for large allocations => Fixed in SVN
Reported by: | blakej | Owned by: | |
---|---|---|---|
Component: | other | Version: | VirtualBox 4.1.16 |
Keywords: | Cc: | artem.kachitchkine@… | |
Guest type: | all | Host type: | Solaris |
Description
A user of VirtualBox 4.1.16, with Solaris s11u1_16 as the host, commented that he was seeing "virtualbox's guru-meditation-screen-of-death due to not being able to allocate and lock memory [...] even though plenty of memory is available." The relevant comment from his logfile was:
PGMR3PhysAllocateLargePage: allocating large pages takes too long
(last attempt 8462 ms; nr of timeouts 1); DISABLE
I asked him to look into what VirtualBox was doing when it generated that error message. He found that it was in the vbox driver, making a call from vbi_internal_alloc() -> contig_alloc() -> page_create_io() -> delay().
Further investigation showed that vbi_internal_alloc() was being called with a size of 2MB, an alignment of 2MB, a high-address limitation of 264 - 1 (i.e. no limitation), and a request for contiguity. In other words, it just wanted a single large page.
Looking at the VBox 4.1.16 source, vbi_internal_alloc() calls contig_alloc() to get this page. As it turns out, contig_alloc() is just about the slowest imaginable way to allocate a single large page -- it assumes that it needs to respect the rest of the ddi_dma_attr_t, and it doesn't assume that it's nicely page-aligned, so it may end up trawling through huge amounts of the system's memory trying to stitch together a valid region for the caller.
If you would like information on a better approach for doing this, please contact me directly -- my email is blake.a.jones at oracle.com. (I am the technical lead for the VM2 project in Solaris.) I expect that a different approach might go 100x faster.
Attachments
Change History
Changed 11 years ago by blakej
-
attachment
vbi.c
added
draft implementation of large page allocation/free support for Solaris
comment:2 Changed 11 years ago by blakej
I've attached a sample implementation of how to allocate and free large pages using proper Solaris VM interfaces. I haven't tried compiling it, much less running it, but you're welcome to give it a try. Please feel free to contact me if you have problems or questions.
comment:3 Changed 11 years ago by blakej
For those in the Solaris group looking at this, I believe my fix should work both before and after the VM2 putback.
comment:5 Changed 11 years ago by ramshankar
Calling contig_alloc() for the large page (2M contiguous allocation via PGM/GMM) is a bug (see #if 0 in memobj-r0drv-solaris). We should call into vbi_large_page_alloc() and not vbi_phys_alloc()->contig_alloc().
We shouldn't have any code that requires > 2M contiguous physical memory.
comment:6 Changed 11 years ago by ramshankar
- Summary changed from vbi_internal_alloc() should not call contig_alloc() to vbi_internal_alloc() should not call contig_alloc() for large allocations
comment:7 Changed 11 years ago by ramshankar
Fixed in SVN and backported to 4.1. Fix should be available in 4.1.20.
Does this path still go through vbi_pages_alloc(), calling in to page_resv() with KM_NOSLEEP?