Changeset 5017

Timestamp:

09/24/07 23:38:20 (1 year ago)

Author:

vboxsync

Message:

GVM kick-off.

Files:

• trunk/include/VBox/gmm.h (added)
• trunk/src/VBox/VMM/PGM.cpp (modified) (13 diffs)
• trunk/src/VBox/VMM/PGMInternal.h (modified) (2 diffs)
• trunk/src/VBox/VMM/VMMR0/GMMR0.cpp (added)

trunk/src/VBox/VMM/PGM.cpp

@@ -108 +108 @@
-
-/** @page pg_pgmPhys PGMPhys - Physical Guest Memory Management.
- 
- 
+
+
- * Objectives:
- 
+
- *        zero pages and general page sharing.
- *      - Moving or mirroring a VM onto a different physical machine.
- *
- 
+
- * @subsection subsec_pgmPhys_Definitions       Definitions
- 
- 
- 
- 
- 
- 
- 
+
+
+
+
+
+
+
- * @subsection subsec_pgmPhys_AllocPage         Allocating a page.
- 
- 
+
+
- * means that none of the read functions will cause pages to be allocated.
- 
+
- * Exception, access bit in page tables that have been shared. This must
- 
+
- * unnecessary modifications.
- 
+
- * Allocation points:
- *      - PGMPhysWriteGCPhys and PGMPhysWrite.
@@ -139 +139 @@
- *      - ROM registration (currently MMR3RomRegister).
- *      - VM restore (pgmR3Load).
- 
- 
- 
- 
- 
- 
+
+
+
+
+
+
- * the number of non-zero pages in the saved state and allocate
- * them up front. This would allow the ring-0 component to refuse
- * the request if the isn't sufficient memory available for VM use.
- 
- 
+
+
- * zeroed pages as they are going to be filled instantly.
- *
- *
- * @subsection subsec_pgmPhys_FreePage          Freeing a page
- 
+
- * There are a few points where a page can be freed:
- *      - After being replaced by the zero page.
@@ -161 +161 @@
- *      - At reset.
- *      - At restore.
- 
+
- * When freeing one or more pages they will be returned to the ring-0
- * component and replaced by the zero page.
@@ -167 +167 @@
- * The reasoning for clearing out all the pages on reset is that it will
- * return us to the exact same state as on power on, and may thereby help
- 
+
- * (temporary) positive influence on memory fragmentation (@see subsec_pgmPhys_Fragmentation).
- 
- 
+
+
- * freed / allocated depending on how many is actually required by the
- * new VM state. The simplest approach is to do like on reset, and free
- * all non-ROM pages and then allocate what we need.
- 
+
- * A measure to prevent some fragmentation, would be to let each allocation
- * chunk have some affinity towards the VM having allocated the most pages
- * from it. Also, try make sure to allocate from allocation chunks that
- * are almost full. Admittedly, both these measures might work counter to
- 
+
- * cpu time or memory into this.
- 
- 
+
+
- * @subsection subsec_pgmPhys_SharePage         Sharing a page
- 
- 
- 
- 
+
+
+
+
- * them byte-by-byte to see if they actually are identical. If found to be
- 
+
- * the page didn't change while doing this, and finally request both the
- 
- 
+
+
- * to replace it with the zero page.
- 
+
- * To make this efficient, we will have to make sure not to try share a page
- * that will change its contents soon. This part requires the most work.
- 
+
- * a while to make sure it isn't modified any time soon. Also, it may
- * make sense to skip pages that are being write monitored since this
- 
+
- * per-VM guest memory structures (presently called PGMRAMRANGE).
- 
- 
+
+
- * @subsection subsec_pgmPhys_Fragmentation     Fragmentation Concerns and Counter Measures
- 
+
- * The pages are organized in allocation chunks in ring-0, this is a necessity
- 
+
- * could easily work on a page-by-page basis if we liked. Whether this is possible
- 
+
- * become a problem as part of the idea here is that we wish to return memory to
- 
- 
+
+
- * For instance, starting two VMs at the same time, they will both allocate the
- 
- 
- 
+
+
+
- * mixed up in the same allocation chunks.
- 
- 
+
+
- * ballooned or become shared/zero the whole idea is that the page is supposed
- * to be reused by another VM or returned to the host system. This will cause
- * allocation chunks to contain pages belonging to different VMs and prevent
- * returning memory to the host when one of those VM shuts down.
- 
- 
- 
+
+
+
- * that will be responsible for finding sharable/zero pages. The mechanisms
- 
+
- * the same as when telling it to share/zero a page.
- *
- 
+
- * @subsection subsec_pgmPhys_Tracking      Tracking Structures And Their Cost
- 
- 
- 
+
+
+
- * memory. We have limited virtual memory resources available when operating in
- 
+
- * tracking structures will be attemted designed such that we can deal with up
- * to 32GB of memory on a 32-bit system and essentially unlimited on 64-bit ones.
- 
- 
+
+
- * @subsubsection subsubsec_pgmPhys_Tracking_Kernel     Kernel Space
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
+
- *
- * @subsubsection subsubsec_pgmPhys_Tracking_PerVM      Per-VM
- 
- 
+
+
- * (described above). Theoretically we'll need 48(-12) bits for the HCPhys part.
- * Today we've restricting ourselves to 40(-12) bits because this is the current
- 
- 
+
+
- * tracking mappings of a page. 48-12 = 36. That leaves 28 bits, which means a
- * decent range for the page id: 2^(28+12) = 1024TB.
- 
- 
- 
+
+
+
- * for instance the ROM shouldn't depend on having a write handler installed
- * in order for it to become read-only. A RO/RW bit should be considered so
- 
+
- * flag combinations (ROM || RW handler || write monitored) in order to
- 
+
- * priority at present. Current this requires 12 bits, but could probably
- * be optimized to ~8.
- 
- 
- 
- 
+
+
+
+
- * purpose wouldn't hurt IIRC.
- 
+
- * Then there is a new bit in which we need to record what kind of page
- 
- 
+
+
- * write monitored page has been written to. And yet another one or
- * two for tracking migration status. 3-4 bits total then.
- 
+
- * Whatever is left will can be used to record the sharabilitiy of a
- * page. The page checksum will not be stored in the per-VM table as
- 
+
- * It will instead have to keep its own working set of potentially
- * shareable pages and their check sums and stuff.
- 
- 
+
+
- * PGMRAMRANGE::aHCPhys to keep the changes simple, only of course,
- 
+
- * our disposal.
- 
+
- * The initial layout will be like this:
- * @verbatim
-       
+
-        63:40                   Current shadow PT tracking stuff.
-        39:12                   The physical page frame number.
@@ -351 +300 @@
-    uint32_t u32Reserved;       Reserved for later, mostly sharing stats.
- @endverbatim
- 
+
- * The final layout will be something like this:
- * @verbatim
-       
+
-        63:48                   High page id (12+).
-        47:12                   The physical page frame number.
@@ -367 +316 @@
-    uint32_t u32Tracking;       The shadow PT tracking stuff, roughly.
- @endverbatim
- 
- 
+
+
- * around that I'm afraid. It means that the cost of dealing out 32GB of memory
- 
+
- * example, the VM heap cost when assigning 1GB to a VM will be: 4MB.
- 
- 
+
+
- * 32-bit Windows and 32-bit linux:
- *      1GB guest ram, 256K pages:  4MB +  2MB(+) =   6MB
@@ -382 +331 @@
- *      4GB guest ram, 1M pages:   16MB + 12MB(+) =  28MB
- *     32GB guest ram, 8M pages:  128MB + 96MB(+) = 224MB
- 
- 
+
+
- * @subsection subsec_pgmPhys_Serializing       Serializing Access
- 
+
- * Initially, we'll try a simple scheme:
- 
- 
+
+
- *        by the EMT thread of that VM while in the pgm critsect.
- *      - Other threads in the VM process that needs to make reliable use of
@@ -397 +346 @@
- *        data when performing it tasks as the EMT thread will be the one to
- *        do the actual changes later anyway. So, as long as it only accesses
- 
- 
- 
+
+
+
- *      - The over-commitment management, including the allocating/freeing
- *        chunks, is serialized by a ring-0 mutex lock (a fast one since the
- *        more mundane mutex implementation is broken on Linux).
- 
- 
- 
+
+
+
- *        one, but not the otherway around.
- 
- 
+
+
- * @subsection subsec_pgmPhys_Request           VM Request interface
- 
+
- * When in ring-0 it will become necessary to send requests to a VM so it can
- * for instance move a page while defragmenting during VM destroy. The idle
- 
+
- * pages and to perform write monitoring of pages.
- 
- 
- 
- 
+
+
+
+
- * realize.
- *
@@ -426 +375 @@
- *      -# Update all shadow page tables involved with the page.
- *
- 
+
- * physical handler, see pgmHandlerPhysicalSetRamFlagsAndFlushShadowPTs.
- 
- 
- 
+
+
+
- * @section sec_pgmPhys_MappingCaches   Mapping Caches
- 
- 
+
+
- * guest with more RAM than we've got virtual address space, we'll employing
- * a mapping cache. There is already a tiny one for GC (see PGMGCDynMapGCPageEx)
- * and we'll create a similar one for ring-0 unless we decide to setup a dedicate
- * memory context for the HWACCM execution.
- 
- 
+
+
- * @subsection subsec_pgmPhys_MappingCaches_R3  Ring-3
- 
+
- * We've considered implementing the ring-3 mapping cache page based but found
- 
- 
- 
+
+
+
- * worked out.
- 
+
- * Instead, we'll be mapping allocation chunks into the VM process. This simplifies
- 
+
- * only some minor RTR0MEMOBJ mapping stuff. The main concern here is that mapping
- * compared to the previous idea is that mapping or unmapping a 1MB chunk is more
- 
+
- * try address this by using a very big cache, preferably bigger than the actual
- 
- 
+
+
- * unlimited cache.
- *
- * The cache have to parts, as already indicated, the ring-3 side and the
- 
- 
- 
- 
+
+
+
+
- * discussed in @ref subsec_pgmPhys_Serializing. We
- 
+
- * VMMR0.r0 and RTR0MemObj will keep track of mapping relataions
- 
- 
- 
- 
+
+
+
+
- * for serializing changes to the mapping cache as this would reduce potential
- * contention with other threads accessing mappings unrelated to the changes
@@ -475 +424 @@
- * up in the statistics anyway, so it'll be simple to tell.
- *
- 
+
- * chunks are organized in ring-0, that is in an AVL tree by chunk id. To avoid
- * having to walk the tree all the time, we'll have a couple of lookaside entries
@@ -486 +435 @@
- *      -# Check the lookaside entries and then the AVL tree for the Chunk ID.
- *         If not found in cache:
- 
+
- *                 a chunk to be unmapped if the cache is maxed out already.
- 
+
- *      -# Update the relevant lookaside entry and return the mapping address.
- *      -# Do the read/write according to monitoring flags and everything.
- *      -# Leave the critsect.
- *
- 
+
- * @section sec_pgmPhys_Fallback            Fallback
- 
+
- * Current all the "second tier" hosts will not support the RTR0MemObjAllocPhysNC
- * API and thus require a fallback.
- 
+
- * So, when RTR0MemObjAllocPhysNC returns VERR_NOT_SUPPORTED the page allocator
- * will return to the ring-3 caller (and later ring-0) and asking it to seed
- * the page allocator with some fresh pages (VERR_GMM_SEED_ME). Ring-3 will
- 
+
- * "SeededAllocPages" call to ring-0.
- 
+
- * The first time ring-0 sees the VERR_NOT_SUPPORTED failure it will disable
- * all page sharing (zero page detection will continue). It will also force
- 
+
- * measures are taken to make sure that there will never be any need for
- * mapping anything into ring-3 - everything will be mapped already.
- *
- 
+
- * for this I don't quite know (I'd prefer not to and just ditch that all
- * togther), we'll see what's simplest to do.
- 
- 
- 
+
+
+
- * @section sec_pgmPhys_Changes             Changes
- 
+
- * Breakdown of the changes involved?
- */
@@ -925 +874 @@
-        return rc;
-
- 
+
-     * Initialize the PGM critical section and flush the phys TLBs
-     */
@@ -2046 +1995 @@
-                    pRam->GCPhys, pRam->GCPhysLast, pRam->cb, pRam->pvHC ? "bits" : "nobits",
-                    GCPhys, GCPhysLast, cb, fHaveBits ? "bits" : "nobits"));
- 
- 
+
+
-             * the MMIO[2] and ROM stuff isn't 100% right, just skip the mismatches.
-             */

trunk/src/VBox/VMM/PGMInternal.h

@@ -31 +31 @@
-#include <VBox/dbgf.h>
-#include <VBox/log.h>
+#include <VBox/gmm.h>
-#include <iprt/avl.h>
-#include <iprt/assert.h>
@@ -550 +551 @@
-                                        do { (pPage)->HCPhys = (((pPage)->HCPhys) & UINT64_C(0xffff000000000fff)) \
-                                                             | ((_HCPhys) & UINT64_C(0x0000fffffffff000)); } while (0)
-
-/** The chunk shift. (2^20 = 1 MB) */
-#define GMM_CHUNK_SHIFT                 20
-/** The allocation chunk size. */
-#define GMM_CHUNK_SIZE                  (1U << GMM_CHUNK_SHIFT)
-/** The shift factor for converting a page id into a chunk id. */
-#define GMM_CHUNKID_SHIFT               (GMM_CHUNK_SHIFT - PAGE_SHIFT)
-/** The NIL Chunk ID value. */
-#define NIL_GMM_CHUNKID                 0
-/** The NIL Page ID value. */
-#define NIL_GMM_PAGEID                  0
-
-/**