HostMemoryLow VM error for 3.5GB OpenSolaris guest on MacOS X

[Mason Lee]

My OpenSolaris (64) guest goes into suspended state after running for several hours, seemingly due to some memory issues. The guest has 3584MB ram assigned, and is not using nearly all of that, as you can see below. Its memory usage seems stable over time. The host system, Mac OS X 10.6.2 running VBox 3.1 (btw, same problem on 3.0.10) has plenty of available RAM, and its memory usage seems stable as well. This happens to the guest running in both headless and non-headless mode.

In the guest, memory seems to be relatively stable over time:

$ top
Memory: 3584M phys mem, 1865M free mem, 512M total swap, 512M free swap

$ isainfo
amd64 i386
(Hmm. OpenSolaris not running in 64 bit mode?)

VirtualBox Log shows the following error:

03:59:48.183 PGM: Failed to procure handy pages; rc=VERR_NO_MEMORY rcAlloc=VERR_NO_MEMORY rcSeed=VINF_SUCCESS cHandyPages=0x8
03:59:48.183 cAllPages=0xe106f cPrivatePages=0xc28c1 cSharedPages=0x0 cZeroPages=0x1e7ae
03:59:48.183 PGM: Failed to procure handy pages; rc=VERR_NO_MEMORY rcAlloc=VERR_NO_MEMORY rcSeed=VINF_SUCCESS cHandyPages=0x7
03:59:48.183 cAllPages=0xe106f cPrivatePages=0xc28c2 cSharedPages=0x0 cZeroPages=0x1e7ad
03:59:48.183 VM: Raising runtime error 'HostMemoryLow' (fFlags=0x2)
03:59:48.183 Changing the VM state from 'RUNNING' to 'SUSPENDING'.
03:59:48.184 Changing the VM state from 'SUSPENDING' to 'SUSPENDED'.
03:59:48.185 ERROR [COM]: aRC=NS_ERROR_INVALID_ARG (0x80070057) aIID={e2a38ebc-d854-4a3e-bc2e-fdf5ac4a0000} aComponent={Display} aText={Argument address is NULL} aWarning=false, preserve=true
03:59:48.185 Console: VM runtime error: fatal=false, errorID=HostMemoryLow message="Unable to allocate and lock memory. The virtual machine will be paused. Please close applications to free up memory or close the VM"

If I resume the machine, the same error is repeated and the machine goes into aborted state.

Ideas or advice as to how to further isolate the problem? Thanks!