VirtualBox

Opened 4 years ago

Last modified 4 months ago

#19726 new defect

Reported memory usage doubled in 6.1.4 on MacOS Mojave and later

Reported by: arthens Owned by:
Component: other Version: VirtualBox 6.1.10
Keywords: catalina memory Cc:
Guest type: other Host type: Mac OS X

Description

Hi there,

we recently noticed that memory usage doubled on Catalina (confirmed on 10.15.5). Upon investigating the problem we figured out that this issue began with 6.1.4. The problem is still present in 6.1.10.

Up until 6.1.2 memory and real memory were basically the same. Allocating 4GB in the VM resulted in a memory usage of 4GB.

From 6.1.4 onwards, memory is always twice as large as real memory:

  • allocating 4GB results in 8GB being used
  • allocating 8GB results in 16GB being used

and so on (see attached images)

Reverting back to 6.1.2 fixes the problem. We couldn't find a bug report about it, but we did find a few reports on the forums:

It's possible this bug might specific to Catalina, as I personally only noticed running out on memory after upgrading from Mojave. I think I was already on 6.1.4, but I don't have a Mojave machine so I can't test this theory.

Everyone in our development team had to install 6.1.2 because we generally allocate 8GB to the virtual machine, which ended up using 16GB. As most of us have 16GB of ram, this made our machines unusable.

Please let me know if you need further information

Attachments (6)

6.1.2 - 4GB allocated.png (20.9 KB ) - added by arthens 4 years ago.
6.1.4 - 4GB allocated.png (38.4 KB ) - added by arthens 4 years ago.
6.1.10 - 4GB allocated.png (32.5 KB ) - added by arthens 4 years ago.
6.1.10 - 8GB allocated.png (21.4 KB ) - added by arthens 4 years ago.
Win10Dev-2020-07-22-14-07-19.log.zip (39.3 KB ) - added by ncj 4 years ago.
Additional log.
osearbhain-1.jpg (86.0 KB ) - added by osearbhain 4 years ago.

Download all attachments as: .zip

Change History (62)

by arthens, 4 years ago

Attachment: 6.1.2 - 4GB allocated.png added

by arthens, 4 years ago

Attachment: 6.1.4 - 4GB allocated.png added

by arthens, 4 years ago

Attachment: 6.1.10 - 4GB allocated.png added

by arthens, 4 years ago

Attachment: 6.1.10 - 8GB allocated.png added

comment:1 by sygibson, 4 years ago

Confirmed on my side. Since 6.1.3 and through 6.1.12 - any VM assigned memory will end up with actual consumption on host (Mac OS X 10.14.6 Mojave) consuming just over 2x the amount of memory.

Downloading and installing the 6.1.2 version, and restarting the VM returns to just a touch over 1x memory usage.

comment:2 by paulson, 4 years ago

Status: newawaitsfeedback
Summary: Memory usage doubled in 6.1.4 on CatalinaReported memory usage doubled in 6.1.4 on MacOS Mojave and later

There were some low-level virtual memory management changes made during VirtualBox 6.1.4 development which now trigger a bug in the MacOS memory usage reporting which happens on Mojave and later. The Docker developers ran into this as well and created a nice writeup of the issue here:

https://docs.google.com/document/d/17ZiQC1Tp9iH320K-uqVLyiJmk4DHJ3c4zgQetJiKYQM/edit

In short, the memory usage of your VirtualBox virtual machine hasn't changed as you can confirm with the MacOS Activity Monitor's "Real Mem" field, the "RSS" field of ps(1) (e.g. $ ps -x -o pid,rss,vsz,command), or the output from 'vmmap -summary <pid>'.

Is this really causing problems for you on your system(s)? How exactly did the machines become "unusable"?

comment:3 by vboxgpz, 4 years ago

How exactly did the machines become "unusable"?

Well consuming 2x the virtual memory added a bunch of "memory pressure" (mac os terminology), which resulted in all other apps being starved for memory. My browsers, outlook etc all slowed to a crawl, and if I try to do any heavy compiling, the system even locks up periodically.

comment:4 by arthens, 4 years ago

Is this really causing problems for you on your system(s)? How exactly did the machines become "unusable"?

It's a significant problem. Before realising I could fix the bug by downgrading to 6.1.2 I had to stop using my editor of choice (Intellij) in favour of an editor with lower memory requirements (vscode) because the excessive memory pressure meant my mac was extremely slow. Even trivial actions like opening a new file or switching tab had a noticeable delay.

I've read about the difference between memory and real memory before opening this ticket, but I'm finding it hard to reconcile it with the fact that the slowness was noticeable and went away after downgrading (I'm back using intellij now)

by ncj, 4 years ago

Additional log.

comment:5 by ncj, 4 years ago

I have attached another log with the "double RAM" use. Host macOS 10.15.6, VirtualBox 6.1.12, guest Win10 x64, 3GB RAM (consumes over 6G).

Version 0, edited 4 years ago by ncj (next)

comment:6 by paulson, 4 years ago

Thanks for the additional feedback. In addition to the double-counting memory reporting MacOS bug being triggered it appears that there is some aspect of the virtual memory changes made in 6.1.4 development which is leading to additional memory usage on Mac hosts. Further investigation is required.

comment:7 by sujithrs, 4 years ago

I see the same issue on my Mac as well (Catalina 10.15.6, VirtualBox 6.1.12 r139181). Because of this issue, I am unable to run multiple VM's like I used to before.

One thing I noticed was the "Real memory" usage of kernel_task process seems to increase along with VirtualBox VM process.

With no VirtualBox VM running:

                 Real Mem       Memory
                 --------       ------
kernel_task      2.77 GB        84.2 MB

After starting a saved VM (Ubuntu 20.4 with 4 GB RAM):

                 Real Mem       Memory
                 --------       ------
VirtualBox VM    3.42 GB        6.44 GB
kernel_task      5.87 GB        88.0 MB

After running few memory intensive applications inside the VM:

                 Real Mem       Memory
                 --------       ------
VirtualBox VM    4.30 GB        8.23 GB
kernel_task      6.79 GB        88.0 MB

After closing (saving) VM:

                 Real Mem       Memory
                 --------       ------
kernel_task      2.68 GB        86.1 MB

And finally reopening the saved VM:

                 Real Mem       Memory
                 --------       ------
VirtualBox VM    4.28 GB        8.22 GB
kernel_task      6.76 GB        88.1 MB

PS: "Real Mem" and "Memory" as reported by "Activity Monitor"

comment:8 by Rubex, 4 years ago

I faced the same issue in Mac Pro (Catalina 10.15.6, VirtualBox 6.1.12 r139181) having 8GB RAM. After reverting to Virtual Box 6.1.2 and corresponding Guest Addition, kernel task is not showing 6GB, but still shows 3GB - resulting in sluggish performance of Mac.

Without VM —————-

kernel_task : Memory 143.4MB Real Memory: 2.53GB

After starting VM ( Windows 7 with allocated RAM size of 4GB) ————————————————————————————-

Virtual Box VM: Memory : 8.3GB Real Memory: 4.31GB kernel_task : Memory : 142.8MB Real Memory: 6.36GB

With heavy application running in guest VM ———————————————————-

Virtual Box VM: Memory : 8.51GB Real Memory: 4.21GB kernel_task : Memory : 162.3MB Real Memory: 6.66GB

comment:9 by vboxgpz, 4 years ago

I wonder if the kernel_task issue is the one addressed in the 10.15.6 supplemental update that was released yesterday.

comment:10 by Asif_A, 4 years ago

I had the same issue with MacOS Catalina 10.15.6 and VBox 6.1.12. Guest OS is Windows 10. Memory assigned to VM as 6 GB and memory taken up by VBox process was > 12 GB. If I left the VM running overnight, my Mac would be totally unresponsive or crashed by the morning. Reverting to 6.1.2 has brought down the memory usage and hopefully will remove the instability. Fingers crossed and waiting for the fix.

Last edited 4 years ago by Asif_A (previous) (diff)

comment:11 by osearbhain, 4 years ago

I still have this issue with VirtualBox 6.1.12 on MacOS 10.15.6 (19G2021) (i.e. with the 10.15.6 supplemental update installed). I have a VM configured to use 4GB RAM. Activity Monitor says it's currently using 6.86GB Memory (3.60 GB Real Mem), and kernel_task is ~86 MB Memory (but 5.50 GB Real Mem). See attachment osearbhain-1.jpg.

by osearbhain, 4 years ago

Attachment: osearbhain-1.jpg added

comment:12 by sujithrs, 4 years ago

Finally have a stable system with VirtualBox 6.1.2 r135662 and MacOS 10.15.6 Supplemental update.

https://support.apple.com/kb/DL2049

Based on @osearbhain's observation, I shall stick with this setup.

comment:13 by elupus, 4 years ago

Just a small me too. @sujithrs is that osx patch needed for it to work property with 6.1.2?

comment:14 by arthens, 4 years ago

AFAIK you need that patch regardless what version of virtualbox you use. It fixes a (different) virtualisation bug that would cause memory usage to continually increase to the point that you mac would crash.

This bug never makes your mac crash, it just makes it slow.

comment:15 by BStopp, 4 years ago

Confirmed issue on MacOS 10.14.6 and VirtualBox 6.1.14 r140239 (Qt5.6.3) and Version 6.1.16 r140961 (Qt5.6.3)

Downgrading really isn't a solution. Is there any way to get some acknowledgement that the dev team recognizes and has added the issue to the backlog?

Last edited 4 years ago by BStopp (previous) (diff)

comment:16 by dangelovich, 4 years ago

Status: awaitsfeedbacknew

Confirmed on 10.15.7 with 6.1.14 and 6.1.16

comment:17 by nixorg, 4 years ago

Earlier on Catalina OS I was able to install old version 6.0.24 and ram consumption was ok but now after upgrading to Big Sur I can install only last version 6.1.16 and there is issue with x2 RAM consumption. Could you please prioritise this issue?

comment:18 by Ernesto Baschny, 4 years ago

In our team we can confirm the issue with latest MacOS (Catalina and Big Sur):

Since VBox 6.1.4 the memory consumption for any Linux VM (in our case a docker-machine, boot2docker based) is doubled. Downgrading to 6.1.2 solves the issue, but unfortunately this is not an option on Big Sur, since 6.1.16 is required.

It would be very cool if it could be addressed on the next releases.

comment:19 by Yauheni, 4 years ago

Is there any ETA for this? Having Big Sur on board makes impossible using of VirtualBox. That's more than critical.

Last edited 4 years ago by Yauheni (previous) (diff)

comment:20 by Klaus Espenlaub, 4 years ago

We're not working on anything since we don't believe that there is an actual doubling of the memory consumption.

I checked what we changed, and it's not making any sense that it causes more than double accounting: https://www.virtualbox.org/log/vbox/trunk/src/VBox/Runtime/r0drv/darwin (the relevant changes between 6.1.2 and 6.1.4 are r83070 to r83098). The changes were necessary to resolve macOS misbehavior when the macOS hardened runtime was enabled (which is a hard requirement for getting anything through the tightened notarization rules which became effective around that time).

comment:21 by arthens, 4 years ago

Klaus are you suggesting the double memory usage is just a visual bug? Or are you skeptical that it's happening at all? Either way I'd be happy to demonstrate the problem on a remote session, and/or provide additional information. Feel free to email me at [username]@gmail.com

comment:22 by bbyingto, 4 years ago

I'm really a bit confused why this issue could be at risk of dismissal when so many different users have verified being affected. I've only lurked this issue so far because I didn't have an Oracle account, and it looked like there were enough reproducing users that I didn't need to bother making one. You can add me to the pile now though, there is simply no way this isn't a real issue. My primary usage for my work macbook is to run a Centos VM, so that VM gets dedicated the majority of my CPU and Memory resources. When I run 6.1.2 I have a stable environment with no issues. When I run a newer version it's only a matter of time until my host system starts paging like mad and everything locks up until I manage to kill off the VM. There is no way this is not a real issue.

I'm less savvy than others when it comes to producing VirtualBox diagnostic information, but if anyone wants to tell me what is required then I too am happy to produce whatever proof would be useful.

comment:23 by tcarlos, 4 years ago

This is a problem for anyone using VirtualBox since 6.1.6. There is no working Virtualbox Installation since 6.1.6 without this extremely annoying bug. This bug makes VirtualBox on actual Macs completely unusable. There is not one working installation of VirtualBox on MacOS with Big Sur. I will now move on to another solution of an other vendor, since Oracle seems not capable to solve this obvious major problem for months. VirtualBox on MacOS is dead now.

Last edited 4 years ago by tcarlos (previous) (diff)

comment:24 by FrankErensAndrome, 4 years ago

Can add another confirmation, if that matters. On MacOS 10.14.6, VirtualBox 6.1.16 (latest at the time of writing), a VM with 2 GB memory allocated reports usage between 3 and 4 GB. Five such VMs on a MacBook Pro with 16 GB of RAM causes the system to crawl to a halt.

Issue is resolved completely by downgrading to VirtualBox 6.1.2.

comment:25 by xyu, 4 years ago

I have the same observation for the past year as many other reported. Compared with 6.1.2, the double memory issue is definitely not just the number reported but the actual resource occupied because the system was sluggish when VM is running in any version after 6.1.2 while it is running fine with 6.1.2. I am still on Catalina right now and can still use the old version. But after Big Sur upgrade it won't be usable with all the other apps running on my host.

I guess one could reproduce by creating a few VMs in parallel with total VMs' ram a bit below host's ram and run ubuntu CD at the same time and compare 6.1.2 with 6.1.4 experience.

comment:26 by Rnnr, 4 years ago

I have Virtualbox 6.1.18 installed on macOS Catalina (10.15.7) and got hit by the bug too.

comment:27 by CSantz, 4 years ago

Same problem here with Virtualbox 6.1.18 on MacOS Big Sur (11.2.2).

Windows10 VM with 3.4 GB and Virtualbox using 7.35 GB.

After start VM:

PIDCOMMAND %CPU TIME #TH #WQ #PORT MEM PURG CMPRSPGRP PPID
637VirtualBoxVM 38.5 05:11.09 32 1 301 7355M 159M 69M- 637 623
841 VirtualBox 0.0 00:14.58 9 1 261 117M 0B 103M 841 1

Sadly I can't downgrade to 6.1.2 to check if works.

I have been notice my Mac is running very slow for a few months(~1y ago), only now I realize its running off memory because VirtualBox.

The problem occurs with any Guest OS, my Windows and Linux VM all double memory usage.

vmmap https://pastebin.com/raw/JJPPrpDj

Last edited 4 years ago by CSantz (previous) (diff)

in reply to:  20 comment:28 by CSantz, 4 years ago

Replying to klaus:

We're not working on anything since we don't believe that there is an actual doubling of the memory consumption.

I checked what we changed, and it's not making any sense that it causes more than double accounting: https://www.virtualbox.org/log/vbox/trunk/src/VBox/Runtime/r0drv/darwin (the relevant changes between 6.1.2 and 6.1.4 are r83070 to r83098). The changes were necessary to resolve macOS misbehavior when the macOS hardened runtime was enabled (which is a hard requirement for getting anything through the tightened notarization rules which became effective around that time).

Hi Klaus, did you guys start working on this issue ? Need any additional info to help or we are in the dark here ?

comment:29 by opharabot, 4 years ago

Same issue here. Just like many others, I downgraded to 6.1.2 some months ago in order to have a stable system. I didn't bother reporting the issue since there were so many reporting that I was sure it would be fixed. Unfortunately, this is no longer an option for me since I upgraded to BigSur yesterday. I can confirm that for me VirtualBox on Mac is dead. My main usage was a docker engine, so I will uninstall VirtualBox and go for the native docker setup. The inability of Oracle to even acknowledge the problem is really disappointing and sad.

comment:30 by FusionX86, 4 years ago

Add me to the list of affected users. I have 32GB of ram and have been living with the problem until now, but I'm seriously considering a different tool at this point. I use VirtualBox with vagrant heavily and it's starting to limit what I can run on my laptop when while VMs are running. For the record, I appreciate VirtualBox and what it offers for free, but this is a serious problem that's been going on too long now.

comment:31 by velna, 4 years ago

Some issue here. I have to set 2GB of ram for all VMs on my laptop, and luckly I didn't update my Mac Mini to Big Sur.

comment:32 by stuart, 3 years ago

One thing worth noting is the memory reported in macintosh "Activity" monitor shows that the "Memory" value is roughly double the "Real Mem" value (when you double the "Real mem" value). EG when running a 4GB VM on my mac it reports 7.90 GB "Memory" and 3.90 GB "Real Mem" (presumably the memory allocated in RAM). This sort of backs up the comment:20. However the real world peformance has certainly gotten much worse since upgrading from virtualbox 5 to virtualbox 6. Previously on my 16GB mac I could run three instances of 4GB RAM VM's (although it was a it slow), however now I cannot run more than one instance of a 4GB RAM VM without the mac becoming completely unusuable. Presumably page sawpping (or something similar) is the cause of the slowness. I do not really understand the internals of the macintosh memory management, but the changes to virtualbox from 6.1.4 have certainly resulted in significant performance degradation on Macintosh which is obviously related to the change made in 6.1.4. Perhaps the memory doubling has resulted in some odd behaviour for how macintosh swaps memory between RAM and disk???

comment:33 by tcarlos, 3 years ago

Please give a statement about the future of Virtualbox on Mac. Is it dead with no new Apple sytems with an intel processor are in the product pipeline? All versions of Virtualbox since 6.1.6 are completely unusable on the Mac because of this bug.

comment:34 by Hayden Bickerton, 3 years ago

Just made an account to chime in; I'm having the exact same issue. I actually found this thread after googling "virtualbox taking exactly twice as much memory", on OSX 10.15, after upgrading to Virtualbox 6.1.18. Downgrading to 6.1.2 fixed the issue immediately. It was near exact doubling from 4gb->8gb. I've been using the same configuration across multiple machines for years now.

Surprised to read that this is being ignored and chalked up to a user error. If affected users were given benefit of the doubt that we know how to use the computer, what could we do to proof the issue is real? I can dump all 8gig of the RAM to disk and upload it to google drive if it'd help.

EDIT

For anyone else that stumbles on this thread from Google, here's the links to the last working versions, 6.1.2. I didn't have to uninstall beforehand but YMMV:

VirtualBox 6.1.2

Last edited 3 years ago by Hayden Bickerton (previous) (diff)

comment:35 by paulson, 3 years ago

In order to converge on the key issue at hand let's clarify a few points:

First, there was a separate issue mentioned earlier in this bug report regarding a memory leak in macOS affecting VMWare Fusion and VirtualBox VMs which could lead to a host crash which was addressed in 'macOS Catalina 10.15.6 supplemental update (19G2021)' -- see ticket #19772 for details. The issue appears to have been specific to macOS Catalina.

Second, as mentioned in Comment:2 the Docker developers discovered and showed that the typical memory reporting tools on macOS Mojave, Catalina, and Big Sur, i.e. the 'Memory' column of the Activity Monitor.app and the 'MEM' column in top(1) are misleading as they double-count certain types of memory allocations. If the 'Real Mem' field of the Activity Monitor.app is referenced or the output of the footprint(1) CLI or the RSS field of the ps(1) command or the output of 'vmmap -summary' is checked then it will be seen that the amount of memory used by a guest VM hasn't changed with 6.1.4. For example, the attachment in Comment:11 shows that 3.6GB of memory is being used for the 4GB VM.

However, the way VirtualBox allocates memory for guests did change in 6.1.4. Prior to VirtualBox 6.1.4 memory for a guest VM was allocated using malloc(3) and mapped to the kernel when necessary which you can see in the following output of footprint(1) from a VM configured to use 4GB of memory:

======================================================================
VirtualBoxVM [52614]: 64-bit    Footprint: 4414 MB (4096 bytes per page)
======================================================================

  Dirty      Clean  Reclaimable    Regions    Category
    ---        ---          ---        ---    ---
4196 MB        0 B      3200 KB       2166    MALLOC_LARGE
 167 MB        0 B       8192 B         59    IOKit (IOMemoryDescriptor)
  35 MB        0 B      8076 KB         12    MALLOC_SMALL
7640 KB        0 B          0 B         12    MALLOC_TINY
3604 KB        0 B          0 B          3    CG backing stores
3338 KB     232 KB          0 B        352    __DATA
 896 KB        0 B        28 KB         66    stack
 660 KB        0 B          0 B         32    untagged ("VM_ALLOCATE")
 376 KB        0 B          0 B          6    CoreUI image data
 136 KB        0 B          0 B          3    CG raster data
 128 KB        0 B          0 B          1    Accelerate image backing stores
  88 KB        0 B          0 B         19    malloc metadata
  72 KB        0 B          0 B          1    libdispatch
  58 KB        0 B          0 B        446    unused dyld shared cache area
  52 KB        0 B          0 B          1    CG image
  32 KB        0 B          0 B          9    IOSurface
  32 KB        0 B          0 B          1    Activity Tracing
  24 KB        0 B          0 B          4    IOAccelerator
  24 KB        0 B          0 B          2    __DATA_CONST
  16 KB      18 MB          0 B         34    mapped file
  12 KB        0 B          0 B          2    __OBJC_RW
 8192 B        0 B          0 B          2    CoreImage
 8192 B        0 B          0 B          1    os_alloc_once
 4096 B        0 B          0 B          1    Foundation
 4096 B        0 B          0 B          1    CoreGraphics
    0 B      49 MB          0 B        340    __TEXT
    0 B    2404 KB          0 B         39    __LINKEDIT
    0 B        0 B          0 B          1    __FONT_DATA
    0 B        0 B          0 B          1    __OBJC_RO
    0 B        0 B          0 B          1    __UNICODE
    ---        ---          ---        ---    ---
4414 MB      69 MB        11 MB       3618    TOTAL

Auxiliary data:
        phys_footprint_peak: 4589 MB
        phys_footprint: 4579 MB

VM Object Dirty Analysis: Enabled

Starting with 6.1.4 guest memory is allocated in the kernel using the macOS IOKit memory descriptor routines and then mapped to user space when needed which can be seen in the following footprint(1) output from the same VM with the latest VMM changes in place:

======================================================================
VirtualBoxVM [51377]: 64-bit    Footprint: 4360 MB (4096 bytes per page)
======================================================================

  Dirty      Clean  Reclaimable    Regions    Category
    ---        ---          ---        ---    ---
4261 MB        0 B       8192 B       2106    IOKit (IOMemoryDescriptor)
  43 MB        0 B          0 B         13    MALLOC_SMALL
  40 MB        0 B        12 MB        118    MALLOC_LARGE
7716 KB        0 B          0 B         14    MALLOC_TINY
3604 KB        0 B          0 B          3    CG backing stores
3334 KB     232 KB          0 B        352    __DATA
 896 KB        0 B          0 B         66    stack
 660 KB        0 B          0 B         31    untagged ("VM_ALLOCATE")
 376 KB        0 B          0 B          6    CoreUI image data
 136 KB        0 B          0 B          3    CG raster data
 128 KB        0 B          0 B          1    Accelerate image backing stores
  88 KB        0 B          0 B         19    malloc metadata
  68 KB        0 B          0 B          1    libdispatch
  64 KB        0 B          0 B         17    IOSurface
  58 KB        0 B          0 B        446    unused dyld shared cache area
  52 KB        0 B          0 B          1    CG image
  32 KB        0 B          0 B          1    Activity Tracing
  24 KB        0 B          0 B          4    IOAccelerator
  24 KB        0 B          0 B          2    __DATA_CONST
  16 KB      17 MB          0 B         34    mapped file
  12 KB        0 B          0 B          2    __OBJC_RW
 8192 B        0 B          0 B          1    os_alloc_once
 8192 B        0 B          0 B          2    CoreImage
 4096 B        0 B          0 B          1    Foundation
 4096 B        0 B          0 B          1    CoreGraphics
    0 B      49 MB          0 B        340    __TEXT
    0 B    2404 KB          0 B         39    __LINKEDIT
    0 B        0 B          0 B          1    __FONT_DATA
    0 B        0 B          0 B          1    __OBJC_RO
    0 B        0 B          0 B          1    __UNICODE
    ---        ---          ---        ---    ---
4360 MB      69 MB        12 MB       3627    TOTAL

Auxiliary data:
        phys_footprint_peak: 8619 MB
        phys_footprint: 8616 MB

VM Object Dirty Analysis: Enabled

The 'phys_footprint' value seen in the footprint(1) output above corresponds to the kernel's "physical footprint" ledger and this is the value used by the 'Memory' column of the Activity Monitor.app and top(1)'s 'MEM' column and is what is apparently susceptible to double-counting certain types of memory allocations as the Docker team found.

So while the amount of memory used by a VirtualBox VM didn't change the way it is allocated did change with 6.1.4 and this has apparently caused some usability impact for macOS hosts as noted here. It isn't clear at this stage why this change is limiting the number of VMs macOS users can run starting with 6.1.4.

in reply to:  35 ; comment:36 by antg, 3 years ago

Replying to paulson:

In order to converge on the key issue at hand let's clarify a few ...


Thanks paulson.

  1. Are you saying the MacOS reporting of memory usage is doubled, but practically it is not doubled and, just a wrong reporting (and that the actual usage correct as expected), and that the performance should not be impacted?
  1. Either way could you (or anyone) help out and give shell commands/steps to help easily validate it? (clearly show how much memory is really used, and if performance is severely impacted or not)
  1. Is anyone actively working to resolve this? (even if only a mem usage reporting error?)
  1. Can't revert to 6.1.2 from the latest (now it's 6.1.26), due to error:

    VBoxManage: error: Unknown option: --clipboard

Since the older version expects:

--clipboard-mode

But I don't know how to wipe that --clipboard from the env. Checked a bit, it is not in my Vagrantfile and even after re-provision with that flag gone from Vagrantfile (on new virtualbox), still getting the same issue.

Last edited 3 years ago by antg (previous) (diff)

in reply to:  36 comment:37 by tcarlos, 3 years ago

Replying to antg:

  1. memory usage is doubled, it is not wrong reporting, the performance is extremely impacted
  2. just use any tool which measures the free usable memory
  3. obviously not

in reply to:  36 ; comment:38 by fth0, 3 years ago

Replying to antg:

  1. Are you saying the MacOS reporting of memory usage is doubled, but practically it is not doubled and, just a wrong reporting (and that the actual usage correct as expected), and that the performance should not be impacted?

From my POV, the VirtualBox memory usage is not doubled, but depending on the type of memory statistics, macOS versions since Mojave count major parts of this memory twice, and this impacts the macOS memory handling in general and ultimately the performance of the macOS-based device. If you Google hard enough, you'll discover that other well-known hypervisors are equally affected (e.g. VMware Fusion, Parallels).

  1. Either way could you (or anyone) help out and give shell commands/steps to help easily validate it? (clearly show how much memory is really used, and if performance is severely impacted or not)

Well, I cannot really answer that, because there is no easy way to analyze a highly complex topic like the memory management of contemporary operating systems. In comment:35, paulson already gave the tools, ranging from the high level Activity Monitor.app over intermediate level tools like ps(1) and top(1) to the low level vmmap(1) and footprint(1) tools. They will show you the current memory situation from several different angles, but they cannot explain to you what it really means.

For example, the second output of footprint(1) in comment:35 shows that although VirtualBox is only using 4360 MB of memory, phys_footprint is 8616 MB. Try it yourself.

in reply to:  38 comment:39 by scarhill, 3 years ago

Replying to fth0:

From my POV, the VirtualBox memory usage is not doubled, but depending on the type of memory statistics, macOS versions since Mojave count major parts of this memory twice, and this impacts the macOS memory handling in general and ultimately the performance of the macOS-based device. If you Google hard enough, you'll discover that other well-known hypervisors are equally affected (e.g. VMware Fusion, Parallels).

But the experience of users (including me) is that VirtualBox's performance impact changed radically after 6.1.2, independent of any OS changes. The fact that other hypervisors are having the same issue isn't really relevant, when we know that 6.1.2 doesn't have the problem. It may well be that there is a macOS issue, but that didn't impact VirtualBox until after some change that apparently occurred in 6.1.4.

comment:40 by fth0, 3 years ago

FWIW, this "some change" was also clearly described, paulson just didn't argue why it was necessary. Hypervisors like VirtualBox need the guest memory to be allocated in the kernel of the host OS, and not doing this right away could lead to consequences like promised memory not being available later on. And after the change VirtualBox was in the same boat as the other hypervisors.

comment:41 by wouterv-42, 3 years ago

One more report from a user that cannot practically use any recent version of VirtualBox because it affects the performance of the host computer to the point of being totally unusable.

Reading the discussion above, people seem to agree on these two points:

  • the OS reports double memory usage, at least with some ways of displaying that;
  • the computer becomes extremely slow.

Apparently the *way* memory is allocated *was* changed in 6.1.4, and I'd say focussing on the amount of memory reported doesn't really help since the reporting is inconclusive and probably faulty. Still, the performance issues remain, and would probably be caused by the change mentioned (I don't see other probable causes in https://www.virtualbox.org/wiki/Changelog).

Although this problem apparently affects quite a few people, it's hard to believe that it affects everyone. If so, it may result from particular combinations of hardware and/or OS. My setup (quite dated by now) is:

  • macOS 10.14.6 'Mojave'
  • MacBook Pro (Retina, 15-inch, Mid 2014)
  • 2.2 GHz Intel Core i7
  • 16 GB 1600 MHz DDR3

Are there any plans for an investigation or solution for this ticket?

comment:42 by Hayden Bickerton, 3 years ago

I would be willing to pay out of my own pocket for a plane ticket to the nearest virtualbox developers local coffee shop and show you the issue in person, if you are unable to get your hands on a recent MacBook to see for yourselves.

comment:43 by mezey, 3 years ago

The issue does not seem to be with VirtualBox, but with MacOS. From the paper the Docker team wrote:

"Since the qemu and hyperkit processes show the same double-accounting problem in Activity Monitor despite them being completely separate codebases, we conclude the bug must be in macOS itself, probably the Hypervisor.framework. We have reported the bug to Apple as bug number 48714544."

comment:44 by mporhel, 3 years ago

Hi,

I recently reported the same issue on Apple Feedback Assistant, and they have just answered with the following message:

...
Thanks for your report.  This is an issue specific to a third-party, VirtualBox.

This needs to be resolved in the third party software.  VirtualBox needs to adopt the hv_vm_allocate()/hv_vm_deallocate() APIs that were introduced in macOS Monterey.

There is no Apple issue here.  Please escalate this to VirtualBox, and please close this report.
...

It seems that contrary to comment:43 the issue might be in Virtual Box.

Regards

comment:45 by tcarlos, 2 years ago

It is a virtual box problem, created by the Virtual Box Team with 6.1.4. With Catalina being the last MacOS which works with VirtualBox Releases without this major bug, This major bug affects any Mac User with an recent OS. VirtualBox for Mac is dead now, for everyone, everywhere.

Last edited 2 years ago by tcarlos (previous) (diff)

comment:46 by elatllat, 2 years ago

An Alternative:

brew install --cask utm

https://mac.getutm.app/

comment:47 by ggbasic, 2 years ago

Is this ticket still open? The issue still ocurre on 6.1.34 (Intel) with the macOS 11.6.6 (intel)

comment:48 by mporhel, 2 years ago

The issue is also still present with VirtualBox 7.0.2 on macOS Ventura.

in reply to:  48 comment:49 by tcarlos, 19 months ago

The issue is still present with VirtualBox 7.0.6. This ongoing problem which is according to Apple, not a MacOS problem, should be solved in the next release. The last working release of VirtualBox without this very annoying bug was 6.1.2, which is supported by Big Sur onwards.

comment:50 by cepal, 19 months ago

I just installed stable Virtualbox 7.0.7 (build 156262) and the problem is still there (double RAM allocation on MacOS host).

comment:51 by Klaus Espenlaub, 19 months ago

If the problem is still there with VirtualBox 7.0 then you have proven that macOS does incorrect resource counting. VirtualBox 7 does not use any kernel extensions and therefore relies on the Hypervisor.framework API.

comment:52 by tcarlos, 15 months ago

According to Apple, there is no error in MacOS. After all this useless postings for this bug from the Developers, there should be more than this cryptic statement. Since the edge of time, there are bugs in Operating Systems like Windows. It's the task of real developers to make things work, even, when there is a bug.

comment:53 by Freewill, 5 months ago

This bug is still present in Virtualbox 7.0.18 on Sonoma 14.5. I have attempted to roll back to the 4 year old 6.1.2, but I could not start any VMs.

I have been a happy use of virtualbox for many years now. I am saddened to be forced to use another hypervisor after all these years. Thanks for the good times, but alas I have to switch. I cannot work effectively with the rest of my OS swapping and compressing memory.

comment:54 by aaaaa2, 5 months ago

Hitting this too. My VM has base memory set to 6GB. However, after running the VM for several hours the real mem as reported by Activity Monitor goes over 12GB and eventually leads to the guest locking up (the host has 16GB of RAM, I'm guessing it tries to allocated even more and just fails). The only work around is to regularly quit VirtualBox completely, not just restart the VM.

VB 7.0.18 on Mojave

comment:55 by Freewill, 4 months ago

Since my posting 3 weeks ago, I have been using VMware fusion. It also appears to double-allocate RAM. This confirms the suspicions on https://forums.virtualbox.org/viewtopic.php?t=110445.

comment:56 by Freewill, 4 months ago

I found an interesting thing just now. I was running an outdated version of the guest tools (6.0.0) while VB is on 7.0.18. Once I upgraded, things got a lot better. I was having issues with memory usage during heavy I/O on a shared drive.

Note: See TracTickets for help on using tickets.

© 2024 Oracle Support Privacy / Do Not Sell My Info Terms of Use Trademark Policy Automated Access Etiquette