Opened 4 years ago
Last modified 4 months ago
#19726 new defect
Reported memory usage doubled in 6.1.4 on MacOS Mojave and later
Reported by: | arthens | Owned by: | |
---|---|---|---|
Component: | other | Version: | VirtualBox 6.1.10 |
Keywords: | catalina memory | Cc: | |
Guest type: | other | Host type: | Mac OS X |
Description
Hi there,
we recently noticed that memory usage doubled on Catalina (confirmed on 10.15.5). Upon investigating the problem we figured out that this issue began with 6.1.4. The problem is still present in 6.1.10.
Up until 6.1.2 memory and real memory were basically the same. Allocating 4GB in the VM resulted in a memory usage of 4GB.
From 6.1.4 onwards, memory is always twice as large as real memory:
- allocating 4GB results in 8GB being used
- allocating 8GB results in 16GB being used
and so on (see attached images)
Reverting back to 6.1.2 fixes the problem. We couldn't find a bug report about it, but we did find a few reports on the forums:
- https://forums.virtualbox.org/viewtopic.php?f=8&t=97149
- https://forums.virtualbox.org/viewtopic.php?f=8&t=97181
It's possible this bug might specific to Catalina, as I personally only noticed running out on memory after upgrading from Mojave. I think I was already on 6.1.4, but I don't have a Mojave machine so I can't test this theory.
Everyone in our development team had to install 6.1.2 because we generally allocate 8GB to the virtual machine, which ended up using 16GB. As most of us have 16GB of ram, this made our machines unusable.
Please let me know if you need further information
Attachments (6)
Change History (62)
by , 4 years ago
Attachment: | 6.1.2 - 4GB allocated.png added |
---|
by , 4 years ago
Attachment: | 6.1.4 - 4GB allocated.png added |
---|
by , 4 years ago
Attachment: | 6.1.10 - 4GB allocated.png added |
---|
by , 4 years ago
Attachment: | 6.1.10 - 8GB allocated.png added |
---|
comment:1 by , 4 years ago
comment:2 by , 4 years ago
Status: | new → awaitsfeedback |
---|---|
Summary: | Memory usage doubled in 6.1.4 on Catalina → Reported memory usage doubled in 6.1.4 on MacOS Mojave and later |
There were some low-level virtual memory management changes made during VirtualBox 6.1.4 development which now trigger a bug in the MacOS memory usage reporting which happens on Mojave and later. The Docker developers ran into this as well and created a nice writeup of the issue here:
https://docs.google.com/document/d/17ZiQC1Tp9iH320K-uqVLyiJmk4DHJ3c4zgQetJiKYQM/edit
In short, the memory usage of your VirtualBox virtual machine hasn't changed as you can confirm with the MacOS Activity Monitor's "Real Mem" field, the "RSS" field of ps(1) (e.g. $ ps -x -o pid,rss,vsz,command), or the output from 'vmmap -summary <pid>'.
Is this really causing problems for you on your system(s)? How exactly did the machines become "unusable"?
comment:3 by , 4 years ago
How exactly did the machines become "unusable"?
Well consuming 2x the virtual memory added a bunch of "memory pressure" (mac os terminology), which resulted in all other apps being starved for memory. My browsers, outlook etc all slowed to a crawl, and if I try to do any heavy compiling, the system even locks up periodically.
comment:4 by , 4 years ago
Is this really causing problems for you on your system(s)? How exactly did the machines become "unusable"?
It's a significant problem. Before realising I could fix the bug by downgrading to 6.1.2 I had to stop using my editor of choice (Intellij) in favour of an editor with lower memory requirements (vscode) because the excessive memory pressure meant my mac was extremely slow. Even trivial actions like opening a new file or switching tab had a noticeable delay.
I've read about the difference between memory and real memory before opening this ticket, but I'm finding it hard to reconcile it with the fact that the slowness was noticeable and went away after downgrading (I'm back using intellij now)
comment:5 by , 4 years ago
I have attached another log with the "double RAM" use. Host macOS 10.15.6, VirtualBox 6.1.12, guest Win10 x64, 3GB RAM (consumes over 6G).
comment:6 by , 4 years ago
Thanks for the additional feedback. In addition to the double-counting memory reporting MacOS bug being triggered it appears that there is some aspect of the virtual memory changes made in 6.1.4 development which is leading to additional memory usage on Mac hosts. Further investigation is required.
comment:7 by , 4 years ago
I see the same issue on my Mac as well (Catalina 10.15.6, VirtualBox 6.1.12 r139181). Because of this issue, I am unable to run multiple VM's like I used to before.
One thing I noticed was the "Real memory" usage of kernel_task process seems to increase along with VirtualBox VM process.
With no VirtualBox VM running:
Real Mem Memory -------- ------ kernel_task 2.77 GB 84.2 MB
After starting a saved VM (Ubuntu 20.4 with 4 GB RAM):
Real Mem Memory -------- ------ VirtualBox VM 3.42 GB 6.44 GB kernel_task 5.87 GB 88.0 MB
After running few memory intensive applications inside the VM:
Real Mem Memory -------- ------ VirtualBox VM 4.30 GB 8.23 GB kernel_task 6.79 GB 88.0 MB
After closing (saving) VM:
Real Mem Memory -------- ------ kernel_task 2.68 GB 86.1 MB
And finally reopening the saved VM:
Real Mem Memory -------- ------ VirtualBox VM 4.28 GB 8.22 GB kernel_task 6.76 GB 88.1 MB
PS: "Real Mem" and "Memory" as reported by "Activity Monitor"
comment:8 by , 4 years ago
I faced the same issue in Mac Pro (Catalina 10.15.6, VirtualBox 6.1.12 r139181) having 8GB RAM. After reverting to Virtual Box 6.1.2 and corresponding Guest Addition, kernel task is not showing 6GB, but still shows 3GB - resulting in sluggish performance of Mac.
Without VM —————-
kernel_task : Memory 143.4MB Real Memory: 2.53GB
After starting VM ( Windows 7 with allocated RAM size of 4GB) ————————————————————————————-
Virtual Box VM: Memory : 8.3GB Real Memory: 4.31GB kernel_task : Memory : 142.8MB Real Memory: 6.36GB
With heavy application running in guest VM ———————————————————-
Virtual Box VM: Memory : 8.51GB Real Memory: 4.21GB kernel_task : Memory : 162.3MB Real Memory: 6.66GB
comment:9 by , 4 years ago
I wonder if the kernel_task issue is the one addressed in the 10.15.6 supplemental update that was released yesterday.
comment:10 by , 4 years ago
I had the same issue with MacOS Catalina 10.15.6 and VBox 6.1.12. Guest OS is Windows 10. Memory assigned to VM as 6 GB and memory taken up by VBox process was > 12 GB. If I left the VM running overnight, my Mac would be totally unresponsive or crashed by the morning. Reverting to 6.1.2 has brought down the memory usage and hopefully will remove the instability. Fingers crossed and waiting for the fix.
comment:11 by , 4 years ago
I still have this issue with VirtualBox 6.1.12 on MacOS 10.15.6 (19G2021) (i.e. with the 10.15.6 supplemental update installed). I have a VM configured to use 4GB RAM. Activity Monitor says it's currently using 6.86GB Memory (3.60 GB Real Mem), and kernel_task is ~86 MB Memory (but 5.50 GB Real Mem). See attachment osearbhain-1.jpg.
by , 4 years ago
Attachment: | osearbhain-1.jpg added |
---|
comment:12 by , 4 years ago
Finally have a stable system with VirtualBox 6.1.2 r135662 and MacOS 10.15.6 Supplemental update.
https://support.apple.com/kb/DL2049
Based on @osearbhain's observation, I shall stick with this setup.
comment:13 by , 4 years ago
Just a small me too. @sujithrs is that osx patch needed for it to work property with 6.1.2?
comment:14 by , 4 years ago
AFAIK you need that patch regardless what version of virtualbox you use. It fixes a (different) virtualisation bug that would cause memory usage to continually increase to the point that you mac would crash.
This bug never makes your mac crash, it just makes it slow.
comment:15 by , 4 years ago
Confirmed issue on MacOS 10.14.6 and VirtualBox 6.1.14 r140239 (Qt5.6.3) and Version 6.1.16 r140961 (Qt5.6.3)
Downgrading really isn't a solution. Is there any way to get some acknowledgement that the dev team recognizes and has added the issue to the backlog?
comment:16 by , 4 years ago
Status: | awaitsfeedback → new |
---|
Confirmed on 10.15.7 with 6.1.14 and 6.1.16
comment:17 by , 4 years ago
Earlier on Catalina OS I was able to install old version 6.0.24 and ram consumption was ok but now after upgrading to Big Sur I can install only last version 6.1.16 and there is issue with x2 RAM consumption. Could you please prioritise this issue?
comment:18 by , 4 years ago
In our team we can confirm the issue with latest MacOS (Catalina and Big Sur):
Since VBox 6.1.4 the memory consumption for any Linux VM (in our case a docker-machine, boot2docker based) is doubled. Downgrading to 6.1.2 solves the issue, but unfortunately this is not an option on Big Sur, since 6.1.16 is required.
It would be very cool if it could be addressed on the next releases.
comment:19 by , 4 years ago
Is there any ETA for this? Having Big Sur on board makes impossible using of VirtualBox. That's more than critical.
follow-up: 28 comment:20 by , 4 years ago
We're not working on anything since we don't believe that there is an actual doubling of the memory consumption.
I checked what we changed, and it's not making any sense that it causes more than double accounting: https://www.virtualbox.org/log/vbox/trunk/src/VBox/Runtime/r0drv/darwin (the relevant changes between 6.1.2 and 6.1.4 are r83070 to r83098). The changes were necessary to resolve macOS misbehavior when the macOS hardened runtime was enabled (which is a hard requirement for getting anything through the tightened notarization rules which became effective around that time).
comment:21 by , 4 years ago
Klaus are you suggesting the double memory usage is just a visual bug? Or are you skeptical that it's happening at all? Either way I'd be happy to demonstrate the problem on a remote session, and/or provide additional information. Feel free to email me at [username]@gmail.com
comment:22 by , 4 years ago
I'm really a bit confused why this issue could be at risk of dismissal when so many different users have verified being affected. I've only lurked this issue so far because I didn't have an Oracle account, and it looked like there were enough reproducing users that I didn't need to bother making one. You can add me to the pile now though, there is simply no way this isn't a real issue. My primary usage for my work macbook is to run a Centos VM, so that VM gets dedicated the majority of my CPU and Memory resources. When I run 6.1.2 I have a stable environment with no issues. When I run a newer version it's only a matter of time until my host system starts paging like mad and everything locks up until I manage to kill off the VM. There is no way this is not a real issue.
I'm less savvy than others when it comes to producing VirtualBox diagnostic information, but if anyone wants to tell me what is required then I too am happy to produce whatever proof would be useful.
comment:23 by , 4 years ago
This is a problem for anyone using VirtualBox since 6.1.6. There is no working Virtualbox Installation since 6.1.6 without this extremely annoying bug. This bug makes VirtualBox on actual Macs completely unusable. There is not one working installation of VirtualBox on MacOS with Big Sur. I will now move on to another solution of an other vendor, since Oracle seems not capable to solve this obvious major problem for months. VirtualBox on MacOS is dead now.
comment:24 by , 4 years ago
Can add another confirmation, if that matters. On MacOS 10.14.6, VirtualBox 6.1.16 (latest at the time of writing), a VM with 2 GB memory allocated reports usage between 3 and 4 GB. Five such VMs on a MacBook Pro with 16 GB of RAM causes the system to crawl to a halt.
Issue is resolved completely by downgrading to VirtualBox 6.1.2.
comment:25 by , 4 years ago
I have the same observation for the past year as many other reported. Compared with 6.1.2, the double memory issue is definitely not just the number reported but the actual resource occupied because the system was sluggish when VM is running in any version after 6.1.2 while it is running fine with 6.1.2. I am still on Catalina right now and can still use the old version. But after Big Sur upgrade it won't be usable with all the other apps running on my host.
I guess one could reproduce by creating a few VMs in parallel with total VMs' ram a bit below host's ram and run ubuntu CD at the same time and compare 6.1.2 with 6.1.4 experience.
comment:26 by , 4 years ago
I have Virtualbox 6.1.18 installed on macOS Catalina (10.15.7) and got hit by the bug too.
comment:27 by , 4 years ago
Same problem here with Virtualbox 6.1.18 on MacOS Big Sur (11.2.2).
Windows10 VM with 3.4 GB and Virtualbox using 7.35 GB.
After start VM:
PID | COMMAND | %CPU | TIME | #TH | #WQ | #PORT | MEM | PURG | CMPRS | PGRP | PPID |
637 | VirtualBoxVM | 38.5 | 05:11.09 | 32 | 1 | 301 | 7355M | 159M | 69M- | 637 | 623 |
841 | VirtualBox | 0.0 | 00:14.58 | 9 | 1 | 261 | 117M | 0B | 103M | 841 | 1 |
Sadly I can't downgrade to 6.1.2 to check if works.
I have been notice my Mac is running very slow for a few months(~1y ago), only now I realize its running off memory because VirtualBox.
The problem occurs with any Guest OS, my Windows and Linux VM all double memory usage.
comment:28 by , 4 years ago
Replying to klaus:
We're not working on anything since we don't believe that there is an actual doubling of the memory consumption.
I checked what we changed, and it's not making any sense that it causes more than double accounting: https://www.virtualbox.org/log/vbox/trunk/src/VBox/Runtime/r0drv/darwin (the relevant changes between 6.1.2 and 6.1.4 are r83070 to r83098). The changes were necessary to resolve macOS misbehavior when the macOS hardened runtime was enabled (which is a hard requirement for getting anything through the tightened notarization rules which became effective around that time).
Hi Klaus, did you guys start working on this issue ? Need any additional info to help or we are in the dark here ?
comment:29 by , 4 years ago
Same issue here. Just like many others, I downgraded to 6.1.2 some months ago in order to have a stable system. I didn't bother reporting the issue since there were so many reporting that I was sure it would be fixed. Unfortunately, this is no longer an option for me since I upgraded to BigSur yesterday. I can confirm that for me VirtualBox on Mac is dead. My main usage was a docker engine, so I will uninstall VirtualBox and go for the native docker setup. The inability of Oracle to even acknowledge the problem is really disappointing and sad.
comment:30 by , 4 years ago
Add me to the list of affected users. I have 32GB of ram and have been living with the problem until now, but I'm seriously considering a different tool at this point. I use VirtualBox with vagrant heavily and it's starting to limit what I can run on my laptop when while VMs are running. For the record, I appreciate VirtualBox and what it offers for free, but this is a serious problem that's been going on too long now.
comment:31 by , 4 years ago
Some issue here. I have to set 2GB of ram for all VMs on my laptop, and luckly I didn't update my Mac Mini to Big Sur.
comment:32 by , 3 years ago
One thing worth noting is the memory reported in macintosh "Activity" monitor shows that the "Memory" value is roughly double the "Real Mem" value (when you double the "Real mem" value). EG when running a 4GB VM on my mac it reports 7.90 GB "Memory" and 3.90 GB "Real Mem" (presumably the memory allocated in RAM). This sort of backs up the comment:20. However the real world peformance has certainly gotten much worse since upgrading from virtualbox 5 to virtualbox 6. Previously on my 16GB mac I could run three instances of 4GB RAM VM's (although it was a it slow), however now I cannot run more than one instance of a 4GB RAM VM without the mac becoming completely unusuable. Presumably page sawpping (or something similar) is the cause of the slowness. I do not really understand the internals of the macintosh memory management, but the changes to virtualbox from 6.1.4 have certainly resulted in significant performance degradation on Macintosh which is obviously related to the change made in 6.1.4. Perhaps the memory doubling has resulted in some odd behaviour for how macintosh swaps memory between RAM and disk???
comment:33 by , 3 years ago
Please give a statement about the future of Virtualbox on Mac. Is it dead with no new Apple sytems with an intel processor are in the product pipeline? All versions of Virtualbox since 6.1.6 are completely unusable on the Mac because of this bug.
comment:34 by , 3 years ago
Just made an account to chime in; I'm having the exact same issue. I actually found this thread after googling "virtualbox taking exactly twice as much memory", on OSX 10.15, after upgrading to Virtualbox 6.1.18. Downgrading to 6.1.2 fixed the issue immediately. It was near exact doubling from 4gb->8gb. I've been using the same configuration across multiple machines for years now.
Surprised to read that this is being ignored and chalked up to a user error. If affected users were given benefit of the doubt that we know how to use the computer, what could we do to proof the issue is real? I can dump all 8gig of the RAM to disk and upload it to google drive if it'd help.
EDIT
For anyone else that stumbles on this thread from Google, here's the links to the last working versions, 6.1.2. I didn't have to uninstall beforehand but YMMV:
follow-up: 36 comment:35 by , 3 years ago
In order to converge on the key issue at hand let's clarify a few points:
First, there was a separate issue mentioned earlier in this bug report regarding a memory leak in macOS affecting VMWare Fusion and VirtualBox VMs which could lead to a host crash which was addressed in 'macOS Catalina 10.15.6 supplemental update (19G2021)' -- see ticket #19772 for details. The issue appears to have been specific to macOS Catalina.
Second, as mentioned in Comment:2 the Docker developers discovered and showed that the typical memory reporting tools on macOS Mojave, Catalina, and Big Sur, i.e. the 'Memory' column of the Activity Monitor.app and the 'MEM' column in top(1) are misleading as they double-count certain types of memory allocations. If the 'Real Mem' field of the Activity Monitor.app is referenced or the output of the footprint(1) CLI or the RSS field of the ps(1) command or the output of 'vmmap -summary' is checked then it will be seen that the amount of memory used by a guest VM hasn't changed with 6.1.4. For example, the attachment in Comment:11 shows that 3.6GB of memory is being used for the 4GB VM.
However, the way VirtualBox allocates memory for guests did change in 6.1.4. Prior to VirtualBox 6.1.4 memory for a guest VM was allocated using malloc(3) and mapped to the kernel when necessary which you can see in the following output of footprint(1) from a VM configured to use 4GB of memory:
====================================================================== VirtualBoxVM [52614]: 64-bit Footprint: 4414 MB (4096 bytes per page) ====================================================================== Dirty Clean Reclaimable Regions Category --- --- --- --- --- 4196 MB 0 B 3200 KB 2166 MALLOC_LARGE 167 MB 0 B 8192 B 59 IOKit (IOMemoryDescriptor) 35 MB 0 B 8076 KB 12 MALLOC_SMALL 7640 KB 0 B 0 B 12 MALLOC_TINY 3604 KB 0 B 0 B 3 CG backing stores 3338 KB 232 KB 0 B 352 __DATA 896 KB 0 B 28 KB 66 stack 660 KB 0 B 0 B 32 untagged ("VM_ALLOCATE") 376 KB 0 B 0 B 6 CoreUI image data 136 KB 0 B 0 B 3 CG raster data 128 KB 0 B 0 B 1 Accelerate image backing stores 88 KB 0 B 0 B 19 malloc metadata 72 KB 0 B 0 B 1 libdispatch 58 KB 0 B 0 B 446 unused dyld shared cache area 52 KB 0 B 0 B 1 CG image 32 KB 0 B 0 B 9 IOSurface 32 KB 0 B 0 B 1 Activity Tracing 24 KB 0 B 0 B 4 IOAccelerator 24 KB 0 B 0 B 2 __DATA_CONST 16 KB 18 MB 0 B 34 mapped file 12 KB 0 B 0 B 2 __OBJC_RW 8192 B 0 B 0 B 2 CoreImage 8192 B 0 B 0 B 1 os_alloc_once 4096 B 0 B 0 B 1 Foundation 4096 B 0 B 0 B 1 CoreGraphics 0 B 49 MB 0 B 340 __TEXT 0 B 2404 KB 0 B 39 __LINKEDIT 0 B 0 B 0 B 1 __FONT_DATA 0 B 0 B 0 B 1 __OBJC_RO 0 B 0 B 0 B 1 __UNICODE --- --- --- --- --- 4414 MB 69 MB 11 MB 3618 TOTAL Auxiliary data: phys_footprint_peak: 4589 MB phys_footprint: 4579 MB VM Object Dirty Analysis: Enabled
Starting with 6.1.4 guest memory is allocated in the kernel using the macOS IOKit memory descriptor routines and then mapped to user space when needed which can be seen in the following footprint(1) output from the same VM with the latest VMM changes in place:
====================================================================== VirtualBoxVM [51377]: 64-bit Footprint: 4360 MB (4096 bytes per page) ====================================================================== Dirty Clean Reclaimable Regions Category --- --- --- --- --- 4261 MB 0 B 8192 B 2106 IOKit (IOMemoryDescriptor) 43 MB 0 B 0 B 13 MALLOC_SMALL 40 MB 0 B 12 MB 118 MALLOC_LARGE 7716 KB 0 B 0 B 14 MALLOC_TINY 3604 KB 0 B 0 B 3 CG backing stores 3334 KB 232 KB 0 B 352 __DATA 896 KB 0 B 0 B 66 stack 660 KB 0 B 0 B 31 untagged ("VM_ALLOCATE") 376 KB 0 B 0 B 6 CoreUI image data 136 KB 0 B 0 B 3 CG raster data 128 KB 0 B 0 B 1 Accelerate image backing stores 88 KB 0 B 0 B 19 malloc metadata 68 KB 0 B 0 B 1 libdispatch 64 KB 0 B 0 B 17 IOSurface 58 KB 0 B 0 B 446 unused dyld shared cache area 52 KB 0 B 0 B 1 CG image 32 KB 0 B 0 B 1 Activity Tracing 24 KB 0 B 0 B 4 IOAccelerator 24 KB 0 B 0 B 2 __DATA_CONST 16 KB 17 MB 0 B 34 mapped file 12 KB 0 B 0 B 2 __OBJC_RW 8192 B 0 B 0 B 1 os_alloc_once 8192 B 0 B 0 B 2 CoreImage 4096 B 0 B 0 B 1 Foundation 4096 B 0 B 0 B 1 CoreGraphics 0 B 49 MB 0 B 340 __TEXT 0 B 2404 KB 0 B 39 __LINKEDIT 0 B 0 B 0 B 1 __FONT_DATA 0 B 0 B 0 B 1 __OBJC_RO 0 B 0 B 0 B 1 __UNICODE --- --- --- --- --- 4360 MB 69 MB 12 MB 3627 TOTAL Auxiliary data: phys_footprint_peak: 8619 MB phys_footprint: 8616 MB VM Object Dirty Analysis: Enabled
The 'phys_footprint' value seen in the footprint(1) output above corresponds to the kernel's "physical footprint" ledger and this is the value used by the 'Memory' column of the Activity Monitor.app and top(1)'s 'MEM' column and is what is apparently susceptible to double-counting certain types of memory allocations as the Docker team found.
So while the amount of memory used by a VirtualBox VM didn't change the way it is allocated did change with 6.1.4 and this has apparently caused some usability impact for macOS hosts as noted here. It isn't clear at this stage why this change is limiting the number of VMs macOS users can run starting with 6.1.4.
follow-ups: 37 38 comment:36 by , 3 years ago
Replying to paulson:
In order to converge on the key issue at hand let's clarify a few ...
Thanks paulson.
- Are you saying the MacOS reporting of memory usage is doubled, but practically it is not doubled and, just a wrong reporting (and that the actual usage correct as expected), and that the performance should not be impacted?
- Either way could you (or anyone) help out and give shell commands/steps to help easily validate it? (clearly show how much memory is really used, and if performance is severely impacted or not)
- Is anyone actively working to resolve this? (even if only a mem usage reporting error?)
- Can't revert to 6.1.2 from the latest (now it's 6.1.26), due to error:
VBoxManage: error: Unknown option: --clipboard
Since the older version expects:
--clipboard-mode
But I don't know how to wipe that
--clipboard
from the env. Checked a bit, it is not in my Vagrantfile and even after re-provision with that flag gone from Vagrantfile (on new virtualbox), still getting the same issue.
comment:37 by , 3 years ago
Replying to antg:
- memory usage is doubled, it is not wrong reporting, the performance is extremely impacted
- just use any tool which measures the free usable memory
- obviously not
follow-up: 39 comment:38 by , 3 years ago
Replying to antg:
- Are you saying the MacOS reporting of memory usage is doubled, but practically it is not doubled and, just a wrong reporting (and that the actual usage correct as expected), and that the performance should not be impacted?
From my POV, the VirtualBox memory usage is not doubled, but depending on the type of memory statistics, macOS versions since Mojave count major parts of this memory twice, and this impacts the macOS memory handling in general and ultimately the performance of the macOS-based device. If you Google hard enough, you'll discover that other well-known hypervisors are equally affected (e.g. VMware Fusion, Parallels).
- Either way could you (or anyone) help out and give shell commands/steps to help easily validate it? (clearly show how much memory is really used, and if performance is severely impacted or not)
Well, I cannot really answer that, because there is no easy way to analyze a highly complex topic like the memory management of contemporary operating systems. In comment:35, paulson already gave the tools, ranging from the high level Activity Monitor.app over intermediate level tools like ps(1) and top(1) to the low level vmmap(1) and footprint(1) tools. They will show you the current memory situation from several different angles, but they cannot explain to you what it really means.
For example, the second output of footprint(1) in comment:35 shows that although VirtualBox is only using 4360 MB of memory, phys_footprint is 8616 MB. Try it yourself.
comment:39 by , 3 years ago
Replying to fth0:
From my POV, the VirtualBox memory usage is not doubled, but depending on the type of memory statistics, macOS versions since Mojave count major parts of this memory twice, and this impacts the macOS memory handling in general and ultimately the performance of the macOS-based device. If you Google hard enough, you'll discover that other well-known hypervisors are equally affected (e.g. VMware Fusion, Parallels).
But the experience of users (including me) is that VirtualBox's performance impact changed radically after 6.1.2, independent of any OS changes. The fact that other hypervisors are having the same issue isn't really relevant, when we know that 6.1.2 doesn't have the problem. It may well be that there is a macOS issue, but that didn't impact VirtualBox until after some change that apparently occurred in 6.1.4.
comment:40 by , 3 years ago
FWIW, this "some change" was also clearly described, paulson just didn't argue why it was necessary. Hypervisors like VirtualBox need the guest memory to be allocated in the kernel of the host OS, and not doing this right away could lead to consequences like promised memory not being available later on. And after the change VirtualBox was in the same boat as the other hypervisors.
comment:41 by , 3 years ago
One more report from a user that cannot practically use any recent version of VirtualBox because it affects the performance of the host computer to the point of being totally unusable.
Reading the discussion above, people seem to agree on these two points:
- the OS reports double memory usage, at least with some ways of displaying that;
- the computer becomes extremely slow.
Apparently the *way* memory is allocated *was* changed in 6.1.4, and I'd say focussing on the amount of memory reported doesn't really help since the reporting is inconclusive and probably faulty. Still, the performance issues remain, and would probably be caused by the change mentioned (I don't see other probable causes in https://www.virtualbox.org/wiki/Changelog).
Although this problem apparently affects quite a few people, it's hard to believe that it affects everyone. If so, it may result from particular combinations of hardware and/or OS. My setup (quite dated by now) is:
- macOS 10.14.6 'Mojave'
- MacBook Pro (Retina, 15-inch, Mid 2014)
- 2.2 GHz Intel Core i7
- 16 GB 1600 MHz DDR3
Are there any plans for an investigation or solution for this ticket?
comment:42 by , 3 years ago
I would be willing to pay out of my own pocket for a plane ticket to the nearest virtualbox developers local coffee shop and show you the issue in person, if you are unable to get your hands on a recent MacBook to see for yourselves.
comment:43 by , 3 years ago
The issue does not seem to be with VirtualBox, but with MacOS. From the paper the Docker team wrote:
"Since the qemu and hyperkit processes show the same double-accounting problem in Activity Monitor despite them being completely separate codebases, we conclude the bug must be in macOS itself, probably the Hypervisor.framework. We have reported the bug to Apple as bug number 48714544."
comment:44 by , 3 years ago
Hi,
I recently reported the same issue on Apple Feedback Assistant, and they have just answered with the following message:
... Thanks for your report. This is an issue specific to a third-party, VirtualBox. This needs to be resolved in the third party software. VirtualBox needs to adopt the hv_vm_allocate()/hv_vm_deallocate() APIs that were introduced in macOS Monterey. There is no Apple issue here. Please escalate this to VirtualBox, and please close this report. ...
It seems that contrary to comment:43 the issue might be in Virtual Box.
Regards
comment:45 by , 2 years ago
It is a virtual box problem, created by the Virtual Box Team with 6.1.4. With Catalina being the last MacOS which works with VirtualBox Releases without this major bug, This major bug affects any Mac User with an recent OS. VirtualBox for Mac is dead now, for everyone, everywhere.
comment:47 by , 2 years ago
Is this ticket still open? The issue still ocurre on 6.1.34 (Intel) with the macOS 11.6.6 (intel)
follow-up: 49 comment:48 by , 2 years ago
The issue is also still present with VirtualBox 7.0.2 on macOS Ventura.
comment:49 by , 19 months ago
The issue is still present with VirtualBox 7.0.6. This ongoing problem which is according to Apple, not a MacOS problem, should be solved in the next release. The last working release of VirtualBox without this very annoying bug was 6.1.2, which is supported by Big Sur onwards.
comment:50 by , 19 months ago
I just installed stable Virtualbox 7.0.7 (build 156262) and the problem is still there (double RAM allocation on MacOS host).
comment:51 by , 19 months ago
If the problem is still there with VirtualBox 7.0 then you have proven that macOS does incorrect resource counting. VirtualBox 7 does not use any kernel extensions and therefore relies on the Hypervisor.framework API.
comment:52 by , 15 months ago
According to Apple, there is no error in MacOS. After all this useless postings for this bug from the Developers, there should be more than this cryptic statement. Since the edge of time, there are bugs in Operating Systems like Windows. It's the task of real developers to make things work, even, when there is a bug.
comment:53 by , 5 months ago
This bug is still present in Virtualbox 7.0.18 on Sonoma 14.5. I have attempted to roll back to the 4 year old 6.1.2, but I could not start any VMs.
I have been a happy use of virtualbox for many years now. I am saddened to be forced to use another hypervisor after all these years. Thanks for the good times, but alas I have to switch. I cannot work effectively with the rest of my OS swapping and compressing memory.
comment:54 by , 5 months ago
Hitting this too. My VM has base memory set to 6GB. However, after running the VM for several hours the real mem as reported by Activity Monitor goes over 12GB and eventually leads to the guest locking up (the host has 16GB of RAM, I'm guessing it tries to allocated even more and just fails). The only work around is to regularly quit VirtualBox completely, not just restart the VM.
VB 7.0.18 on Mojave
comment:55 by , 4 months ago
Since my posting 3 weeks ago, I have been using VMware fusion. It also appears to double-allocate RAM. This confirms the suspicions on https://forums.virtualbox.org/viewtopic.php?t=110445.
comment:56 by , 4 months ago
I found an interesting thing just now. I was running an outdated version of the guest tools (6.0.0) while VB is on 7.0.18. Once I upgraded, things got a lot better. I was having issues with memory usage during heavy I/O on a shared drive.
Confirmed on my side. Since 6.1.3 and through 6.1.12 - any VM assigned memory will end up with actual consumption on host (Mac OS X 10.14.6 Mojave) consuming just over 2x the amount of memory.
Downloading and installing the 6.1.2 version, and restarting the VM returns to just a touch over 1x memory usage.