[vbox-dev] Problem with kernel 5.8

Larry Finger Larry.Finger at lwfinger.net
Thu Jun 11 22:21:03 UTC 2020

On 6/8/20 4:12 AM, Frank Batschulat wrote:
> Larry, thanks - this work will be tracked with:
> Ticket #19644 (accepted defect)
> Linux kernel version: 5.8 - we need changes
> https://www.virtualbox.org/ticket/19644

With each update to the mainline kernel, another API change for Kernel 5.8 
emerges. Thus far, there have been three different changes:

1. In struct mm_struct, member mmap_sem is renamed to mmap_lock. The comment in 
the commit says "Rename the mmap_sem field to mmap_lock.  Any new uses of this 
lock should now go through the new mmap locking api.  The mmap_lock is still
implemented as a rwsem, though this could change in the future." Until that 
unknown future time, only the member name needs to change, and that is what I 

2. The shadow set of kernel quantities stored in cpu_tlbstate are no longer 
available. The only quantity used from that structure is CR4. My fix is to use 
__read_cr4() rather than this_cpu_read(cpu_tlbstate.cr4) to read CR4, and 
ASMSetCR4(uNew) rather than this_cpu_write(cpu_tlbstate.cr4, uNew) to write a 
new value into CR4. This change has tested OK.

3. Kernel routine __get_vm_area() was first changed to __get_vm_area_caller(), 
and then the latter routine was no longer exported. In addition, routine 
map_vm_area() has been removed. I do not have a fix for this. At first it seemed 
that reverting to the method used before kernel 2.6.13 would work by not 
Unfortunately, this code fails to allocate the memory correctly, and VMs fail to 
start. I have been unable to fix that problem.

Until there is a fix for the code using RTMEMALLOC_EXEC_HEAP, I will be 
recommending that the openSUSE versions of 5.8 include a patch that exports 
__get_vm_area_caller() and map_kernel_range(). The latter routine is used to 
replace map_vm_area. With these two changes, the VB modules build and execute. I 
will be posting the patches for vboxdrv and the proposed kernel patch at 

As an editorial aside, it is clear that VirtualBox needs to get at least part of 
vboxdrv into the kernel as a module in the way that VMWare has done so that this 
incompatibility with memory management never occurs again.


More information about the vbox-dev mailing list