VirtualBox

Ticket #18911 (closed defect: fixed)

Opened 2 months ago

Last modified 2 months ago

Fixes for Linux kernel 5.3 wanted - part 1

Reported by: fbatschu Owned by: fbatschu
Component: host support Version: VirtualBox 6.0.12
Keywords: linux kernel 5.3 Cc:
Guest type: Linux Host type: Linux

Description

for the upcomming 5.3 Linux kernel we will require changes to the !Virtualbox code base.

Here's a first batch already identified:

https://www.virtualbox.org/pipermail/vbox-dev/2019-July/015297.html
 https://forums.virtualbox.org/viewtopic.php?f=3&t=93944

To quote the submitter:

Larry Finger Larry.Finger at lwfinger.net Wed Jul 24 00:13:47 UTC 2019

Yes, I do. The MIT-licensed patch is posted below. After I finish here, I will also post this material on the VBox forum.

The API changes are as follows:

  1. The for_ifa() and endfor_ifa() macros are removed. They are so simple that it is better to turn then into in-line code.
  1. Routine smp_call_function() is changed from type int to void.
Index: VirtualBox-6.0.10/src/VBox/HostDrivers/VBoxNetFlt/linux/VBoxNetFlt-linux.c
===================================================================
--- VirtualBox-6.0.10.orig/src/VBox/HostDrivers/VBoxNetFlt/linux/VBoxNetFlt-linux.c
+++ VirtualBox-6.0.10/src/VBox/HostDrivers/VBoxNetFlt/linux/VBoxNetFlt-linux.c
@@ -2123,7 +2123,10 @@ static int vboxNetFltLinuxEnumeratorCall
  #endif
      if (in_dev != NULL)
      {
-        for_ifa(in_dev) {
+       /* macros for_ifa() and endfor_ifa() disappear for kernel 5.3
+        * Code them directly */
+        struct in_ifaddr *ifa;
+       for (ifa = in_dev->ifa_list; ifa; ifa = ifa->ifa_next) {
              if (VBOX_IPV4_IS_LOOPBACK(ifa->ifa_address))
                  return NOTIFY_OK;

@@ -2137,7 +2140,7 @@ static int vboxNetFltLinuxEnumeratorCall

              pThis->pSwitchPort->pfnNotifyHostAddress(pThis->pSwitchPort,
                  /* :fAdded */ true, kIntNetAddrType_IPv4, &ifa->ifa_address);
-        } endfor_ifa(in_dev);
+        }
      }

      /*
Index: VirtualBox-6.0.10/src/VBox/Runtime/r0drv/linux/mp-r0drv-linux.c
===================================================================
--- VirtualBox-6.0.10.orig/src/VBox/Runtime/r0drv/linux/mp-r0drv-linux.c
+++ VirtualBox-6.0.10/src/VBox/Runtime/r0drv/linux/mp-r0drv-linux.c
@@ -283,12 +283,16 @@ RTDECL(int) RTMpOnAll(PFNRTMPWORKER pfnW
      if (RTCpuSetCount(&OnlineSet) > 1)
      {
          /* Fire the function on all other CPUs without waiting for completion. */
-# if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 3, 0)
+       smp_call_function(rtmpLinuxAllWrapper, &Args, 0 /* wait */);
+#elif LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
          int rc = smp_call_function(rtmpLinuxAllWrapper, &Args, 0 /* wait */);
-# else
+#else
          int rc = smp_call_function(rtmpLinuxAllWrapper, &Args, 0 /* retry */, 
0 /* wait */);
-# endif
+#endif
+# if LINUX_VERSION_CODE < KERNEL_VERSION(5, 3, 0)
          Assert(!rc); NOREF(rc);
+#endif
      }
  #endif

@@ -326,7 +330,9 @@ RTDECL(int) RTMpOnOthers(PFNRTMPWORKER p
  {
  #ifdef CONFIG_SMP
      IPRT_LINUX_SAVE_EFL_AC();
+#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 3, 0)
      int rc;
+#endif
      RTMPARGS Args;

      RTTHREADPREEMPTSTATE PreemptState = RTTHREADPREEMPTSTATE_INITIALIZER;
@@ -337,14 +343,18 @@ RTDECL(int) RTMpOnOthers(PFNRTMPWORKER p
      Args.cHits = 0;

      RTThreadPreemptDisable(&PreemptState);
-# if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 3, 0)
+    smp_call_function(rtmpLinuxWrapper, &Args, 1 /* wait */);
+#elif LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
      rc = smp_call_function(rtmpLinuxWrapper, &Args, 1 /* wait */);
-# else /* older kernels */
+#else /* older kernels */
      rc = smp_call_function(rtmpLinuxWrapper, &Args, 0 /* retry */, 1 /* wait */);
-# endif /* older kernels */
+#endif /* older kernels */
      RTThreadPreemptRestore(&PreemptState);

+#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 3, 0)
      Assert(rc == 0); NOREF(rc);
+#endif
      IPRT_LINUX_RESTORE_EFL_AC();
  #else
      RT_NOREF(pfnWorker, pvUser1, pvUser2);

Change History

comment:1 Changed 2 months ago by fbatschu

  • Owner set to fbatschu
  • Status changed from new to accepted
  • Summary changed from Fixes for Kernel 5.3 wanted - part 1 to Fixes for Linux kernel 5.3 wanted - part 1

comment:2 Changed 2 months ago by fbatschu

  • Keywords linux kernel 5.3 added

comment:3 Changed 2 months ago by fbatschu

the compilation errors we get leading to this bug are in detail:

On Tue, 20 Aug 2019 04:16:52 +0200, George R Goffe <> wrote:

Fedora Core seems to have just transitioned from FC (rawhide)(fc31) to FC (rawhide)(fc32). This problem exists with both FC31 and FC32... BOTH are rawhide and BOTH come from the Fedora rawhide repositories. My current kernel is: uname -a Linux fc31 5.3.0-0.rc4.git1.1.fc32.x86_64 #1 SMP Wed Aug 14 15:05:26 UTC 2019 x86_64 GNU/Linux On Mon, 19 Aug 2019 14:14:37 +0200, George R Goffe via VBox-users-community <vbox-users-community@…> wrote:

I've seen this problem for the past few weeks but have been waiting to see if it gets fixed. Sadly, it has not. I have tried to fix the problem and got these messages resolved but further into the build another failure appears. Obviously the kernel has changed... NOT so obvious is what the change is attempting to accomplish or what VB needs to do to resolve the problem.

>> ./tools/objtool/objtool orc generate  --module --no-fp --retpoline 
>> --uaccess /tmp/vbox.0/r0drv/linux/initterm-r0drv-linux.o
>> /tmp/vbox.0/r0drv/linux/mp-r0drv-linux.c: In function 
>> ‘VBoxHost_RTMpOnAll’:
>> /tmp/vbox.0/r0drv/linux/mp-r0drv-linux.c:287:18: error: void value not 
>> ignored as it ought to be
>> 287 |        int rc = smp_call_function(rtmpLinuxAllWrapper, &Args, 0/*  
>> wait */);
>> |                  ^~~~~~~~~~~~~~~~~
>> /tmp/vbox.0/r0drv/linux/mp-r0drv-linux.c: In function 
>> ‘VBoxHost_RTMpOnOthers’:
>> /tmp/vbox.0/r0drv/linux/mp-r0drv-linux.c:341:8: error: void value not 
>> ignored as it ought to be
>> 341 |    rc = smp_call_function(rtmpLinuxWrapper, &Args, 1 /* wait */);
>> |        ^
>>  
>>  
>> /tmp/vbox.0/linux/VBoxNetFlt-linux.c: In function 
>> ‘vboxNetFltLinuxEnumeratorCallback’:
>> /tmp/vbox.0/linux/VBoxNetFlt-linux.c:2126:9: error: implicit declaration 
>> of function ‘for_ifa’ [-Werror=implicit-function-declaration]
>> 2126 |        for_ifa(in_dev) {
>> |        ^~~~~~~
>> /tmp/vbox.0/linux/VBoxNetFlt-linux.c:2126:24: error: expected ‘;’ before 
>> ‘{’ token
>> 2126 |        for_ifa(in_dev) {
>  
>> |                        ^~
Last edited 2 months ago by fbatschu (previous) (diff)

comment:4 Changed 2 months ago by fbatschu

trunk/src/VBox/Runtime/r0drv/linux/mp-r0drv-linux.c

259 RTDECL(int) RTMpOnAll(PFNRTMPWORKER pfnWorker, void *pvUser1, void *pvUser2)                                                       
[...]
285         /* Fire the function on all other CPUs without waiting for completion. */                                                  
286 # if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)                                                                                
287         int rc = smp_call_function(rtmpLinuxAllWrapper, &Args, 0 /* wait */);                                                      
288 # else                                                                                                                             
289         int rc = smp_call_function(rtmpLinuxAllWrapper, &Args, 0 /* retry */, 0 /* wait */);                                       
290 # endif   

### in the latest mainline kernel we find:

 https://github.com/torvalds/linux/blob/master/kernel/smp.c
 https://elixir.bootlin.com/linux/v5.3-rc7/source/kernel/smp.c

/**
 * smp_call_function(): Run a function on all other CPUs.
 * @func: The function to run. This must be fast and non-blocking.
 * @info: An arbitrary pointer to pass to the function.
 * @wait: If true, wait (atomically) until function has completed
 *        on other CPUs.
 *
 * Returns 0.
 *
 * If @wait is true, then returns once @func has returned; otherwise
 * it returns just before the target cpu calls @func.
 *
 * You must not call this function with disabled interrupts or from a
 * hardware interrupt handler or from a bottom half handler.
 */
void smp_call_function(smp_call_func_t func, void *info, int wait)
{
	preempt_disable();
	smp_call_function_many(cpu_online_mask, func, info, wait);
	preempt_enable();
}

### this change started with 5.3-rc1

 https://elixir.bootlin.com/linux/v5.3-rc1/source/kernel/smp.c

### in the latest stable 5.2.12 kernel it still returns an INT:  https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/kernel/smp.c?h=v5.2.12

/**
 * smp_call_function(): Run a function on all other CPUs.
 * @func: The function to run. This must be fast and non-blocking.
 * @info: An arbitrary pointer to pass to the function.
 * @wait: If true, wait (atomically) until function has completed
 *        on other CPUs.
 *
 * Returns 0.
 *
 * If @wait is true, then returns once @func has returned; otherwise
 * it returns just before the target cpu calls @func.
 *
 * You must not call this function with disabled interrupts or from a
 * hardware interrupt handler or from a bottom half handler.
 */
int smp_call_function(smp_call_func_t func, void *info, int wait)
{
	preempt_disable();
	smp_call_function_many(cpu_online_mask, func, info, wait);
	preempt_enable();

	return 0;
}
Last edited 2 months ago by fbatschu (previous) (diff)

comment:5 Changed 2 months ago by fbatschu

The changes to Trunk will be:

Index: src/VBox/HostDrivers/VBoxNetFlt/linux/VBoxNetFlt-linux.c
===================================================================
--- src/VBox/HostDrivers/VBoxNetFlt/linux/VBoxNetFlt-linux.c	(revision 133156)
+++ src/VBox/HostDrivers/VBoxNetFlt/linux/VBoxNetFlt-linux.c	(working copy)
@@ -2123,7 +2123,9 @@
 #endif
     if (in_dev != NULL)
     {
-        for_ifa(in_dev) {
+        struct in_ifaddr *ifa;
+
+        for (ifa = in_dev->ifa_list; ifa; ifa = ifa->ifa_next) {
             if (VBOX_IPV4_IS_LOOPBACK(ifa->ifa_address))
                 return NOTIFY_OK;
 
@@ -2137,7 +2139,7 @@
 
             pThis->pSwitchPort->pfnNotifyHostAddress(pThis->pSwitchPort,
                 /* :fAdded */ true, kIntNetAddrType_IPv4, &ifa->ifa_address);
-        } endfor_ifa(in_dev);
+        }
     }
 
     /*
Index: src/VBox/Runtime/r0drv/linux/mp-r0drv-linux.c
===================================================================
--- src/VBox/Runtime/r0drv/linux/mp-r0drv-linux.c	(revision 133156)
+++ src/VBox/Runtime/r0drv/linux/mp-r0drv-linux.c	(working copy)
@@ -283,12 +283,15 @@
     if (RTCpuSetCount(&OnlineSet) > 1)
     {
         /* Fire the function on all other CPUs without waiting for completion. */
-# if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
+# if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 3, 0)
+        smp_call_function(rtmpLinuxAllWrapper, &Args, 0 /* wait */);
+# elif LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
         int rc = smp_call_function(rtmpLinuxAllWrapper, &Args, 0 /* wait */);
+        Assert(!rc); NOREF(rc);
 # else
         int rc = smp_call_function(rtmpLinuxAllWrapper, &Args, 0 /* retry */, 0 /* wait */);
-# endif
         Assert(!rc); NOREF(rc);
+# endif
     }
 #endif
 
@@ -326,7 +329,6 @@
 {
 #ifdef CONFIG_SMP
     IPRT_LINUX_SAVE_EFL_AC();
-    int rc;
     RTMPARGS Args;
 
     RTTHREADPREEMPTSTATE PreemptState = RTTHREADPREEMPTSTATE_INITIALIZER;
@@ -337,14 +339,17 @@
     Args.cHits = 0;
 
     RTThreadPreemptDisable(&PreemptState);
-# if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
-    rc = smp_call_function(rtmpLinuxWrapper, &Args, 1 /* wait */);
+# if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 3, 0)
+    smp_call_function(rtmpLinuxWrapper, &Args, 1 /* wait */);
+# elif LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
+    int rc = smp_call_function(rtmpLinuxWrapper, &Args, 1 /* wait */);
+    Assert(rc == 0); NOREF(rc);
 # else /* older kernels */
-    rc = smp_call_function(rtmpLinuxWrapper, &Args, 0 /* retry */, 1 /* wait */);
+    int rc = smp_call_function(rtmpLinuxWrapper, &Args, 0 /* retry */, 1 /* wait */);
+    Assert(rc == 0); NOREF(rc);
 # endif /* older kernels */
     RTThreadPreemptRestore(&PreemptState);
 
-    Assert(rc == 0); NOREF(rc);
     IPRT_LINUX_RESTORE_EFL_AC();
 #else
     RT_NOREF(pfnWorker, pvUser1, pvUser2);

comment:6 follow-up: ↓ 10 Changed 2 months ago by joeAverage

ping @fbatschu

two questions:

  1. will it here be readable/noticeable when the patches went into trunk ? currently it isn't.

Don't hurry. I just want to get a sign/hint somehow when it is.

  1. is it / will it be sufficing to only copy the two patched files into an existing tree of the released Vbox 6.0.12_133076 to get modules compiled for kernel 5.3-rc8 ?
Last edited 2 months ago by joeAverage (previous) (diff)

comment:7 Changed 2 months ago by socratis

@joeAverage

For your first question, you can always take a look at the Timeline. Not always up to date, and remember that it's the development builds (6.0.97), not the next 6.0.14 release.

In fact the latest update (as of this writing) in the timeline was done on 2019-07-26...

comment:8 follow-up: ↓ 9 Changed 2 months ago by joeAverage

thanks @socratis for quick answer !

somehow I'm unable to see anything elder then 08/10/2019 in the timeline. start date is to today and then I put 90 days back in. nada, it's ends at 08/10/2019 !

But, to me it's not that much important to get used with timeline.

I was just looking for an error free way to patch the current installed VBox to play somewhat with kernel 5.3-rc8 by just picking the patched files out of your tree and copy it over. (if this will work out ever ?!)

I guess I best start reading "man patch" to figure out what I already can do with comment 5 and how to handle patches ...

comment:9 in reply to: ↑ 8 Changed 2 months ago by socratis

Replying to joeAverage:

somehow I'm unable to see anything elder then 08/10/2019 in the timeline.

I already told you that the oldest update is from 2019-07-26, nothing newer than that as of this writing (2019-09-10). ;)

comment:10 in reply to: ↑ 6 Changed 2 months ago by fbatschu

Replying to joeAverage:

ping @fbatschu

two questions:

  1. will it here be readable/noticeable when the patches went into trunk ? currently it isn't.

Don't hurry. I just want to get a sign/hint somehow when it is.

  1. is it / will it be sufficing to only copy the two patched files into an existing tree of the released Vbox 6.0.12_133076 to get modules compiled for kernel 5.3-rc8 ?

1), yes this bug will be updated with the trunk and 6.0 branch commit revision numbers for reference

2) I don't know yet, I am working on compiling against a 5.3-rc8 kernel release, that is why I haven't committed the changes yet.

comment:11 Changed 2 months ago by joeAverage

@socratis aha I missed that, I'm getting old ! :-)

@fbatschu, okay I need to wait & see.

thanks ${ALL} !!!

comment:12 Changed 2 months ago by fbatschu

I can confirm that the suggested fix enables a successful compile run against the kernel sources from:  https://git.kernel.org/torvalds/t/linux-5.3-rc8.tar.gz

Will go ahead with integration into trunk and backport to the 6.0 branch.

comment:13 Changed 2 months ago by fbatschu

fix has been integrated into trunk:

$svn commit -m "linux kernel 5.3 fixes: tickref:18911, contribution from Larry.Finger at lwfinger.net"
Sending trunk/src/VBox/HostDrivers/VBoxNetFlt/linux/VBoxNetFlt-linux.c
Sending        trunk/src/VBox/Runtime/r0drv/linux/mp-r0drv-linux.c
Transmitting file data ..done
Committing transaction...
Committed revision 133279.

comment:14 Changed 2 months ago by fbatschu

  • Status changed from accepted to closed
  • Resolution set to fixed

comment:15 Changed 2 months ago by fbatschu

backported to the 6.0 branch as revision 133316.

Last edited 2 months ago by fbatschu (previous) (diff)

comment:16 Changed 2 months ago by fbatschu

backported to the 5.2 branch as revision 133319.

comment:17 Changed 2 months ago by BertiN45

I run Ubuntu 19.10 with Linux kernel 5.3 and Virtualbox 6.0.12. Currently booting from Linux 5.2 for Virtualbox. When can I expect a new Virtualbox version?

comment:18 follow-up: ↓ 19 Changed 2 months ago by burdi01

Kernel 5.3 is out now but the above revisions are not publicly available yet ... :(

comment:19 in reply to: ↑ 18 Changed 2 months ago by fbatschu

Replying to burdi01:

Kernel 5.3 is out now but the above revisions are not publicly available yet ... :(

The public source code repo is typically updated whenever a new Vbox release or Beta release has been made available.

comment:20 Changed 2 months ago by burdi01

With "publicly available" I meant the testbuilds, not the sources.

comment:21 Changed 2 months ago by fbatschu

Testbuilds for all branches, trunk, 6.0 and 5.2 have been updated which contain the revisions of this fix pointed out above in the commit comment:

https://www.virtualbox.org/wiki/Testbuilds

Note: See TracTickets for help on using tickets.

www.oracle.com
ContactPrivacy policyTerms of Use