[vbox-dev] Unlink of /dev/null on OS X 10.6 During Build

[Chris Perl]

Hey Everyone,

I'm running MacOS 10.6 Snow Leopard and trying to build Virtual Box
from an SVN checkout (r36702).  It appears to me that the build system
attempts to unlink /dev/null.  This could be a personal problem, but I
wanted to see if anyone else could reproduce this behavior.

In one window, I configured the build like so:

$ ./configure --disable-qt4 --with-openssl-dir=/opt/local
$ source env.sh

while in another I started the following DTrace script:

#!/usr/sbin/dtrace -ws

syscall::unlink:entry
/copyinstr(arg0) == "/dev/null"/
{
        stop();
        system("ps -f -p %d\n", pid);
        system("echo continue | gdb -batch -x /dev/stdin %s %d
>/dev/null 2>&1", execname, pid);
}

Note that in the above script, I used gdb simply to let the process
continue after stopping it.  I'm a Solaris sysadmin and couldn't find
the equivalent of "prun" on MacOS X. ;)

I next invoked the build system with "kmk -j 1" in my original window.
 Very shortly thereafter, I got the following output in my window with
the DTrace script running:

$ sudo ./devnull.d
dtrace: script './devnull.d' matched 1 probe
dtrace: allowing destructive actions
CPU     ID                    FUNCTION:NAME
  0  18659                     unlink:entry Process id: 41721,
attempting unlink of: /dev/null
  UID   PID  PPID   C     STIME TTY           TIME CMD
  501 41721 41717   0   0:00.00 ttys002    0:00.00
/usr/libexec/gcc/i686-apple-darwin10/4.0.1/collect2 -dynamic -arch
i386 -macosx_version_min 10.6.7 -weak_reference_mismatches non-weak -o
/dev/null -L/usr/lib/i686-apple-darwin10/4.0.1
-L/usr/lib/gcc/i686-apple-darwin10/4.0.1
-L/usr/lib/gcc/i686-apple-darwin10/4.0.1
-L/usr/lib/gcc/i686-apple-darwin10/4.0.1/../../../i686-apple-darwin10/4.0.1
-L/usr/lib/gcc/i686-apple-darwin10/4.0.1/../../.. --as-needed -u start
-undefined dynamic_lookup
/var/folders/CY/CYgaS5vOHuqRxHzIrPmHf++++TI/-Tmp-//ccqV1rrD.o

  0  18659                     unlink:entry Process id: 41764,
attempting unlink of: /dev/null
  UID   PID  PPID   C     STIME TTY           TIME CMD
  501 41764 41763   0   0:00.00 ttys002    0:00.00
/usr/libexec/gcc/i686-apple-darwin10/4.0.1/ld -dynamic -arch i386
-macosx_version_min 10.6.7 -weak_reference_mismatches non-weak -o
/dev/null -L/usr/lib/i686-apple-darwin10/4.0.1
-L/usr/lib/gcc/i686-apple-darwin10/4.0.1
-L/usr/lib/gcc/i686-apple-darwin10/4.0.1
-L/usr/lib/gcc/i686-apple-darwin10/4.0.1/../../../i686-apple-darwin10/4.0.1
-L/usr/lib/gcc/i686-apple-darwin10/4.0.1/../../.. -no_compact_linkedit
-u start -undefined dynamic_lookup
/var/folders/CY/CYgaS5vOHuqRxHzIrPmHf++++TI/-Tmp-//ccc7V19w.o

So it would appear that "collect2" and "ld" attempt to unlink
"/dev/null".  I tracked this down to lines 1947 through 1962 of
Config.kmk where "-o /dev/null" is passed as a parameter to the gcc
tests (I can provide a patch if you want).

As a simpler example, I created the following stupid program that does nothing:

#include <stdio.h>

int main()
{
	return 0;
}

and then compiled it with with "gcc -o /dev/null t.c" while running
the same DTrace script above, and saw this:

$ sudo ./devnull.d
dtrace: script './devnull.d' matched 1 probe
dtrace: allowing destructive actions
CPU     ID                    FUNCTION:NAME
0  18659                     unlink:entry Process id: 42062,
attempting unlink of: /dev/null
  UID   PID  PPID   C     STIME TTY           TIME CMD
  501 42062 42061   0   0:00.00 ttys007    0:00.01
/usr/libexec/gcc/i686-apple-darwin10/4.2.1/ld -dynamic -arch x86_64
-macosx_version_min 10.6.7 -weak_reference_mismatches non-weak -o
/dev/null -lcrt1.10.6.o
-L/usr/lib/gcc/i686-apple-darwin10/4.2.1/x86_64
-L/usr/lib/gcc/i686-apple-darwin10/4.2.1/x86_64
-L/usr/lib/i686-apple-darwin10/4.2.1
-L/usr/lib/gcc/i686-apple-darwin10/4.2.1
-L/usr/lib/gcc/i686-apple-darwin10/4.2.1
-L/usr/lib/gcc/i686-apple-darwin10/4.2.1/../../../i686-apple-darwin10/4.2.1
-L/usr/lib/gcc/i686-apple-darwin10/4.2.1/../../..
/var/folders/CY/CYgaS5vOHuqRxHzIrPmHf++++TI/-Tmp-//cc2W6ETH.o -lSystem
-lgcc -lSystem

It looks like at least some utilities invoked by the GCC compiler
driver on MacOS 10.6 attempt to unlink their output file before
writing to it.

This just fails silently when run as a normal user, but it actually
removes /dev/null when run as root.  I know that one should never
compile software as root, and I basically never do.  But I ran into
this problem when attempting to build and install VirtualBox via a
port with macports.  Macports by default builds all the software as
root unless you configure it not to.

As I said at the beginning, this could be a personal problem, but
wanted throw it out there and see if anyone else could reproduce it.

Chris