[vbox-dev] 's/-fPIC/-fPIC -pie/' Config.kmk -- Linking with -pie for ASLR support

[Frank Mehnert]

Hi,

On Friday 05 June 2015 08:38:14 Moez Roy wrote:
> It doesn't matter if the kernel modules cannot be compiled with PIC,
> right? Because the kernel modules are compiled using DKMS when you
> install VBox. So then why is there a -fno-PIC in various locations in
> the Config.kmk file?
> 
> Why is -fPIC in the LDFLAGS when there is already -fPIC in the CFLAGS
> & CXXFLAGS?
> 
> I am trying to build VirtualBox with ASLR support so it passes the
> checksec test.
> 
> I did: sed -i 's/-fPIC/-fPIC -pie/' Config.kmk
> 
> 
> However this causes the build to fail (also happens with v4.3.28) :

I don't think this is the right way. AFAICS -pie does only make sense
for executables, not for shared libraries. And the code which is
linked with -pie should be compiled with -fpie -pie or with -fPIE -pie.
See the gcc manpage:

  -pie
    Produce a position independent executable on targets that support
    it. For predictable results, you must also specify the same set of
    options used for compilation (-fpie, -fPIE, or model suboptions)
    when you specify this linker option.

> kmk: *** [/out/linux.amd64/release/obj/VBoxRTImp/VBoxRTImp.so]
> Error 1
> The failing command:
> @g++ -shared               '-Wl,-rpath,/opt/VirtualBox' -fPIC -pie
> -Wl,-z,noexecstack,-z,relro -Wl,--as-needed -m64   -o
> /out/linux.amd64/release/obj/VBoxRTImp/VBoxRTImp.so
> -Wl,-soname=VBoxRT.so
> /out/linux.amd64/release/obj/VBoxRTImp/gen/VBoxRTImpImp.o
>     -lpthread   -lm   -lrt   -ldl
> /usr/lib/gcc/x86_64-redhat-linux/4.9.2/../../../../lib64/Scrt1.o: In
> function `_start':
> (.text+0x20): undefined reference to `main'
> collect2: error: ld returned 1 exit status
> kmk: *** [/out/linux.amd64/release/obj/VMMR3Imp/VMMR3Imp.so]
> Error 1
> The failing command:
> @g++ -shared               '-Wl,-rpath,/opt/VirtualBox' -fPIC -pie
> -Wl,-z,noexecstack,-z,relro -Wl,--as-needed -m64   -o
> /out/linux.amd64/release/obj/VMMR3Imp/VMMR3Imp.so
> -Wl,-soname=VBoxVMM.so
> /out/linux.amd64/release/obj/VMMR3Imp/gen/VMMR3ImpImp.o
>     -lpthread   -lm   -lrt   -ldl
> kmk: *** Exiting with status 2
> 
> Another attempt: sed -i 's/-fPIC/-shared -fPIC -pie/' Config.kmk
> 
> kmk_builtin_append -n
> "/media/sf_trunk/56248/out/linux.amd64/release/obj/VBoxRTImp/gen/VBoxRTImpIm
> p.o.dep" ""
> "/media/sf_trunk/56248/out/linux.amd64/release/obj/VBoxRTImp/VBoxRTImpImp.c
> :" ""
> kBuild: Linking VBoxRTImp =>
> /media/sf_trunk/56248/out/linux.amd64/release/obj/VBoxRTImp/VBoxRTImp.so
> kmk_builtin_rm -f --
> /media/sf_trunk/56248/out/linux.amd64/release/obj/VBoxRTImp/VBoxRTImp.dep
> /media/sf_trunk/56248/out/linux.amd64/release/obj/VBoxRTImp/VBoxRTImp.so
>  /media/sf_trunk/56248/out/linux.amd64/release/obj/VBoxRTImp/VBoxRTImp.map
> g++ -shared               '-Wl,-rpath,/opt/VirtualBox' -shared -fPIC
> -pie  -Wl,-z,noexecstack,-z,relro -Wl,--as-needed -m64   -o
> /media/sf_trunk/56248/out/linux.amd64/release/obj/VBoxRTImp/VBoxRTImp.so
> -Wl,-soname=VBoxRT.so
> /media/sf_trunk/56248/out/linux.amd64/release/obj/VBoxRTImp/gen/VBoxRTImpImp
> .o -lpthread   -lm   -lrt   -ldl
> /usr/lib/gcc/x86_64-redhat-linux/4.9.2/../../../../lib64/Scrt1.o: In
> function `_start':
> (.text+0x20): undefined reference to `main'
> collect2: error: ld returned 1 exit status
> kmk: ***
> [/media/sf_trunk/56248/out/linux.amd64/release/obj/VBoxRTImp/VBoxRTImp.so]
> Error 1

VBoxRTImp.so is a shared library.

> PS. I have managed to get VirtualBox to build with -Wl,-z,now (FULL
> RELRO) but I cannot enable PIE on VBoxSVC and VBoxXPCOMIPCD:
> 
> checksec --proc-all
> 
> VBoxXPCOMIPCD  19080 Full RELRO        No canary found        NX
> enabled    No PIE
>          VBoxSVC  19087 Full RELRO        Canary found           NX
> enabled    No PIE
> 
> Any help in getting VirtualBox to build with ASLR support appreciated.

We didn't try this yet. Originally most of our code was not even compiled
with -fPIC because position independent code is slower, in particular on
32-bit hosts and the benefit from saving some memory because the .text
segment can be shared between multiple processes is negligible in comparison
to the memory required by the VM process.

There must be some rules when a compiler / linker flag should be used to
satisfy checksec.sh. For example is -fpie enough or is -fPIE required.

Kind regards,

Frank
-- 
Dr.-Ing. Frank Mehnert | Software Development Director, VirtualBox
ORACLE Deutschland B.V. & Co. KG | Werkstr. 24 | 71384 Weinstadt, Germany

ORACLE Deutschland B.V. & Co. KG
Hauptverwaltung: Riesstraße 25, D-80992 München
Registergericht: Amtsgericht München, HRA 95603

Komplementärin: ORACLE Deutschland Verwaltung B.V.
Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697
Geschäftsführer: Alexander van der Ven, Astrid Kepper, Val Maher