That doesn't look right, looks like something is holding vboxnet0 referenced.
Anyway, you can forcefully unload the drivers after kicking out vboxnetX.
Try this as root:
ifconfig vboxnet0 unplumb /opt/VirtualBox/vboxconfig.sh --preremove