Solaris guest kernel panic with ctf_modopen

[peter.dennis@oracle.com]

Solaris guest debug kernel panic:

panic[cpu0]/thread=ffffff00ad234100: 
assertion failed: MUTEX_HELD(&mod_lock), file: ../../common/ctf/ctf_mod.c, line:
 157



> $C                                  
ffffff00029fe5a0 vpanic()
ffffff00029fe5e0 assfail+0x89(fffffffff8232dc0, fffffffff8232de8, 9d)
ffffff00029fe6e0 ctf_modopen+0x19a(fffffffffbcb8a58, ffffff00029fe6fc)
ffffff00029fe710 vbi_init+0x131()
ffffff00029fe730 rtR0InitNative+0x15()
ffffff00029fe760 RTR0Init+0x28()
ffffff00029fe790 _init+0x13()
ffffff00029fe7c0 modinstall+0x11e(ffffff00ae42fa10)
ffffff00029fe810 mod_hold_installed_mod+0x84(ffffff00abf6d968, 1, 0, ffffff00029fe82c)
ffffff00029fe870 modrload+0xd1(fffffffffbfe8640, ffffff00a5d8c260, 0)
ffffff00029fe890 modload+0x18(fffffffffbfe8640, ffffff00a5d8c260)
ffffff00029fe8d0 mod_hold_dev_by_major+0x94(11b)
ffffff00029fe8f0 ndi_hold_driver+0x2a(ffffff00a6196008)
ffffff00029fe920 probe_node+0x3b(ffffff00a6196008)
ffffff00029fe970 i_ndi_config_node+0x136(ffffff00a6196008, 6, 0)
ffffff00029fe9a0 i_ddi_attachchild+0x6e(ffffff00a6196008)
ffffff00029fe9e0 devi_attach_node+0x10e(ffffff00a6196008, 4000)
ffffff00029fea40 config_immediate_children+0xf5(ffffff00a6197748, 4000, ffffffff)
ffffff00029fea90 devi_config_common+0xa6(ffffff00a6197748, 4000, ffffffff)
ffffff00029fead0 ndi_devi_config+0x3a(ffffff00a6197748, 4000)
ffffff00029fec20 dv_filldir+0x7a(ffffff00aa9e5bc0)
ffffff00029fed80 devfs_readdir+0x113(ffffff00ab3a6400, ffffff00029fee40, ffffff00a70f9008, ffffff00029fee2c, 0, 0)
ffffff00029fee10 fop_readdir+0xe8(ffffff00ab3a6400, ffffff00029fee40, ffffff00a70f9008, ffffff00029fee2c, 0, 0)
ffffff00029feeb0 getdents64+0xd5(3, fede4000, 2000)
ffffff00029fef00 _sys_sysenter_post_swapgs+0x237()
> ffffff00abf6d968/s
0xffffff00abf6d968:             drv/vboxguest

The problem is that the vboxguest driver is calling the ctf_modopen without the mod_lock mutex being held. This is a problem in the source code of

root/trunk/src/VBox/Runtime/r0drv/solaris/vbi/i86pc/os/vbi.c

It does a mod_hold_by_name() which sets the mod_busy but does not hold the mutex: mod_lock