<html><head><style type='text/css'>p { margin: 0; }</style></head><body><div style='font-family: Times New Roman; font-size: 12pt; color: #000000'><br> The thing you're doing "wrong" is that you're trying to implement your own bridge device. No one else does that and the PCI/device code in VirtualBox simply never was architected to support that. Just like it was never anticipated that someone would want to implement devices with no 1:1 PDM device : PCI device relationship.<br><br> This is just something that has never come up before. For all devices VirtualBox currently emulates, bridges don't matter and guest software doesn't care. Bridges in VirtualBox only serve to overcome the number of devices per PCI bus limit.<br><br> You either need to restructure your approach to fit what VirtualBox actually supports, or modify VirtualBox.<br><br> Michal<br><br>----- Original Message -----<br>From: ric_vilbig@mentor.com<br>To: klaus.espenlaub@oracle.com, vbox-dev@virtualbox.org<br>Sent: Thursday, April 7, 2016 10:47:46 PM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna<br>Subject: Re: [vbox-dev] VM crash, NS_ERROR_FAILURE<br><br>
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:black;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";
color:black;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:black;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";
color:black;}
span.EmailStyle20
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#002060;}
span.EmailStyle21
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle22
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle23
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div lang="EN-US">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D">Hi Klaus,</span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="color:#1F497D">I am blocked on my MSI issues for the moment, so I went back to look at this SecBus issue. I think I have figured out what is going wrong, and I would not be surprised if it’s also a factor in some of the other
troubles I am having (discussed in other threads). But I am not sure what to do about it and would really appreciate some advice.</span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="color:#1F497D">My device constructor prepares the PCIDevice structure and then calls PDMDevHlpPCIRegister() to register the device. That in turn calls ich9pciRegisterInternal(), which adds my device to the set of bridges,
here:</span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Courier New";color:windowtext"> pBus->papBridgesR3[pBus->cBridges] = pPciDev;</span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Courier New";color:windowtext"> pBus->cBridges++;</span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="color:#1F497D">The problem comes later when VBox calls ich9pciInitBridgeTopology() recursively to enumerate the busses. As it’s walking through the list of bridges, it makes the assumption that achInstanceData in the device
instance is of type ICH9PCIBUS, but for my device it instead contains my own device property structure. This confuses the next deeper recursion layer, and as you correctly guessed, it enters a recursion loop that breaks the stack.</span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Courier New";color:windowtext"> for (uint32_t iBridge = 0; iBridge < pBus->cBridges; iBridge++)</span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Courier New";color:windowtext"> {</span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Courier New";color:windowtext"> PPCIDEVICE pBridge = pBus->papBridgesR3[iBridge];</span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Courier New";color:windowtext"> AssertMsg(pBridge && pciDevIsPci2PciBridge(pBridge),</span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Courier New";color:windowtext"> ("Device is not a PCI bridge but on the list of PCI bridges\n"));</span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Courier New";color:windowtext">
</span><span style="font-size:8.0pt;font-family:"Courier New";color:#C00000">PICH9PCIBUS pChildBus = PDMINS_2_DATA(pBridge->pDevIns, PICH9PCIBUS);</span><span style="font-size:8.0pt;font-family:"Courier New";color:windowtext"></span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Courier New";color:windowtext"> pGlobals->uBus++;</span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Courier New";color:windowtext">
</span><span style="font-size:8.0pt;font-family:"Courier New";color:#C00000">ich9pciInitBridgeTopology</span><span style="font-size:8.0pt;font-family:"Courier New";color:windowtext">(pGlobals,
</span><span style="font-size:8.0pt;font-family:"Courier New";color:#C00000">pChildBus</span><span style="font-size:8.0pt;font-family:"Courier New";color:windowtext">, uBusSecondary, pGlobals->uBus);</span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Courier New";color:windowtext"> }</span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="color:#1F497D">It seems that if I want to register my device as a bridge, then I need my achInstanceData buffer to contain an ICH9PCIBUS struct. I thought I could just add this to the top of my own device properties structure,
but it’s declared in DevPciIch9.cpp and therefore not available to me.</span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="color:#1F497D">Clearly I am doing something wrong in the way I register my bridge, so what is the right way to register a bridge such that its achInstanceData buffer contains an ICH9PCIBUS struct and also my own device properties
as well?</span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="color:#1F497D">// RicV</span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext"> vbox-dev-bounces@virtualbox.org [mailto:vbox-dev-bounces@virtualbox.org]
<b>On Behalf Of </b>Vilbig, Ric<br>
<b>Sent:</b> Saturday, April 02, 2016 10:11<br>
<b>To:</b> Klaus Espenlaub; vbox-dev@virtualbox.org<br>
<b>Subject:</b> Re: [vbox-dev] VM crash, NS_ERROR_FAILURE</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><span style="color:#1F497D">Thank you Klaus. I will return to this SecBus issue after I get past the MSI issue that I am having, and this information will certainly be helpful. At the moment I have a work around (hack) for the SecBus
problem which seems to be working, whereas the MSI problem has me blocked. Once I get past that, I should have time to figure out a proper solution for the SecBus problem.</span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="color:#1F497D">// RicV</span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext"> Klaus Espenlaub [<a href="mailto:klaus.espenlaub@oracle.com" target="_blank">mailto:klaus.espenlaub@oracle.com</a>]
<br>
<b>Sent:</b> Friday, April 01, 2016 08:51<br>
<b>To:</b> <a href="mailto:vbox-dev@virtualbox.org" target="_blank">vbox-dev@virtualbox.org</a><br>
<b>Subject:</b> Re: [vbox-dev] VM crash, NS_ERROR_FAILURE</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<p class="MsoNormal" style="margin-bottom:12.0pt">Hi Ric,</p>
<div>
<p class="MsoNormal">On 30.03.2016 02:02, Vilbig, Ric wrote:</p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="color:#1F497D">Hi,</span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="color:#1F497D">I obviously carried on with my investigation after sending the original email, and have figured out what is triggering this abort (not really fair to call it a crash).</span></p>
</blockquote>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:12.0pt;font-family:"Times New Roman","serif"">VBox.log actually is showing that the VM was never fully powered up. So the crash happens before the CPU started executing instructions. See
below, I know that this doesn't make much sense to you.</span></p>
<p class="MsoNormal"><span style="color:#1F497D">When the BIOS starts initializing the PCI Configuration space for my PCIe switch, it reads the secondary bus register (PCI CFG 0x19) before it’s been initialized, so the device model is returning 0. This puts
the BIOS into a loop, repeating the following over 5000 times before aborting the VM session.</span></p>
<p class="MsoNormal"><span style="color:#002060"> </span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt;font-family:"Courier New"">PCI CFG Root Rd 0x0a L 2 = 0x0604 // Class</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt;font-family:"Courier New"">PCI CFG Root Rd 0x00 L 2 = 0x14ab // VendID</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt;font-family:"Courier New"">PCI CFG Root Rd 0x02 L 2 = 0x1000 // DevID</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt;font-family:"Courier New"">PCI CFG Root Wr 0x1c L 1 = 0xd0 // IOBase</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt;font-family:"Courier New"">PCI CFG Root Wr 0x20 L 2 = 0xf000 // MemBase</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt;font-family:"Courier New"">PCI CFG Root Rd 0x19 L 1 =
</span><b><span style="font-size:8.0pt;font-family:"Courier New";color:#C00000">0x00</span></b><span style="font-size:8.0pt;font-family:"Courier New""> // SecBus</span></p>
<p class="MsoNormal"><span style="color:#002060"> </span></p>
<p class="MsoNormal"><span style="color:#002060">If I intercept the secondary bus register read, and return a 3 instead of reading 0 from RTL, then it carries on with root configuration and my VM boots and runs correctly. It’s not detecting the downstream
end point, but that is a separate issue.</span></p>
<p class="MsoNormal"><span style="color:#002060"> </span></p>
<p class="MsoNormal"><span style="color:#002060">Meanwhile, does it make sense for the BIOS to read the secondary bus register
</span><span style="color:#1F497D">before it’s been initialized? It seems like that register should be set up as the BIOS proceeds through the enumeration. That is what the VM with PIIX3 chipset does.</span></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:12.0pt;font-family:"Times New Roman","serif"">It does, but for a non-obvious reason. VirtualBox pre-configures its PCI devices before it starts the BIOS, especially the bus numbers. Looks
like for some reason this isn't done properly (or not making it correctly to your PCIe switch). This confuses the code, most likely causing endless recursion and thus a stack overflow. You should be able to use a debugger on the VM process to find out the
detail, because this is all normal userland code on the host - which wouldn't work if it's BIOS code running inside the VM.<br>
<br>
The motivation for moving the PCI bus configuration out of the BIOS is to some extent historic (in the old days we always fought with the BIOS size restriction, due to the extremely bad code quality by the BCC compiler), to some extent an optimization (it's
far easier and more efficient to do the hairy stuff in 32 bit code on the host, and not in in the actual BIOS, which is annoying 16 bit real mode code).<br>
<br>
Klaus</span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="color:#002060">_____________________________________________</span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;color:#002060"> </span></b></p>
<p class="MsoNormal"><b><span style="color:#002060">Ric Vilbig</span></b></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#002060">Mentor Graphics, Emulation Division</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#002060">46871 Bayside Parkway, Fremont CA, 94538</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#002060">Phone: 510-354-7360</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#002060">Mobile: 408-529-2365</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#002060">email: <a href="mailto:ric_vilbig@mentor.com" target="_blank">
ric_vilbig@mentor.com</a></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#002060"> </span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="color:#002060"> </span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Vilbig, Ric
<br>
<b>Sent:</b> Tuesday, March 29, 2016 11:40<br>
<b>To:</b> <a href="mailto:vbox-dev@virtualbox.org" target="_blank">vbox-dev@virtualbox.org</a><br>
<b>Cc:</b> Vilbig, Ric<br>
<b>Subject:</b> VM crash, NS_ERROR_FAILURE</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><span style="color:#002060">Hi experts,</span></p>
<p class="MsoNormal"><span style="color:#002060"> </span></p>
<p class="MsoNormal"><span style="color:#002060">I would like to ask for some help to figure out why a certain VM crashes on start-up. Although the problem is evidently induced by my PDM plug-in, the crash does not appear to be happening therein. I need some
help to root cause where VBox is aborting the VM session.</span></p>
<p class="MsoNormal"><span style="color:#002060"> </span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Courier New"">> VBoxManage startvm "U14_ICH9_2"</span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Courier New"">Waiting for VM "U14_ICH9_2" to power on...</span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Courier New"">VBoxManage: error: The VM session was aborted</span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Courier New"">VBoxManage: error: Details: code NS_ERROR_FAILURE (0x80004005), component SessionMachine, interface ISession</span></p>
<p class="MsoNormal"><span style="color:#002060"> </span></p>
<p class="MsoNormal"><span style="color:#002060">I created this VM from the VirtualBox GUI, v5.0.16, which I built from the tarball at
<a href="https://www.virtualbox.org/wiki/Downloads" target="_blank">https://www.virtualbox.org/wiki/Downloads</a> and am running on an Ubuntu 14 host. Then I switched the chipset to ICH9, then I installed Ubuntu 14 as the guest. The VM runs well, until I plug my virtual
device model into PDM (it’s a PCIe switch with downstream endpoint). After plugging in my virtual device, the VM crashes as shown above.</span></p>
<p class="MsoNormal"><span style="color:#002060"> </span></p>
<p class="MsoNormal"><span style="color:#002060">I tracked down everywhere NS_ERROR_FAILURE is mentioned in the sources. I found that DirectoryServiceProvider::GetFile() returns that error twice, right away, but that is also true in the working case when my
device is unplugged. In no other place is that specific error ever returned or asserted. However, I found that E_FAIL is #defined to NS_ERROR_FAILURE, and there are hundreds of references to E_FAIL, so I gave up trying to instrument them all.</span></p>
<p class="MsoNormal"><span style="color:#002060"> </span></p>
<p class="MsoNormal"><span style="color:#002060">I need some help to root cause this problem. Log files show that it is getting as far as BIOS starting to initialize the switch, apparently stuck in a loop doing that, but then lights out with no trail that
I can follow.</span></p>
<p class="MsoNormal"><span style="color:#002060"> </span></p>
<p class="MsoNormal"><span style="color:#002060">Log files are attached. Lines bearing the “RicV” prefix were instrumented by me to investigate this problem. Lines bearing the “RemDev” prefix are coming from my PDM plug-in.</span></p>
<p class="MsoNormal"><span style="color:#002060"> </span></p>
<p class="MsoNormal"><span style="color:#002060">Thanks,</span></p>
<p class="MsoNormal"><span style="color:#002060"> </span></p>
<p class="MsoNormal"><span style="color:#002060">_____________________________________________</span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;color:#002060"> </span></b></p>
<p class="MsoNormal"><b><span style="color:#002060">Ric Vilbig</span></b></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#002060">Mentor Graphics, Emulation Division</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#002060">46871 Bayside Parkway, Fremont CA, 94538</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#002060">Phone: 510-354-7360</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#002060">Mobile: 408-529-2365</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#002060">email: <a href="mailto:ric_vilbig@mentor.com" target="_blank">
ric_vilbig@mentor.com</a></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#002060"> </span></p>
<p class="MsoNormal"> </p>
</div>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman","serif""> </span></p>
</div>
</div>
</div>
</div>
</div></body></html>