[vbox-dev] ICH9 BAR5 access
Ananth Pallapothu
apallapothu at gmail.com
Mon Aug 31 14:07:52 GMT 2015
Hello All,
After quite a few exchanges with Michal and from several experimental
runs, we identified that "fakeBIOS" in ICH9 is not re-configuring
"passthrough" device as seen from guest debugger.
Reasons for this claim, memory registrations of all BAR's (incl. BAR5)
in my "passthrough" device are as expected in GRUB as I have configured.
BAR0 = 128 M, BAR2 = 2M, BAR4 = 256, BAR5=256K
With ICH9, after the GRUB and post complete "Ubuntu" boot, I NO longer
see BAR mapping.
Compared with PIIX3, where I continue to see all registrations intact
post complete bootup (as was seen in GRUB).
Can someone please fill me up.
Thanks.
On Thu, Aug 20, 2015 at 9:46 AM, Ananth Pallapothu <apallapothu at gmail.com>
wrote:
> Hi Nikolay,
>
> I think that's exactly what's happening. I am yet to work on the
> skeleton. But from conversations with Michal and looking through the guest
> debugger and now with your statement, here is what I interpret
>
> My code uses BAR0, BAR2, BAR4, BAR5, Extended ROM, config registers,
> settings done using PCIDevSetDWord. BAR5 (odd numbered) didn't have any
> problem in PIIX3 since there was no notion of 64 bit BAR's but ICH9 has.
> Does ICH9 support 32 bit BAR address mapping as well ?
>
> VM debugger "info phys" never showed my "Pass through" device, but
> 'info pci" showed same address mapping as my driver code (lspci), other
> interesting part is, when I typed "info mmio", there is NO memory mapping
> for BAR5. I see 2 segments each of 0x4000000 for BAR0 and then for BAR2 but
> NOTHING for BAR5. That probably explains why guest OS segfaulted when I
> tried to access BAR5 address ?
>
> Can I mix 64 bit address map for BAR0 (BAR0 + BAR1), BAR2 (BAR2 +
> BAR3) and then 32 bit mapping for BAR4, BAR5 ?
>
> Thanks.
>
>
>
>
>
>
>
> On Thu, Aug 20, 2015 at 3:57 AM, Nikolay Igotti <igotti at gmail.com> wrote:
>
>> Ananth,
>>
>> Does your code use 64-bit BARs? In such a case, odd numbered BARs will
>> be technically unaccessible.
>> PIIX3 doesn't use 64-bit BARs, so it will not be a problem here.
>>
>> Nikolay
>>
>> On Wed, Aug 19, 2015 at 5:22 PM, Ananth Pallapothu <apallapothu at gmail.com
>> > wrote:
>>
>>> Hi Michal,
>>>
>>> Not to side track every one. Here is the output of lspci as listed by VM
>>> debugger
>>>
>>> VBoxDbg> info phys
>>> RAM ranges (pVM=00002aaab17d7000)
>>> GC Phys Range pvHC
>>> 0000000000000000-000000009fffffff 0000000000000000 Base RAM
>>> 00000000b0000000-00000000b1ffffff 00002aaaba9ac000 VRam
>>> 00000000c0000000-00000000c001ffff 0000000000000000 E1000
>>> 00000000c0400000-00000000c07fffff 00002aaaba5a8000 VMMDev
>>> 00000000c0800000-00000000c0803fff 00002aaaba9a8000 VMMDev Heap
>>> 00000000c0804000-00000000c0804fff 0000000000000000 USB OHCI
>>> 00000000c8000000-00000000cbffffff 0000000000000000 ZebuPCIe
>>> 00000000cc000000-00000000cfffffff 0000000000000000 ZebuPCIe
>>> 00000000d0000000-00000000d01fffff 0000000000000000 ZebuPCIe
>>> 00000000d0200000-00000000d023ffff 0000000000000000 ZebuPCIe
>>> 00000000d0240000-00000000d0241fff 0000000000000000 AHCI
>>> 00000000fec00000-00000000fec00fff 0000000000000000 I/O APIC Memory
>>> 00000000fee00000-00000000fee00fff 0000000000000000 APIC Memory
>>> 00000000ffff0000-00000000ffffffff 0000000000000000 PC BIOS - 0xffffffff
>>>
>>> apparently ICH9 never sees this passthrough device. "ZebuPCIe" device is
>>> my VGA device name.
>>>
>>> Thanks.
>>>
>>> On Wed, Aug 19, 2015 at 11:05 AM, Michal Necasek <
>>> michal.necasek at oracle.com> wrote:
>>>
>>>>
>>>> Yes, I agree that this does not appear to be MSI related at all. I
>>>> still don't quite understand the utility of working with old source code
>>>> with known bugs. What's more, we can't fix old versions, and you'll almost
>>>> certainly need either us to fix something or tell us what to fix it. But if
>>>> you really want to start with an old version then that's of course your
>>>> choice.
>>>>
>>>> I saw two VGA devices listed in your lspci output. I'm not saying it
>>>> has to cause problems, but it is an untried/untested configuration.
>>>>
>>>> With EFI, you should expect the firmware to initialize the BARs. That
>>>> is the significant difference compared to the "classic" BIOS in VirtualBox.
>>>>
>>>> Comparing DevPciIch9.cpp with DevPCI.cpp is the right thing to do,
>>>> although there are intentional differences. The problem is very likely
>>>> there. I'd suggest tracing ich9pciUpdateMappings() as all BAR changes
>>>> should go through there.
>>>>
>>>> I can also tell you right away that I have no idea how well 64-bit
>>>> BARs work because that's a feature more or less totally unused by the
>>>> VirtualBox devices (it only comes into play with PCI passthrough, which is
>>>> very rarely used). FYI, the device with most BARs (six) that we have is
>>>> AHCI.
>>>>
>>>> I'm afraid I don't have any more specific suggestions because I don't
>>>> know what your code is doing. If you can distill the problem into a
>>>> testcase that we can look at, we'd be happy to.
>>>>
>>>> Regards,
>>>> Michal
>>>>
>>>> ----- Original Message -----
>>>> From: apallapothu at gmail.com
>>>> To: michal.necasek at oracle.com
>>>> Cc: vbox-dev at virtualbox.org
>>>> Sent: Wednesday, August 19, 2015 2:46:55 PM GMT +01:00 Amsterdam /
>>>> Berlin / Bern / Rome / Stockholm / Vienna
>>>> Subject: Re: [vbox-dev] ICH9 BAR5 access
>>>>
>>>> Hi Michal,
>>>>
>>>> I successfully used all BAR's with PIIX3 chipset.
>>>>
>>>> Correct, MSI supporting is the piece forcing me move to ICH9. I
>>>> understand changes have been made to MSI since then, but this particular
>>>> issue doesn't seem to have anything to do with MSI, right? I was looking
>>>> more interms of getting basic functionality of all BAR's on ICH9 working
>>>> first before upgrading myself to latest VirtualBox.
>>>>
>>>> My device is the VGA device. Did you find any duplicate mention
>>>> somewhere ?
>>>>
>>>> I will try with EFI, see if it makes any difference. What should I
>>>> expect with this change ? Can you please recommend expected debug points in
>>>> the source code that first invoke ICH9 BAR5 source code ? VBoxManage
>>>> modifyvm --name VM_name ----firmware efi
>>>>
>>>> I tried to compare basic function calls between ICH9 vs PIIX3, but
>>>> eventually got lost at some point. Did find some similarities using gdb
>>>> EM.cpp -> EMHM.cpp -> HM.cpp -> DevPciIch9.cpp / DevPCI.cpp.
>>>>
>>>> Suggestions / recommendations would really help.
>>>>
>>>> Thanks.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, Aug 19, 2015 at 7:51 AM, Michal Necasek <
>>>> michal.necasek at oracle.com> wrote:
>>>>
>>>>>
>>>>> Hi Ananth,
>>>>>
>>>>> PCIDevSetDWord in the device won't do it, that's expected. It will
>>>>> get rewritten.
>>>>>
>>>>> I don't think VirtualBox supplies any device with so many BARs so
>>>>> it's entirely possible that something isn't working. The trouble is that
>>>>> you have your code + ours, and we only have our code, so it's quite hard
>>>>> for us to debug the problem.
>>>>>
>>>>> One hint... what happens if you switch your VM to use EFI instead of
>>>>> the classic BIOS? I believe EFI should initialize all the PCI resources
>>>>> itself, so you might be able to see what happens even before an OS boots.
>>>>>
>>>>> And two more comments. I see that you have both the VirtualBox VGA
>>>>> device and your own. No one can tell you for certain that that won't cause
>>>>> problems. And you presumably want to use MSIs (otherwise you wouldn't
>>>>> bother with ICH9), but you're using quite old version of VirtualBox. There
>>>>> were several fixes in the MSI handling area in VirtualBox 5.0.
>>>>>
>>>>> Regards,
>>>>> Michal
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>> From: apallapothu at gmail.com
>>>>> To: klaus.espenlaub at oracle.com
>>>>> Cc: vbox-dev at virtualbox.org
>>>>> Sent: Tuesday, August 18, 2015 9:04:05 PM GMT +01:00 Amsterdam /
>>>>> Berlin / Bern / Rome / Stockholm / Vienna
>>>>> Subject: Re: [vbox-dev] ICH9 BAR5 access
>>>>>
>>>>> Hi Klaus,
>>>>>
>>>>> Thanks for the response.
>>>>>
>>>>> In the PCIe config space I set all BAR's to 0,
>>>>> using PCIDevSetDWord .... no address mapping from myside.
>>>>> I first register the region using PDMDevHlpPCIIORegionRegister
>>>>> with respective size.
>>>>> BAR0 = 0x800,0000 128 M
>>>>> BAR5 = 0x4,0000 256K
>>>>> Then I register memory call backs using PDMDevHlpMMIORegister for
>>>>> the size.
>>>>>
>>>>> While experimenting, I played with different base address in
>>>>> DevPciIch9.cpp "pGlobals->uPciBiosMmio = UINT32_C(0xB8000000);" which moved
>>>>> BAR's accordingly.
>>>>>
>>>>> While experimenting, I have tried setting start address for BAR's
>>>>> in PCIe config space, I saw that this address eventually gets overwritten
>>>>> by FakeBIOS and then OS.
>>>>>
>>>>> When I ran lspci on the guest I did not see conflict in address
>>>>> mapping. Attached lspci output. My pass through device shows up under VGA.
>>>>> I have added lspci output of fully functional PIIX3 chipset for
>>>>> comparison.
>>>>>
>>>>> At the end of the attachment, please note the VM debugger output
>>>>> in ICH9 chip set which apparently doesn't see the VGA device.
>>>>>
>>>>> Just to keep my message crisp, with the same device code, in PIIX3
>>>>> I am able to see memory access to the addresses specified in BAR0, BAR5
>>>>> while in ICH9, I only see access coming through in BAR0 and NOT BAR5.
>>>>>
>>>>> Thanks.
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Aug 18, 2015 at 2:25 PM, Klaus Espenlaub <
>>>>> klaus.espenlaub at oracle.com> wrote:
>>>>>
>>>>>> Hi Ananth,
>>>>>>
>>>>>> On 18.08.2015 20:04, Ananth Pallapothu wrote:
>>>>>>
>>>>>> Hello Experts,
>>>>>>
>>>>>> I have added new pass through device to the VirtualBox setup .
>>>>>> Registered all 6 BAR's with appropriate sizes, none exceeding 256M. All
>>>>>> BAR's were set to reflect hardware functionality (assigned 0), eventually
>>>>>> OS programmed base address regions with the sizes I have registered them
>>>>>> with.
>>>>>>
>>>>>> Can you provide more details (how you map the regions initially and
>>>>>> on remap)? Hard to say what's happening otherwise.
>>>>>>
>>>>>> My fundamental problem is, I have been able to access BAR0 and NOT
>>>>>> BAR5. I tried playing with BIOSmmio address setting in DevPciIch9, that
>>>>>> made difference in relocating BAR's but functionality remained the same
>>>>>> (Not able to access BAR5)
>>>>>>
>>>>>> Interesting part is, PIIX3 chipset is fully functional ie., can
>>>>>> access BAR5 in PIIX3 but NOT in ICH9.
>>>>>>
>>>>>> Could be that with PIIX3 there's no remap and with ICH9 there's a
>>>>>> remap... really difficult to say anything with the sparse information.
>>>>>>
>>>>>> Klaus
>>>>>>
>>>>>>
>>>>>> PS: There seems to be some problem with my email ID, so not sure
>>>>>> whether my last email came through. Sorry if you have already received it.
>>>>>>
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> vbox-dev mailing list
>>>>>> vbox-dev at virtualbox.org
>>>>>> https://www.virtualbox.org/mailman/listinfo/vbox-dev
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>> _______________________________________________
>>> vbox-dev mailing list
>>> vbox-dev at virtualbox.org
>>> https://www.virtualbox.org/mailman/listinfo/vbox-dev
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.virtualbox.org/pipermail/vbox-dev/attachments/20150831/9c88bee3/attachment.html>
More information about the vbox-dev
mailing list