[vbox-dev] D3D support in custom frontend
Klaus Espenlaub
klaus.espenlaub at oracle.com
Tue Feb 10 14:57:01 GMT 2015
On 09.02.2015 11:54, Rūdolfs Bundulis wrote:
> Hi Klaus,
>
> thanks for the response, yeah I suspected that the path was read from
> somewhere, since running regsvr32 to unregister VBoxC.dll from
> installation directory and then registering it from my path didn't
> chagne anything, but I'm happy - it's not an issue to run my binaries
> from the installation folder, the main thing is that things seem to be
> working. Now the only thing I need is to see if I can make OpenGL
> render to a custom FBO so I can live without the window and access the
> OpenGL pixel data, but that is my issue, if I come up with anything
> interesting maybe that can be use fro you too. At least as far as I
> understand right now, due to the need of a window on the host,
> VBoxHeadless.exe does not provide any 3D acceleration support right?
From what I understood it does work for quite a while now (at least
4.3)... all the FBO stuff should be already in place and working for
VBoxHeadless, as long as the process gets started in a context offers
working GDI/Direct3D functions.
The only question is if this does what you need or only does the
necessary work for the VRDE interface...
Vitali?
> Sorry to bother you so much, but the issue I'm more troubled by is the
> crash I get - I've already described it before but I have made little
> progress. Basically what happens is whenever I start my VM with my
> custom frontend with more than 1 CPU it crashes on Windows startup
> (longjmp_internal from Microsoft C runtime fails with unaligned stack,
> the last stack entry I saw from VBoxREM.dll was cpu_exit_loop or smth)
> - since VBoxREM.dll is built with MinGW visual studio does not help a
> lot, and when running under debug version under MinGW I also was
> unable to obtain any valuable info. With 1 CPU it always works fine.
> I've tried CRT debug heap, Application Verifier etc. to find potential
> issues with my code (since the default Qt frontend does not crash with
> the same configuration) I blame my part of the code, but still maybe
> there are some peculiarities with the COM interface. Do you know any
> bugs that have reported similar issues to what I'm seeing? Or maybe
> you can give any pointers since currently I'm kind of stuck.
>
> Again, great thanks for the support, my research would have hit dead
> end without your help, yesterday I finally finished splitting so
> thanks to VirtualBox I am able to run a single virtual 9600x5400
> display on VirtualBox (what is weird, even with WDDM drivers) and
> split/encode it to 25 displays, basically this cannot be done with
> hardware (well in terms of a single homogeneous display) so I'm very
> happy.
Could be that we're just over cautious with WDDM (assuming the behavior
of certain greedy apps) and that there is actually no hard requirement
for a 3rd buffer.
>
> Best Regards,
> Rudolfs Bundulis
>
> 2015-02-09 12:21 GMT+02:00 Klaus Espenlaub <klaus.espenlaub at oracle.com
> <mailto:klaus.espenlaub at oracle.com>>:
>
> Rūdolfs,
>
> On 09.02.2015 10:28, Rūdolfs Bundulis wrote:
>> Hi,
>>
>> yeah it turned out that when running from the VirtualBox
>> installation path 3D support works for my frontend, seems that it
>> comes from the fact that the COM API is using the dll's from the
>> installation folder. I'm just gonna leave this information here
>> in case anyone else stumbles on the same issue.
> VBoxC.dll contains "in-process" COM components (which live in the
> "API client" process), and the path to it is kept in the registry.
> Normal COM stuff.
>>
>> 2015-02-07 18:37 GMT+02:00 Rūdolfs Bundulis
>> <rudolfs.bundulis at gmail.com <mailto:rudolfs.bundulis at gmail.com>>:
>>
>> Hi,
>>
>> again thanks for all the previous help, I managed to build
>> VBox with the LogRel() but when I copied the dll's over the
>> installation i got a ton of errors about invalid image hashes
>> (I guess that just copying over is not the way to go).
>> Anyhow, I came up with a different approach and made a small
>> frontend that would basically do the same stuff as my app but
>> also draw the output to a window. When I launch the
>> application from the VirtualBox installation folder
>> everything works fine - 3D support works, when I take the app
>> and all the virtualbox binaries and put them in a random
>> folder, it again stops working. And what is interresting that
>> using Process Explorer I was able to see that actually two
>> copies of VBoxC.dll are loaded (one from the installation
>> folder and one from my folder) and one VBoxRT.dll (only from
>> the VirtualBox installation folder). I assume the ones from
>> the installation folder are loaded by the COM object? Is this
>> a very very bad practice to try and deploy the VirtualBox
>> binaries outside the installation path?
>>
> The invalid image hashes most likely mean that VirtualBox isn't
> accepting the signature in your binaries. It only accepts the "one
> signer" of everything, so if you mix Oracle and your own build
> then it won't work very well.
>
> Deploying VirtualBox binaries outside the installation path isn't
> exactly forbidden, but we don't test it right now. This will stay
> for a while, we have still way too much noise on the hardening
> related tickets.
>
> Klaus
>
>>
>> 2015-01-26 15:25 GMT+02:00 Rūdolfs Bundulis
>> <rudolfs.bundulis at gmail.com <mailto:rudolfs.bundulis at gmail.com>>:
>>
>> Hi Vitali,
>>
>> this was what I was looking for, much appreciated :)
>>
>> Best Regards,
>> Rudolfs Bundulis
>>
>> 2015-01-26 14:59 GMT+02:00 Vadim Galitsyn
>> <vadim.galitsyn at oracle.com
>> <mailto:vadim.galitsyn at oracle.com>>:
>>
>> Hi Rudolfs,
>>
>> you could add LogRel() messages and they should
>> appear in release log (VBox.log file) once VBox has
>> been built w/ corresponding flag. Note, that this
>> only works for guest R0 code (miniport). Adding
>> LogRel() to R3 code (display driver) will not result
>> in extra messages in log.
>>
>> Please consider to add VBOX_WITH_R0_LOGGING=1 into
>> LocalConfig.kmk in order to enable LogRel()’s.
>>
>> Vadim
>>
>>> On 26 Jan 2015, at 15:26, Rūdolfs Bundulis
>>> <rudolfs.bundulis at gmail.com
>>> <mailto:rudolfs.bundulis at gmail.com>> wrote:
>>>
>>> Hi Vadim,
>>>
>>> ok clear, thanks for the info. We'll so far I
>>> haven't patched anything in the VBox code itself, as
>>> I said I have a custom frontend. Does that code help
>>> you in any way? I'll make a debug build and look for
>>> more messages and mail back if it gives any
>>> additional info, thanks for all the help so far. The
>>> main thing I wanted to understand was is there any
>>> way to see the output from the WARN macros in the
>>> miniport driver, and as far as I understand there is
>>> none.
>>>
>>> Best Regards,
>>> Rudolfs Bundulis
>>>
>>> 2015-01-26 13:42 GMT+02:00 Vadim Galitsyn
>>> <vadim.galitsyn at oracle.com
>>> <mailto:vadim.galitsyn at oracle.com>>:
>>>
>>> Hi Rudolfs,
>>>
>>> Debug build of VBox itself would probably add a
>>> bit more messages to VBox.log (and to debug log
>>> as well), but this definitely will not prevent
>>> the fact that debug version of GAs cause BSOD.
>>> Is it possible if you share a patch to OSE tree
>>> with your changes, so I could take a look at it
>>> later this week (or another)?
>>>
>>> Vadim
>>>
>>>> On 26 Jan 2015, at 14:00, Rūdolfs Bundulis
>>>> <rudolfs.bundulis at gmail.com
>>>> <mailto:rudolfs.bundulis at gmail.com>> wrote:
>>>>
>>>> Hi Vadim,
>>>>
>>>> Well since I also came to the the conclusion
>>>> that the WinId could cause this, I tried making
>>>> a dummy window for each of the IFramebuffer
>>>> instances, but nothing changed. Though the
>>>> windows do not have any logic in the message
>>>> loop, and I do not resize them when the
>>>> resolutions change - maybe this causes some
>>>> errors. That is why I asked if any logs from
>>>> the miniport driver is available to see where
>>>> does it actually crash. If I make a debug
>>>> VirtualBox build then would debug GAs work?
>>>>
>>>> Best Regards,
>>>> Rudolfs Bundulis
>>>>
>>>> 2015-01-26 12:50 GMT+02:00 Vadim Galitsyn
>>>> <vadim.galitsyn at oracle.com
>>>> <mailto:vadim.galitsyn at oracle.com>>:
>>>>
>>>> Hi Rudolfs,
>>>>
>>>> As far as I can understand, returning NULL
>>>> WinID might cause the behaviour you are
>>>> observing on Windows host (please note, I
>>>> have not tried that locally, take it like a
>>>> gesture). Seems, you also already have 3D
>>>> host service running. Can you try to
>>>> provide valid WinID to Main/src-client
>>>> stuff as an experiment and see if things
>>>> changed?
>>>>
>>>> Regarding to GAs, I am afraid you’ll have a
>>>> BSOD in guest once you attempt to boot VM
>>>> with debug GAs installed. We currently have
>>>> some debug R0 assertion there.
>>>>
>>>> Vadim
>>>>
>>>>> On 26 Jan 2015, at 13:12, Rūdolfs Bundulis
>>>>> <rudolfs.bundulis at gmail.com
>>>>> <mailto:rudolfs.bundulis at gmail.com>> wrote:
>>>>>
>>>>> Hi Vadim,
>>>>>
>>>>> thanks for the pointers, I'll look up the
>>>>> code, I was already afraid that something
>>>>> is simply missing.
>>>>>
>>>>> To describe what I am doing:
>>>>> I am running a research project in
>>>>> University of Latvia where we are trying
>>>>> to use VirtualBox for large scale high
>>>>> resolution display wall construction -
>>>>> basically we run VirtualBox with a custom
>>>>> frontend that takes the video data from
>>>>> the VRAM, encodes it and streams over to a
>>>>> display wall, currently we've been able to
>>>>> run a 9600x5400 surface with Windows XPDM
>>>>> (we set this resolution as a hint and then
>>>>> internally split it to 5*5 1920x1080
>>>>> displays) , which is really impressive, so
>>>>> thanks again for the great software. In
>>>>> terms of resolution I guess this is as far
>>>>> as the existing 256MB VRAM limit in
>>>>> VirtualBox can go (at least according to
>>>>> the calculations for framebuffer needs
>>>>> provided by Klaus Erspenlaub and mentioned
>>>>> in your code). Since the resolution limit
>>>>> is reached we are trying to address two
>>>>> more issues - 3D support and a weird
>>>>> unaligned stack crash in VBoxREM.dll with
>>>>> more than 1 CPU shared to the machine (for
>>>>> the second issue I need to do some more
>>>>> research as to where it is coming from
>>>>> before I ask any help here). As for the 3D
>>>>> - we'd basically want our software to
>>>>> function in the same way the VirtualBox QT
>>>>> client does, but currently it does not.
>>>>> Thanks for the pointer about the
>>>>> HGCM/OpenGL service, I'll check out that
>>>>> code and see what I can adapt.
>>>>>
>>>>> Our software basically uses the VirtualBox
>>>>> COM interface to create the needed
>>>>> machine, set the number of
>>>>> displays/resolutions, injects the
>>>>> IFramebuffer instances and powers up the
>>>>> machine via the IConsole interface. So I
>>>>> examined all the IFramebuffer
>>>>> implementations in VirtualBox frontends,
>>>>> and didn't see anything very special in
>>>>> the 3D related callbacks there. As for the
>>>>> log - I tried to compare the log of a
>>>>> session from our software with a session
>>>>> from the same machine under QT fronted
>>>>> where the 3D acceleration works. Basically
>>>>> there were no errors and differences, in
>>>>> both cases the logs contained the lines
>>>>> about OpenGL support (i guess this comes
>>>>> from the OpenGL capability test exe), I'll
>>>>> send both logs as soon as I get back to
>>>>> that machine but I doubt they'll give much
>>>>> help, since they don't have any errors,
>>>>> and in both of them the guest additions
>>>>> log line states that acceleration is
>>>>> supported. Basically the only thing that
>>>>> differs is that the D3D/OpenGL (we've
>>>>> tried both API's, since the test
>>>>> application is built on Unity it is
>>>>> trivial to force either mode) device
>>>>> creation fails on our frontend. That is
>>>>> why I asked about the get_WinId, I
>>>>> examined the miniport driver code and as
>>>>> far as I understood it uses the window id
>>>>> value to create a swap chain. Out frontend
>>>>> does not have an actual window, so I am
>>>>> always returning NULL, can this cause an
>>>>> error? I also saw that the miniport driver
>>>>> has a lot of WARN macros, is the output
>>>>> visible anywhere? I tried using DebugView
>>>>> but that didn't help? Should I try
>>>>> building a debug version of guest
>>>>> additions to get some log information?
>>>>>
>>>>> Best Regards,
>>>>> Rudolfs Bundulis
>>>>>
>>>>> 2015-01-26 10:34 GMT+02:00 Vadim Galitsyn
>>>>> <vadim.galitsyn at oracle.com
>>>>> <mailto:vadim.galitsyn at oracle.com>>:
>>>>>
>>>>> Hi Rudolfs,
>>>>>
>>>>> In order to provide 3D acceleration,
>>>>> VBox HGCM/OpenGL host service should
>>>>> be running. It is currently a part of
>>>>> VirtualBoxVM process (ShCrOpenGL
>>>>> thread). In order to provide 3D
>>>>> acceleration support for your
>>>>> frontend, I am afraid, you need to
>>>>> adapt stuff from this thread for your
>>>>> code. And I am afraid this might be
>>>>> not that easy (there is a bit of magic
>>>>> when VBox GUI conversates w/ 3D
>>>>> stuff). Please refer to
>>>>> src/VBox/HostServices/SharedOpenGL and
>>>>> src/VBox/GuestHost/OpenGL sources.
>>>>>
>>>>> Btw, if you describe a little bit more
>>>>> why do you need this and what you
>>>>> already have done, maybe I could
>>>>> explain you more things/steps you need
>>>>> to do in order to reach the goal.
>>>>> VBox.log also might be helpful.
>>>>>
>>>>> Vadim
>>>>>
>>>>>
>>>>> > On 24 Jan 2015, at 14:14, Rūdolfs
>>>>> Bundulis <rudolfs.bundulis at gmail.com
>>>>> <mailto:rudolfs.bundulis at gmail.com>>
>>>>> wrote:
>>>>> >
>>>>> > Hi,
>>>>> >
>>>>> > I'm having a bit of a hard time
>>>>> understanding why my custom
>>>>> framebuffer cannot use the 3D
>>>>> acceleration feature. I have a Windows
>>>>> 7 guest machine (3D acceleration
>>>>> enabled, guest additions installed)
>>>>> and a Unity demo application. Under
>>>>> the default QT GUI the application
>>>>> launches fine, while under my frontend
>>>>> Unity is not able to create the D3D
>>>>> device. First I though that this could
>>>>> be due to incorrect implementation of
>>>>> IFramebuffer, but when I looked at
>>>>> UIFrameBuffer::Notify3DEvent(ULONG
>>>>> uType, BYTE *pData) in
>>>>> UIFrameBuffer.cpp there wasn't any
>>>>> special magic there. Are there any
>>>>> hints to guide me towards
>>>>> understanding the cause of this? Is
>>>>> there any tracing facilities for the
>>>>> guest miniport driver? Also, I don't
>>>>> have a window handle for my frontend
>>>>> since there is no actual window. If I
>>>>> am returning 0 in get_WinId in my
>>>>> framebuffer can that mess things up?
>>>>> >
>>>>> > Best Regards,
>>>>> > Rudolfs Bundulis
>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.virtualbox.org/pipermail/vbox-dev/attachments/20150210/0d87c263/attachment.html>
More information about the vbox-dev
mailing list