[vbox-dev] D3D support in custom frontend

Tue Feb 10 15:50:18 GMT 2015

Hi,

ok, thanks for all the info, I can then at least do some digging in
VBoxHeadless to see how it works, but it's great to know that I don't need
to get into the OpenGL service itself to make FBO work. Again, thanks for
all the feedback.

Best Regards,
Rudolfs Bundulis

2015-02-10 16:57 GMT+02:00 Klaus Espenlaub <klaus.espenlaub at oracle.com>:

>  On 09.02.2015 11:54, Rūdolfs Bundulis wrote:
>
> Hi Klaus,
>
>  thanks for the response, yeah I suspected that the path was read from
> somewhere, since running regsvr32 to unregister VBoxC.dll from installation
> directory and then registering it from my path didn't chagne anything, but
> I'm happy - it's not an issue to run my binaries from the installation
> folder, the main thing is that things seem to be working. Now the only
> thing I need is to see if I can make OpenGL render to a custom FBO so I can
> live without the window and access the OpenGL pixel data, but that is my
> issue, if I come up with anything interesting maybe that can be use fro you
> too. At least as far as I understand right now, due to the need of a window
> on the host, VBoxHeadless.exe does not provide any 3D acceleration support
> right?
>
> From what I understood it does work for quite a while now (at least
> 4.3)... all the FBO stuff should be already in place and working for
> VBoxHeadless, as long as the process gets started in a context offers
> working GDI/Direct3D functions.
>
> The only question is if this does what you need or only does the necessary
> work for the VRDE interface...
>
> Vitali?
>
>  Sorry to bother you so much, but the issue I'm more troubled by is the
> crash I get - I've already described it before but I have made little
> progress. Basically what happens is whenever I start my VM with my custom
> frontend with more than 1 CPU it crashes on Windows startup
> (longjmp_internal from Microsoft C runtime fails with unaligned stack, the
> last stack entry I saw from VBoxREM.dll was cpu_exit_loop or smth) - since
> VBoxREM.dll is built with MinGW visual studio does not help a lot, and when
> running under debug version under MinGW I also was unable to obtain any
> valuable info. With 1 CPU it always works fine. I've tried CRT debug heap,
> Application Verifier etc. to find potential issues with my code (since the
> default Qt frontend does not crash with the same configuration) I blame my
> part of the code, but still maybe there are some peculiarities with the COM
> interface. Do you know any bugs that have reported similar issues to what
> I'm seeing? Or maybe you can give any pointers since currently I'm kind of
> stuck.
>
>  Again, great thanks for the support, my research would have hit dead end
> without your help, yesterday I finally finished splitting so thanks to
> VirtualBox I am able to run a single virtual 9600x5400 display on
> VirtualBox (what is weird, even with WDDM drivers) and split/encode it to
> 25 displays, basically this cannot be done with hardware (well in terms of
> a single homogeneous display) so I'm very happy.
>
> Could be that we're just over cautious with WDDM (assuming the behavior of
> certain greedy apps) and that there is actually no hard requirement for a
> 3rd buffer.
>
>
>  Best Regards,
> Rudolfs Bundulis
>
> 2015-02-09 12:21 GMT+02:00 Klaus Espenlaub <klaus.espenlaub at oracle.com>:
>
>>  Rūdolfs,
>>
>> On 09.02.2015 10:28, Rūdolfs Bundulis wrote:
>>
>> Hi,
>>
>>  yeah it turned out that when running from the VirtualBox installation
>> path 3D support works for my frontend, seems that it comes from the fact
>> that the COM API is using the dll's  from the installation folder. I'm just
>> gonna leave this information here in case anyone else stumbles on the same
>> issue.
>>
>>  VBoxC.dll contains "in-process" COM components (which live in the "API
>> client" process), and the path to it is kept in the registry. Normal COM
>> stuff.
>>
>>
>> 2015-02-07 18:37 GMT+02:00 Rūdolfs Bundulis <rudolfs.bundulis at gmail.com>:
>>
>>> Hi,
>>>
>>>  again thanks for all the previous help, I managed to build VBox with
>>> the LogRel() but when I copied the dll's over the installation i got a ton
>>> of errors about invalid image hashes (I guess that just copying over is not
>>> the way to go). Anyhow, I came up with a different approach and made a
>>> small frontend that would basically do the same stuff as my app but also
>>> draw the output to a window. When I launch the application from the
>>> VirtualBox installation folder everything works fine - 3D support works,
>>> when I take the app and all the virtualbox binaries and put them in a
>>> random folder, it again stops working. And what is interresting that using
>>> Process Explorer I was able to see that actually two copies of VBoxC.dll
>>> are loaded (one from the installation folder and one from my folder) and
>>> one VBoxRT.dll (only from the VirtualBox installation folder). I assume the
>>> ones from the installation folder are loaded by the COM object? Is this a
>>> very very bad practice to try and deploy the VirtualBox binaries outside
>>> the installation path?
>>>
>>   The invalid image hashes most likely mean that VirtualBox isn't
>> accepting the signature in your binaries. It only accepts the "one signer"
>> of everything, so if you mix Oracle and your own build then it won't work
>> very well.
>>
>> Deploying VirtualBox binaries outside the installation path isn't exactly
>> forbidden, but we don't test it right now. This will stay for a while, we
>> have still way too much noise on the hardening related tickets.
>>
>> Klaus
>>
>>
>>> 2015-01-26 15:25 GMT+02:00 Rūdolfs Bundulis <rudolfs.bundulis at gmail.com>
>>> :
>>>
>>>> Hi Vitali,
>>>>
>>>>  this was what I was looking for, much appreciated :)
>>>>
>>>>  Best Regards,
>>>> Rudolfs Bundulis
>>>>
>>>> 2015-01-26 14:59 GMT+02:00 Vadim Galitsyn <vadim.galitsyn at oracle.com>:
>>>>
>>>>>  Hi Rudolfs,
>>>>>
>>>>>  you could add LogRel() messages and they should appear in release
>>>>> log (VBox.log file) once VBox has been built w/ corresponding flag. Note,
>>>>> that this only works for guest R0 code (miniport). Adding LogRel() to R3
>>>>> code (display driver) will not result in extra messages in log.
>>>>>
>>>>>  Please consider to add VBOX_WITH_R0_LOGGING=1 into LocalConfig.kmk
>>>>> in order to enable LogRel()’s.
>>>>>
>>>>>  Vadim
>>>>>
>>>>>  On 26 Jan 2015, at 15:26, Rūdolfs Bundulis <
>>>>> rudolfs.bundulis at gmail.com> wrote:
>>>>>
>>>>>  Hi Vadim,
>>>>>
>>>>>  ok clear, thanks for the info. We'll so far I haven't patched
>>>>> anything in the VBox code itself, as I said I have a custom frontend. Does
>>>>> that code help you in any way? I'll make a debug build and look for more
>>>>> messages and mail back if it gives any additional info, thanks for all the
>>>>> help so far. The main thing I wanted to understand was is there any way to
>>>>> see the output from the WARN macros in the miniport driver, and as far as I
>>>>> understand there is none.
>>>>>
>>>>>  Best Regards,
>>>>> Rudolfs Bundulis
>>>>>
>>>>> 2015-01-26 13:42 GMT+02:00 Vadim Galitsyn <vadim.galitsyn at oracle.com>:
>>>>>
>>>>>>  Hi Rudolfs,
>>>>>>
>>>>>>  Debug build of VBox itself would probably add a bit more messages
>>>>>> to VBox.log (and to debug log as well), but this definitely will not
>>>>>> prevent the fact that debug version of GAs cause BSOD. Is it possible if
>>>>>> you share a patch to OSE tree with your changes, so I could take a look at
>>>>>> it later this week (or another)?
>>>>>>
>>>>>>  Vadim
>>>>>>
>>>>>>  On 26 Jan 2015, at 14:00, Rūdolfs Bundulis <
>>>>>> rudolfs.bundulis at gmail.com> wrote:
>>>>>>
>>>>>>  Hi Vadim,
>>>>>>
>>>>>>  Well since I also came to the the conclusion that the WinId could
>>>>>> cause this, I tried making a dummy window for each of the IFramebuffer
>>>>>> instances, but nothing changed. Though the windows do not have any logic in
>>>>>> the message loop, and I do not resize them when the resolutions change -
>>>>>> maybe this causes some errors. That is why I asked if any logs from the
>>>>>> miniport driver is available to see where does it actually crash. If I make
>>>>>> a debug VirtualBox build then would debug GAs work?
>>>>>>
>>>>>>  Best Regards,
>>>>>> Rudolfs Bundulis
>>>>>>
>>>>>> 2015-01-26 12:50 GMT+02:00 Vadim Galitsyn <vadim.galitsyn at oracle.com>
>>>>>> :
>>>>>>
>>>>>>>  Hi Rudolfs,
>>>>>>>
>>>>>>>  As far as I can understand, returning NULL WinID might cause the
>>>>>>> behaviour you are observing on Windows host (please note, I have not tried
>>>>>>> that locally, take it like a gesture). Seems, you also already have 3D host
>>>>>>> service running. Can you try to provide valid WinID to Main/src-client
>>>>>>> stuff as an experiment and see if things changed?
>>>>>>>
>>>>>>>  Regarding to GAs, I am afraid you’ll have a BSOD in guest once you
>>>>>>> attempt to boot VM with debug GAs installed. We currently have some debug
>>>>>>> R0 assertion there.
>>>>>>>
>>>>>>>  Vadim
>>>>>>>
>>>>>>>  On 26 Jan 2015, at 13:12, Rūdolfs Bundulis <
>>>>>>> rudolfs.bundulis at gmail.com> wrote:
>>>>>>>
>>>>>>>  Hi Vadim,
>>>>>>>
>>>>>>>  thanks for the pointers, I'll look up the code, I was already
>>>>>>> afraid that something is simply missing.
>>>>>>>
>>>>>>>  To describe what I am doing:
>>>>>>> I am running a research project in University of Latvia where we are
>>>>>>> trying to use VirtualBox for large scale high resolution display wall
>>>>>>> construction - basically we run VirtualBox with a custom frontend that
>>>>>>> takes the video data from the VRAM, encodes it and streams over to a
>>>>>>> display wall, currently we've been able to run a 9600x5400 surface with
>>>>>>> Windows XPDM (we set this resolution as a hint and then internally split it
>>>>>>> to  5*5 1920x1080 displays) , which is really impressive, so thanks again
>>>>>>> for the great software. In terms of resolution I guess this is as far as
>>>>>>> the existing 256MB VRAM limit in VirtualBox can go (at least according to
>>>>>>> the calculations for framebuffer needs provided by Klaus Erspenlaub and
>>>>>>> mentioned in your code). Since the resolution limit is reached we are
>>>>>>> trying to address two more issues - 3D support and a weird unaligned stack
>>>>>>> crash in VBoxREM.dll with more than 1 CPU shared to the machine (for the
>>>>>>> second issue I need to do some more research as to where it is coming from
>>>>>>> before I ask any help here). As for the 3D - we'd basically want our
>>>>>>> software to function in the same way the VirtualBox QT client does, but
>>>>>>> currently it does not. Thanks for the pointer about the HGCM/OpenGL
>>>>>>> service, I'll check out that code and see what I can adapt.
>>>>>>>
>>>>>>>  Our software basically uses the VirtualBox COM interface to create
>>>>>>> the needed machine, set the number of displays/resolutions, injects the
>>>>>>> IFramebuffer instances and powers up the machine via the IConsole
>>>>>>> interface. So I examined all the IFramebuffer implementations in VirtualBox
>>>>>>> frontends, and didn't see anything very special in the 3D related callbacks
>>>>>>> there. As for the log - I tried to compare the log of a session from our
>>>>>>> software with a session from the same machine under QT fronted where the 3D
>>>>>>> acceleration works. Basically there were no errors and differences, in both
>>>>>>> cases the logs contained the lines about OpenGL support (i guess this comes
>>>>>>> from the OpenGL capability test exe), I'll send both logs as soon as I get
>>>>>>> back to that machine but I doubt they'll give much help, since they don't
>>>>>>> have any errors, and in both of them the guest additions log line states
>>>>>>> that acceleration is supported. Basically the only thing that differs is
>>>>>>> that the D3D/OpenGL (we've tried both API's, since the test application is
>>>>>>> built on Unity it is trivial to force either mode) device creation fails on
>>>>>>> our frontend. That is why I asked about the get_WinId, I examined the
>>>>>>> miniport driver code and as far as I understood it uses the window id value
>>>>>>> to create a swap chain. Out frontend does not have an actual window, so I
>>>>>>> am always returning NULL, can this cause an error? I also saw that the
>>>>>>> miniport driver has a lot of WARN macros, is  the output visible anywhere?
>>>>>>> I tried using DebugView but that didn't help? Should I try building a debug
>>>>>>> version of guest additions to get some log information?
>>>>>>>
>>>>>>>  Best Regards,
>>>>>>> Rudolfs Bundulis
>>>>>>>
>>>>>>> 2015-01-26 10:34 GMT+02:00 Vadim Galitsyn <vadim.galitsyn at oracle.com
>>>>>>> >:
>>>>>>>
>>>>>>>> Hi Rudolfs,
>>>>>>>>
>>>>>>>> In order to provide 3D acceleration, VBox HGCM/OpenGL host service
>>>>>>>> should be running. It is currently a part of VirtualBoxVM process
>>>>>>>> (ShCrOpenGL thread). In order to provide 3D acceleration support for your
>>>>>>>> frontend, I am afraid, you need to adapt stuff from this thread for your
>>>>>>>> code. And I am afraid this might be not that easy (there is a bit of magic
>>>>>>>> when VBox GUI conversates w/ 3D stuff). Please refer to
>>>>>>>> src/VBox/HostServices/SharedOpenGL and src/VBox/GuestHost/OpenGL sources.
>>>>>>>>
>>>>>>>> Btw, if you describe a little bit more why do you need this and
>>>>>>>> what you already have done, maybe I could explain you more things/steps you
>>>>>>>> need to do in order to reach the goal. VBox.log also might be helpful.
>>>>>>>>
>>>>>>>> Vadim
>>>>>>>>
>>>>>>>>
>>>>>>>> > On 24 Jan 2015, at 14:14, Rūdolfs Bundulis <
>>>>>>>> rudolfs.bundulis at gmail.com> wrote:
>>>>>>>> >
>>>>>>>> > Hi,
>>>>>>>> >
>>>>>>>> > I'm having a bit of a hard time understanding why my custom
>>>>>>>> framebuffer cannot use the 3D acceleration feature. I have a Windows 7
>>>>>>>> guest machine (3D acceleration enabled, guest additions installed) and a
>>>>>>>> Unity demo application. Under the default QT GUI the application launches
>>>>>>>> fine, while under my frontend Unity is not able to create the D3D device.
>>>>>>>> First I though that this could be due to incorrect implementation of
>>>>>>>> IFramebuffer, but when I looked at UIFrameBuffer::Notify3DEvent(ULONG
>>>>>>>> uType, BYTE *pData) in UIFrameBuffer.cpp there wasn't any special magic
>>>>>>>> there. Are there any hints to guide me towards understanding the cause of
>>>>>>>> this? Is there any tracing facilities for the guest miniport driver? Also,
>>>>>>>> I don't have a window handle for my frontend since there is no actual
>>>>>>>> window. If I am returning 0 in get_WinId in my framebuffer can that mess
>>>>>>>> things up?
>>>>>>>> >
>>>>>>>> > Best Regards,
>>>>>>>> > Rudolfs Bundulis
>>>>>>>>
>>>>>>>
> _______________________________________________
> vbox-dev mailing list
> vbox-dev at virtualbox.org
> https://www.virtualbox.org/mailman/listinfo/vbox-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.virtualbox.org/pipermail/vbox-dev/attachments/20150210/ec0c94c7/attachment.html>