[vbox-dev] Running MeeGo UX correctly inside VirtualBox with OpenGL acceleration

[Leonid]

Hi, Haitao.

Could you please point me to the mutter source where it deals with this dual 
layer rendering?

I'm not quite sure on this, but I'd suspect Mutter to just force all other 
application windows to do offscreen rendering via XCompositeRedirectWindow, 
including other OpenGL apps. And then render it own window using 
GLX_texture_from_pixmap to access other application pixmap data in his own
OpenGL context. Please correct me if this is wrong description.

If the above is correct, then there're clearly 2 things to fix in our opengl
implementation to make it work.

1. Window visibility check function, stubIsWindowVisible from 
src\VBox\Additions\common\crOpenGL\context.c doesn't take into account that
window might be redirected offscreen via XCompositeRedirectWindow. 
Adding a check there will solve the issue with overlapping windows.

2. Opengl application (like MeeGo UX plugin) pixmap content updates.
I think that it's best to go with something similar to what you've suggested in
"1). Rendering GL off-screen and reading window contents back into guest." 
Because going 2nd way will require us to "implement guest mutter logic at host 
side" which will require us to keep track of changes in mutter code etc.

So, for a first/fast solution I'd go with using glReadPixels to update guest 
window offscreen pixmap content when application issues glXSwapbuffer or 
glFlush/glFinish if glDrawBuffer is set to GL_FRONT.

Typical flow would be something like this:
1.MeeGo creates a window W.
2.Mutter issues XCompositeRedirectWindow for W, gets access to W offscreen
storage Pwin by using XCompositeNameWindowPixmap and creates opengl pixmap
Pogl to use with GLX_texture_from_pixmap by issuing 
glXCreatePixmap(dpy, ..., Pwin, ...)
3.MeeGo creates an OpenGL content and makes it current with W.
Our code, in stubIsWindowVisible detect that the current window is 
redirected to pixmap Pwin and sets some flag causing us on 
glXSwapbuffers(+glFlush/glFinish) to issue glReadpixels and update Pwin.
4.Mutter goes in render loop and at some point processes MeeGo window(s),
issuing glXBindTexImageEXT(dpy, Pogl, ...)
Our code there, will update associated host texture with content of Pwin
via glTex[Sub]Image calls.
Etc.

As a future enhancement I'd try to avoid unnecessary 
glReadpixels/glTex[Sub]Image calls this way:
When we detect that our W is redirected to Pwin, we could force host part to 
use fbo for window W, binding some texture T as GL_COLOR_ATTACHMENT0_EXT.
(close functionality could be seen in 
src\VBox\HostServices\SharedOpenGL\crserverlib\server_muralfbo.c)
Now when mutter issues glXBindTexImageEXT, we might query host to see if
there's a FBO redirection for a guest pixmap Pwin and if there's one, we 
can just bind texture T and there's no need to upload anymore data to host.

The only problem here is that we need to somehow intercept every guest 
read/write access to a pixmap Pwin and update it content before read accesses 
or update T after writes to Pwin. I'm not quite sure how could we intercept 
calls to XGetImage/XCopyArea etc, but I'm not sure if in this particular case 
Mutter/MeeGo will ever issue those calls as well.

Best regards, Leonid.