[vbox-dev] Minimalist OpenGL program crashing

[Klaus Espenlaub]

spent 5 minutes digging... looks rather clearly like a "use after free" 
of the global "stub" variable on crOpenGL. The function 
stubSPUTearDownLocked is called too early (before the libglut exit 
function).

Since libglut has no explicit uninit and quite a lot of 3D apps (and 
some are bound to use libglut) work without crashing on exit I wonder 
how they initialize GL. Must be in a way which loads our GL driver 
before libglut registers its exit function, because the exit functions 
are executed in reverse order of registration.

Klaus

On 21.11.2016 13:59, Michael Thayer wrote:
> Hello Otacílio,
>
> 21.11.2016 12:57, Otacílio wrote:
>> Yes, I located the exactly line where the seg fault is raised. I did a
>> full report with a description and a stacktrace. Is reported here:
>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=214396
>
> I'm afraid that you must have misunderstood me.  I said that it was
> quite likely that this would get fixed if you were able to understand
> (and by implication explain) the problem.  By understanding though I
> specifically did not mean just pointing out the line in the code where
> the segmentation fault occurs (I saw that too when I ran the programme
> in the debugger).  I meant finding the logical flaw in the reasoning
> behind the code of which this crash is a symptom.
>
> Feel free to open a bug in our bugtracker for this of course, but
> someone will still have to do that analysis.  As I said in my previous
> reply, it is extremely unlikely that we will have resources for this in
> the near future.
>
> Regards,
>
> Michael
>
>> After debugging gdb says that the exception is raised on line 679 of
>> file
>> /usr/ports/emulators/virtualbox-ose-additions/work/VirtualBox-5.1.8/src/VBox/GuestHost/OpenGL/util/hash.c
>>
>>
>> void *crHashtableSearch( const CRHashTable *h, unsigned long key )
>> {
>>     unsigned int index = crHash( key );
>>     CRHashNode *temp;
>> #ifdef CHROMIUM_THREADSAFE
>> BUG IS HERE  =======>  crLockMutex((CRmutex *)&h->mutex);
>> #endif
>>     for ( temp = h->buckets[index]; temp; temp = temp->next )
>>     {
>>         if ( temp->key == key )
>>             break;
>>     }
>> #ifdef CHROMIUM_THREADSAFE
>>     crUnlockMutex((CRmutex *)&h->mutex);
>> #endif
>>     if ( !temp )
>>     {
>>         return NULL;
>>     }
>>     return temp->data;
>> }
>>
>>
>> (gdb) backtrace
>> #0  check_and_init_mutex (mutex=0x20c8, m=0x7fffffffe650) at
>> /usr/src/lib/libthr/thread/thr_mutex.c:588
>> #1  0x0000000802923506 in __pthread_mutex_lock (mutex=0x20c8) at
>> /usr/src/lib/libthr/thread/thr_mutex.c:743
>> #2  0x0000000801e531f1 in crHashtableSearch (h=0x0, key=56623106) at
>> /usr/ports/emulators/virtualbox-ose-additions/work/VirtualBox-5.1.8/src/VBox/GuestHost/OpenGL/util/hash.c:679
>>
>> #3  0x000000080088f1cb in glXMakeCurrent (dpy=0x80481f000,
>> drawable=56623106, ctx=0x1f4) at
>> /usr/ports/emulators/virtualbox-ose-additions/work/VirtualBox-5.1.8/src/VBox/Additions/common/crOpenGL/glx.c:694
>>
>> #4  0x00000008010ce5be in fgSetWindow () from /usr/local/lib/libglut.so.3
>> #5  0x00000008010cc44c in fgDestroyWindow () from
>> /usr/local/lib/libglut.so.3
>> #6  0x00000008010cc895 in fgDestroyStructure () from
>> /usr/local/lib/libglut.so.3
>> #7  0x00000008010c9841 in fgDeinitialize () from
>> /usr/local/lib/libglut.so.3
>> #8  0x0000000801b9add6 in __cxa_finalize (dso=<value optimized out>)
>> at /usr/src/lib/libc/stdlib/atexit.c:237
>> #9  0x0000000801b2a141 in exit (status=0) at
>> /usr/src/lib/libc/stdlib/exit.c:72
>> #10 0x0000000000400976 in _start ()
>> #11 0x0000000800622000 in ?? ()
>> #12 0x0000000000000000 in ?? ()
>>
>>
>>
>> But, if you prefer I can report in a vbox bugtracking also.
>>
>> Cheers
>> -Otacílio
>>
>> Em 21/11/2016 08:25, Michael Thayer escreveu:
>>> Hello Otacílio,
>>>
>>> 10.11.2016 13:07, Otacílio wrote:
>>>> I'm using this minimalist program to test OpenGL. When running on a
>>>> true
>>>> machine (FreeBSD 11, clang 3.8.0) this program runs fine. But, when
>>>> compiling and running in a Virtualbox guest with
>>>> virtualbox-ose-additions-5.1.8 installed (FreeBSD 11, clang 3.8.0,
>>>> OpenGL enabled)  I'm getting a core dump when the program exits. So its
>>>> not clear to me if this is a FreeBSD port problem or not. So, please, I
>>>> wold like ask you to compile and running in your's guest machine
>>>> running
>>>> Linux or FreeBSD and report to me the results.
>>>
>>> I can confirm that this affects Linux guests too.  Unfortunately we do
>>> not have time to investigate X11 guest 3D issues for the foreseeable
>>> future, so your best bet is probably to debug it yourself.  If you can
>>> clearly locate the problem (as in understanding it) chances are good
>>> that we will fix it.
>>>
>>> Regards,
>>>
>>> Michael
>>>
>>>>
>>>> Program:
>>>>
>>>> #include <stdio.h>
>>>> #include <GL/glew.h>
>>>> #include <GL/gl.h>
>>>> #include <GL/glu.h>
>>>> #include <GL/glut.h>
>>>> #include <GL/glext.h>
>>>>
>>>> int main(int argc, char **argv) {
>>>>
>>>>         glutInit(&argc, argv);
>>>>         glutCreateWindow("GLUT");
>>>>         glewInit();
>>>>
>>>>         printf("OpenGL version supported by this platform (%s): \n",
>>>>         glGetString(GL_VERSION));
>>>>
>>>>         fprintf(stdout, "Exiting...\n");
>>>>         fflush(stdout);
>>>>
>>>>         return 0;
>>>> }
>>>>
>>>>
>>>> Compiler command:
>>>>
>>>> cc -I/usr/local/include -L/usr/local/lib -o testegl testegl.c -lGL
>>>> -lGLEW -lGLU   -lglut
>>>>
>>>>
>>>> My results in a guest machine:
>>>>
>>>> [ota at nostromo /usr/home/ota/Desktop]$ ./testegl
>>>> OpenGL version supported by this platform (2.1 Chromium 1.9):
>>>> Exiting...
>>>> Segmentation fault (imagem do núcleo gravada)
>>>>
>>>>
>>>> Thanks a lot!
>>>>
>>>> []'s
>>>>
>>>> -Otacilio