linux libpthread issue corrupted gs register - possible lock cmpxchg emulation issue?

[joeba]

Environment

VB 2.0.2, Host, RHEL 4.0, Guest RHEL 4.0, 32bit

Code only crashes under VB. Runs fine under VMWare and bare metal

Dump of Registers:

(gdb) info registers
eax            0x0      0
ecx            0xbff5237c       -1074453636
edx            0x4      4
ebx            0xbff5237c       -1074453636
esp            0xbff52320       0xbff52320
ebp            0xbff52320       0xbff52320
esi            0x0      0
edi            0x4      4
eip            0x848397 0x848397       /*** Fault instruction right here !!!! ***/
eflags         0x246    582
cs             0x73     115
ss             0x7b     123
ds             0x7b     123
es             0x7b     123
fs             0x0      0
gs             0x33     51

Here is the stack trace ...

(gdb) disassemble 0xcc2397
Dump of assembler code for function __pthread_disable_asynccancel:
0x00cc2390 <__pthread_disable_asynccancel+0>: push %ebp
0x00cc2391 <__pthread_disable_asynccancel+1>: test $0x2,%al
0x00cc2393 <__pthread_disable_asynccancel+3>: mov %esp,%ebp
0x00cc2395 <__pthread_disable_asynccancel+5>: jne 0xcc23b6 <__pthread_disable_asynccancel+38>
0x00cc2397 <__pthread_disable_asynccancel+7>: mov %gs:0x58,%edx /*** Same fault instruction ***/
0x00cc239e <__pthread_disable_asynccancel+14>: mov %edx,%ecx
0x00cc23a0 <__pthread_disable_asynccancel+16>: and $0xfffffffd,%ecx
0x00cc23a3 <__pthread_disable_asynccancel+19>: cmp %edx,%ecx
0x00cc23a5 <__pthread_disable_asynccancel+21>: je 0xcc23b6 <__pthread_disable_asynccancel+38>
0x00cc23a7 <__pthread_disable_asynccancel+23>: mov %edx,%eax
0x00cc23a9 <__pthread_disable_asynccancel+25>: lock cmpxchg

Here is the source of the code that caused the crash. It's the THREAD_GETMEM macro which is part of the libpthreads implementation under linux ...

/* Read member of the thread descriptor directly. */

# define THREAD_GETMEM(descr, member) \
({ __typeof (descr->member) __value; \
if (sizeof (__value) == 1) \
asm volatile ("movb %%gs:%P2,%b0" \
: "=q" (__value) \
: "0" (0), "i" (offsetof (struct pthread, member))); \[[BR]]
else if (sizeof (__value) == 4) \
asm volatile ("movl %%gs:%P1,%0" \
: "=r" (__value) \
: "i" (offsetof (struct pthread, member))); \
else \
{ \
if (sizeof (__value) != Cool \
/* There should not be any value with a size other than 1, \
4 or 8. */ \
abort (); \
\
asm volatile ("movl %%gs:%P1,%%eax\n	" \
"movl %%gs:%P2,%%edx" \ /*** instruction that causes fault *****/
: "=A" (__value) \
: "i" (offsetof (struct pthread, member)), \
"i" (offsetof (struct pthread, member) + 4)); \
} \
__value; })

I'm taking a guess at this, but could the lock instruction emulation cause this?

[Frank Mehnert]

Please use {{{ and }}} next time you quote code.