VirtualBox

Opened 13 years ago

Closed 10 years ago

#9053 closed defect (obsolete)

FSTENV assembler instruction - wrong implementation

Reported by: Franjo Stipanovic Owned by:
Component: other Version: VirtualBox 4.0.8
Keywords: assembler implementation floating point Cc:
Guest type: Windows Host type: Windows

Description (last modified by Frank Mehnert)

Hi,

I believe VirtualBox has wrong implementation of FSTENV assembler instruction. It is not dumping last float-pointing instruction address correctly. I've created a simple application in assembler which can be used as proof.

Try and debug this application with debugger (OllyDbg for example) and take a look at buffer after the FSTENV instruction. It should contain address of FNOP assembler instruction.

format PE GUI
entry main

include 'win32a.inc'

section '.data' data readable writeable
	buffer db "                        ", 0

section '.text' code readable executable
main:
	nop
	mov eax, buffer
	fnop
	fstenv [eax]

	invoke ExitProcess, 0

section '.idata' import data readable
	library kernel32, 'KERNEL32.DLL'
	import kernel32, ExitProcess, 'ExitProcess'

You can compile the example code with FASM.

Change History (7)

comment:1 by Frank Mehnert, 13 years ago

I've tested this with several Linux guests (32-bit guest with/without VT-x, 64-bit guest) and it always worked correctly. Note that the instruction pointer starts at offset 12 of the buffer. Which content has the buffer in your case?

comment:2 by Franjo Stipanovic, 13 years ago

I'm comparing this case with my windows host on which works fine. This is my buffer: 7F 02 FF FF 00 00 FF FF FF FF FF FF 29 FA C4 77 1B 00 7D 07 0C FF 55 01 23 00 FF FF

Instruction pointer is (starting at offset 12) 77C4FA29. It should be 00402006, that's where FNOP is located.

My host is Windows and guest is Windows also.

comment:3 by michaln, 13 years ago

I tried the example on Windows host and guest, in both cases I got the expected result. FYI, you don't need any of the Windows API junk for your tiny example, a simple "ret" does the job.

To eliminate the possibility of a debugger interfering, I wrote the following tiny C program - not very portable, but that doesn't matter, it does the job:

#include <stdio.h>
#include <fenv.h>

int main( void )
{
    fenv_t  fenv;

    __asm fnop;
    fegetenv( &fenv );
    printf( "Last FP EIP: %08x\n", fenv.ins_ptr_offset );
    return( 0 );
}

The program prints something along the lines "Last FP EIP: 00401018" which is the address of the FNOP instruction. The result is identical on my Windows Server 2003 host and inside a Windows XP guest.

The address 0x77C4FA29 looks very much like an address inside a DLL. You should find out what is at that address. My guess is that the debugger you are using is broken and upsets the results. I strongly suggest that rather than using a debugger, you print the last instruction pointer value from the fstenv buffer.

comment:4 by Franjo Stipanovic, 13 years ago

michaln, thanks for the feedback, I'll do the test and get back at you.

comment:5 by Frank Mehnert, 13 years ago

If you can still reproduce this issue, please attach a VBox.log file from such a VM session so we can have a look at the VM configuration.

comment:6 by Franjo Stipanovic, 13 years ago

michaln, frank, sorry for false alarm. When executed w/o debugger the IP is correct, but when in debugger - it's completely wrong. This is strange because debugger (OllyDbg) works on host OS and on other machines ...

Thanks for the response.

comment:7 by Frank Mehnert, 10 years ago

Description: modified (diff)
Resolution: obsolete
Status: newclosed
Note: See TracTickets for help on using tickets.

© 2023 Oracle
ContactPrivacy policyTerms of Use