[vbox-dev] OSE guest additions, VBoxService does not start

[S. Brandt]

S. Brandt schrieb:
> Hi!
> I'm using the self-compiled OSE guest additions, version 3.0.0.
> Unfortunately, the VBoxService.exe service does not start ... neither on
> system boot (system boot takes about a minute longer), nor manually later on.
> Debug version, VBOX_LOG=+all did not produce any log file.
> 
> Only information ...
> event 7022 in the system event log
> "Der Dienst "VirtualBox Guest Service" wurde nicht ordnungsgemäß gestartet."
> (the service VB Guest Service was not correctly started)
> or rather, according to http://support.microsoft.com/kb/319127,
> "Server service hung on startup."
> Service state stays in "starting up" until process is manually killed.
> 
> I compared the service registration with the one from the PUEL GA ... only
> differences: description, and "Tag 0x15" which did not change anything.
> 
> PUEL GA run w/o any problem.
> 
> I've reproduced the problem on various virtual machines.
> Version 2.2.4 did not have this problem.
> Replacing the 2.2.4 .exe with the 3.3.0 one gives the same result - service
> does not start.
> 
> Any way to find out what is the matter?
[Replying to myself]
Additional information:
Attached with VS08 to VBoxService.exe, stopped all threads.
Get the message
"The process appears to be deadlocked (or is not running an user-mode
code). All threads have been stopped."
stracktrace is:
>	ntdll.dll!7c91e514() 	
 	[Frames below may be incorrect and/or missing, no symbols loaded for
ntdll.dll]	
 	ntdll.dll!7c91d9da() 	
 	kernel32.dll!7c801879() 	
 	advapi32.dll!77dc348b() 	
 	advapi32.dll!77dc331f() 	
 	advapi32.dll!77dc3539() 	
 	advapi32.dll!77dc3628() 	
 	VBoxService.exe!0042ea6b() 	
 	VBoxService.exe!0041957e() 	
 	kernel32.dll!7c817077() 	
disassembly:
7C91E506  lea         esp,[esp]
7C91E50D  lea         ecx,[ecx]
7C91E510  mov         edx,esp
7C91E512  sysenter
> 7C91E514  ret

So, obviously, the sysenter call does not return.
Unfortunately, it is very difficult to debug the OnStart code of a
service - needs windbg etc.

One try ... patch the binary and place a CC in the vboxservice code.
Works .. breakpoint in vboxservice.exe before calling advapi.
Unfortunately -
1. how can I determine which advapi function is being called? Just by
the address?
2. when replacing the CC codebyte with the original, it is "jumped
over", although CC should be a "trap"
3. when resetting the "set next statement" to the changed command, the
next "step" command stops the process, w/o any message

Using depends.exe and the load addresses of the dlls I determined the
following:
VBoxService
advapi:StartServiceCtrlDispatcherW
advapi:SetServiceStatus
kernel32:ReadFile
ntdll:ZwReadFile
-> syscall that never returns.
well, seems I have to compile a debug version ...



Yours,
Sebastian