[vbox-dev] Unreliability in guestcontrol

[Brian Campbell]

Hi,

I've been looking at using guest control for some automation, and I've 
encountered some problems that make the guest control part of the 
VBoxService daemon stop working.  It's easy to reproduce this by using 
guestcontrol to execute /bin/true lots of times in a loop.

The root cause appears to be that after one request is handled by the 
per-process thread, the main guest control thread in VBoxService can set 
up another request before the per-process thread expects one.  Two 
problems appear because of this:

1. Occasionally the per-process thread will wipe out the request pointer 
after it has already been updated to the new request, so the new request 
is never processed.  If you're logging you see a "IPC request is 
invalid" message instead.  This is easily solved by moving the 
assignment before the code to wake up the main thread.

2. When the per-process thread realises it can shut down, if the main 
thread produces another request it will deadlock when the per-process 
thread attempts to enter a critical section.  Fixing the deadlock isn't 
enough because it still won't process the request.  I've worked around 
this by making the per-process thread set the shutdown flag and continue 
iterating through the main loop until it's sure that there's no request.

I've attached a patch with these changes, although I think the way that 
guest control is implemented in VBoxService may need to be revisited - 
it seems a little confused and fragile, and my change for 2 is a bit of 
a hack.  Also, there's a memory leak that I haven't tried to diagnose.

Best regards,

   Brian Campbell
-------------- next part --------------
A non-text attachment was scrubbed...
Name: VBoxServiceHack.patch
Type: text/x-patch
Size: 1853 bytes
Desc: not available
URL: <http://www.virtualbox.org/pipermail/vbox-dev/attachments/20120716/b4966dd7/attachment.bin>