Interesting finding. Starting with VirtualBox 4.0, a lot of more X events are exchanged between the GUI and the X server, xtrace clearly shows that. The actual reason is still unknown.