Should be improved with VBox 5.0. Please try the latest version of VirtualBox (currently 5.0.4). For Windows there was another latency fix in the latest 5.0.x test build (>= 102676).