So what you say is that the clock of your host operating system runs slower if you run a VM on that host, did I understand this correct?
I would like to see a VBox.log file of such a VM session, mainly to see the configuration of your VM.