[vbox-dev] Internal design question

[Knut St. Osmundsen]

Hi Jonas,

my apologies for the late answer (too busy with the upcoming release).

Jonas Mellin wrote:
> I am running Windows XP as host and Ubuntu 8.10 with Linux 2.6.24.7-rt27
> (real-time preemption) as guest. I wonder how the high precision timer
> is affected by the use of virtualbox in this setup. I can perceive a few
> scenarios myself:
> 1) The timer runs without interference of the virtualization. The only
> effect is that the host affects the guest by delaying its execution.
> 2) In the switch between host and guest, the value of the timer is saved
> and restored. The effect is that in terms of execution time, there is no
> delays due to the host system. However, w.r.t. absolute time there is a
> devation that goes undetected unless the system clock is accessed.
> 3) The timer does not work. My test programs gives the impression that
> it works, so I have discared this scenario.
> 

Scenario 1 is pretty close to what happens I'd say.

Let me give you a quick rundown of how it all works w.r.t. timers and time.

Time sources are, with the exception of the time stamp counter (TSC),
associated with devices. Some of the devices provide timer service, like
raising interrupts when a counter reaches zero. Most of them expose time
in some way or another and can be used by the guest for time keeping. In
order to keep all the important time sources in sync (so that they show
the "same" time) they are all running on the same clock internally in
the virtual machine monitor (VMM). So, for instance the TSC is a
function of the CPU frequency and the delta since it started counting.
Similarly, the programmable interrupt timer (PIT) finds the current
counter value by the delta since it was started, predefined PIT
frequency and the period length. The PIT also schedules a timer with the
VMM for raising an interrupt when the counter reaches zero.

The VMM timers are executed as soon as we see them expire. But this
means we're subject to several sources of delay:
  1. Host OS scheduling of the VMM threads,
  2. Being too busy executing guest code that we don't notice the
     expired timer, and
  3. Dispatching and executing the interrupt handler of the guest.

The 1st point is unavoidable since we're not in control over the OS.
We're sharing the CPU resources with the rest of the system and we're
subject to scheduling.

The 2nd point is being addressed by upcoming hardware where it will be /
is possible to program VT-x/AMD-V to stop execution of guest code after
a given time. But for the time being, if a guest executes CPU intensive
code that doesn't touch any of the time sources and doesn't cause any
other virtualization exits/traps, we won't get a chance to check for
expired timers until a host interrupt occurs.

The 3rd point is related to the 1st one. It might take a little while
before we manage to dispatch the interrupt to the guest's interrupt gate
and execute it. Partly because we have to do a little bit more work that
real hardware does when it receives an interrupt, and also because we
can be preempted while doing this.

The first two points will result in delays in the execution of the VMM
timers, which in turn will propagate down to the PIT interrupt for
instance. We cope with this problem by gradually speeding up the clock
as we're lagging behind. The lag is measured by how late we're executing
the VMM timers for the typical timer and time source devices (PIT, RTC,
ACPI, APIC, HPET and TSC). In order to provide the guest with a
consistent set of time source, the previously mentioned device will use
a special clock which pauses when there are expired timers waiting to be
executed and is speed up to catch up the lost time. (The alternative
would be to use a clock which keeps on moving when for instance the PIT
interrupt timer expires. This would for instance result in that the PIT
counter will not necessary be zero when the interrupt is raise and upset
the guest. Or in completely incorrect results when the guest pits the
TSC against the RTC or PIT to measure the CPU frequency or TSC reliability.)

The speed up currently starts at 5% and slowly increases to 500% as we
lag more behind. Since there is no guarantee that we will be able to
catch up (CPU is over committed and other factors), we will give up
after 60 seconds (that's 60 seconds monotonic host time). Note that
these parameters are all subject to change (tuning).

Back to your scenario, high precision timers in a RT linux guest running
on Windows XP. From the guest point of view, the timers will be running
almost as expected. From the atomic wall clock point of view, the high
precision guest timers will frequently be delivered as expected, but
sometimes a longer interval occurs and it is followed by a series of
shorter ones. From the VMM point of view, it's an best effort kind of
thing. :-)

Hope this was of some help to you.

-- 

Kind regards / Mit freundlichen Gruessen / Vennlig hilsen,
  Knut

--

Sun Microsystems GmbH        Knut St. Osmundsen
Werkstrasse 24               Senior Staff Engineer, VirtualBox
71384 Weinstadt, Germany     mailto:bird at sun.com


================================================
Sitz der Gesellschaft: Sun Microsystems GmbH,
Sonnenallee 1, 85551 Kirchheim-Heimstetten
Amtsgericht Muenchen: HRB 161028
Geschaeftsfuehrer: Thomas Schroeder,
Wolfgang Engels, Dr. Roland Boehmer
Vorsitzender des Aufsichtsrates: Martin Haering
================================================