[vbox-dev] Modifying CSAM/PATM
Sander van Leeuwen
Sander.Vanleeuwen at Sun.COM
Fri Apr 16 07:11:49 PDT 2010
The software virtualization code scanner was not designed to catch all
occurrences of certain instructions, but
rather as a helper for static scanning upfront and dynamic scanning
after hitting certain faulting instructions.
You can modify CSAM to inject int 3 patches (0xcc instructions) for all
call and ret instructions it encounters.
When you hit such an int3 patch, then you can do whatever it is you wish
and invoke the scanner for a target address
you haven't encountered before. It shouldn't be very hard to adjust the
code to do just that and to emulate all ret and
The only tricky thing is dealing with self-modifying code. We already
invalidate patches when unsupported modifications occur, but
you'll have to take this into account for further scanning too.
On 15-4-2010 23:21, Martim Carbone wrote:
> Thank you for your replies to my previous question "VBox CSAM/PATM in VMX mode".
> I think I should probably make my intentions clearer. My ultimate goal
> is to be able to intercept, from inside the hypervisor, all
> kernel-level executions of CALL and RET instructions performed by a
> guest OS. I understand the performance implications of this.
> Considering the role that CSAM/PATM already play, it seemed natural to
> me that these could be extended so that kernel CALL and RET
> instructions could be dynamically "patched" (as already done with
> sensitive operations) with code which would trap to the hypervisor
> upon execution. If CSAM/PATM are not used in VMX mode, I can live
> without VMX. The question really is to what extent would CSAM/PATM
> need to be modified to implement this kind of CALL/RET tracing.
> I'm not expecting a step-by-step guide, of course; rather an estimate
> from more experienced Vbox developers (as I'm not myself very
> experienced) as to what amount of effort would be involved in this
> addition and any possible pitfalls that I should watch out for when
> modifying the code.
> Thanks again,
> -- Martim
> vbox-dev mailing list
> vbox-dev at virtualbox.org
Kind regards / Mit freundlichen Gruessen / Met vriendelijke groet
Sun Microsystems GmbH Sander van Leeuwen
Werkstrasse 24 Senior Staff Engineer, VirtualBox
71384 Weinstadt, Germany mailto:Sander.Vanleeuwen at sun.com
Sitz der Gesellschaft: Sun Microsystems GmbH,
Sonnenallee 1, 85551 Kirchheim-Heimstetten
Amtsgericht Muenchen: HRB 161028
Geschaeftsfuehrer: Thomas Schroeder,
Wolfgang Engels, Dr. Roland Boehmer
Vorsitzender des Aufsichtsrates: Martin Haering
More information about the vbox-dev