Virtualization and Performance: Understanding VM Exits

VM exits in response to certain instructions and events (e.g., page fault) are a key source of performance degredation in a virtualized systems. But have you ever wondered why? What exactly happens during a VM exit anyway?

A VM exit marks the point at which a transition is made between the VM currently running and the VMM (hypervisor) which must exercise system control for a particular reason. In general, the processor must save a snapshot of the VM's state as it was running at the time of the exit. For Intel architectures, here is an approximation of the steps:

1. Record information about the cause of the VM exit in the VM-exit information fields (exit reason, exit qualification, guest address), and update VM-entry control fields.

2. Save processor state in the guest state area. This includes control registers, debug registers, MSRs (see next item), segment registers, descriptor-table registers, RIP, RSP, and RFLAGS, as well as non-register state like pending debug exceptions.

3. Save MSRs in the VM-exit MSR-store area. MSR stands for Machine Specific Registers, in case you are not familar. They are used to control and report on processor performance.

4. Load processor state based on the host-state area and some VM-exit controls. This includes host control registers, debug registers, MSRs, host table and descriptor-table registers, RIP, RSP, and RFLAGS, page-directory pointer table entries, as well as non-register state.

5. Load MSRs from the VM-exit MSR-load area.

Oh, and don't forget that after the VMM has performed its system management function, then a corresponding VM entry will be performed that transitions processor control from the VMM to the VM!

Now you can see why VM exits generate considerable overhead, to the tune of hundreds or thousands of cycles for a single transition. To mitigate this problem, BTW, considerable effort has gone in to both reducing the number of cycles required by a single trasition, and creating features that obviate the need for system management (and hence VM exits) in the first place.

For more information, see the "Intel 64 and IA-32 Architectures Software Developer's Manual" at In particular, take a look at chapter 23 in Volume 3B: System Programming Guide, Part 2.

David Ott

For more complete information about compiler optimizations, see our Optimization Notice.