I am writing a thin hypervisor that allows 16-bit mode guests. The system boots into my 16-bit boot code which sets up 32-bit protected mode with identty mapped pages, then enables IA-32e compatibility mode and then switches into IA32e mode (64-bit). In this mode, the software sets up a hypervisor to allow unrestricted guests (this includes setting up EPT with proper caching controls refecting the cache setup via MTRRs). Then the software launches a 16-bit guest that runs well -- making BIOS calls for I/O services etc. All this is working quite well.
However, I noticed a small discrepency in behavior when I press the power button. Before the 16-bit guest is launched, the system immediately shuts down when the power button is pushed. This also happens when the host mode is active (i.e., my code is processing a VM Exit). However, when the 16-bit mode guest is active, pushing the power button causes the machine to hang -- even the VM preemption timer does not cause a VM Exit.
Because the power button causes an SMI, I decided to use the TCO watchdog timer to cause an SMI while the 16-bit mode guest is running. This also causes my machine to lockup. However, when I duplcate the code in native 16-bit mode code (i.e., before the VM launches), the watchdog timer does not cause any lockups.
This leads me to believe that something in my guest setup is reacting poorly with SMM code. Does anyone have insights into what I need to do to keep SMM happy? I am not using dual monitor mode VMM so the SMM should handle the SMI and return back without affecting my guest/hypervisor settings.