As part of my research project, I am currently implement my own security kernel. One of the key parts is, naturally, a user/kernel separation. For this I intended to use the sysenter/sysexit combination. Sysenter seems to work correctly, but sysexit causes problems. Digging through the manual I even found an inconsistency. In the developer's manual 3.A chp 18.104.22.168. It states that sysexit causes the use of the following stack segment: IA32_SYSENTER_CS + 40 (in ia32e mode). The manual 2.A at page 4.500 however, states that IA32+SYSENTER_CS + 8 is used.
Trying to resolve the issue, I looked at how Linux implements system calls. Unfortunately it uses syscall/sysret instructions. Surprisingly however, it sets the SYSRET CS value to 0x13. The two least significant bits are set to allow user mode to access the code and stack segments. How is this resolved with the sysenter/sysexit instructions?
Thanks in advance,