Touch keyboard access for Windows* 8 desktop apps

What do you do if your DirectX* desktop game requires keyboard input, but is running on a Windows* 8 platform without a physical keyboard? If your game runs in a window, the user can manually invoke the touch keyboard from the taskbar but that can be unintuitive, and doesn't help at all for full-screen games. You could write a custom keyboard overlay within your game.  That approach can be harder than it seems at first blush, with lots of corner cases to verify with language support, modifier keys, and event handlers to write and debug. 

Things would be much simpler if you could access the built-in keyboards provided with the OS. Unfortunately there is no Win32 API call to invoke the keyboard, so this approach is also not as simple as it could be. It is possible though, and by leveraging your existing keyboard handling it can save quite a bit of development time. This sample illustrates one method of invoking the built-in on-screen keyboards, and points out some of the modifications needed to do so seamlessly.

 

The Windows* touch keyboard has a fixed location in the lower half of the display.  Hint: if your application uses keys that are not part of the default reduced key-set, enable the full keyboard layout via Settings -> Change PC Settings -> General -> Make the standard keyboard layout available.

  

The alternate keyboard is not quite as seamless, but has the advantage that it's location and size can be controlled programmitcally via registry settings.  If your game's input field(s) are positioned where the touch keyboard would obscure them, you'll want to use this keyboard instead.

Many desktop games will be written using full-screen exclusive mode.  This presents a problem when using the built-in keyboards, because either the invoked keyboard will be hidden, or raising the keyboard will force the game out of full-screen mode.  Transitions in/out of full-screen exclusive mode come with a raft of resource re-acquisition or re-sizing considerations, and are generally a pain point that you don't want to add to your keyboard input handling.  The solution demonstrated in this sample is to run the game in a maximized borderless window, rather than full-screen exclusive mode.  I listed some of the details related to implementing borderless windowed mode in a previous blog post.  The advantage of using a borderless full-screen window is that raising the keyboard window does not disturb then game other than temporarily pushing it one level down the z-stack. The game continues to run in the background to process keyboard input, and it regains focus automatically as soon as the keyboard is dismissed.

We can hope for more explicit control of the Windows* keyboards in future APIs, but for the moment the workarounds demonstrated in this sample are one way to achieve on-screen keyboard input today. 

AttachmentSize
Package icon borderlesswindowkeyboard.zip5.53 MB
For more complete information about compiler optimizations, see our Optimization Notice.

7 comments

Top
Ashley's picture

Works a charm! Not sure that hard-coding C:/ is a good idea though r.e.

cString cmdLine(_L("C:\\Program Files\\Common Files\\Microsoft Shared\\ink\\tabtip.exe"));
HINSTANCE result = ShellExecute( NULL, ... ... )

 

Greg's picture

Once we use the Flip swap chain method, it is no longer possible to capture the buffer from the swap chain for screen shot or recording purposes.  The swapChain->GetBuffer() returns successfully, but the buffer contains flat gray pixels.

Dave, do you know of a way to capture the frame buffer contents for screen shot or recording? This is a useful or even necessary part of many applications.

Greg's picture

Just wanted to add that I've used the following methods successfully on Windows 8:

bool IsVisibleOSK()
{
    HWND KeyboardWnd = FindWindow("IPTip_Main_Window", NULL);
    return ( KeyboardWnd != NULL );
}

void HideOSK()
{
    HWND KeyboardWnd = FindWindow("IPTip_Main_Window", NULL);
    PostMessage(KeyboardWnd, WM_SYSCOMMAND, (int)SC_CLOSE, 0);
}

Greg's picture

It would be great if we could tell when the Windows 8 OSK becomes visible and when it is no longer visible so the app could shift text entry UI above the keyboard.

Thanks for responding Dave.  I'm looking forward to your 8.1 update.  I see some new DirectX interfaces have appeared.  By the way, I have not extensively tested, but found vsync was always forced on when using the flip swapchain.  Unfortunately vsync can dramatically affect the frame rate and usability of some real-time apps.

Dave Houlton (Intel)'s picture

@Greg: I didn't specifically test with vsync on/off, but It should be completely orthogonal to this technique. I expect both keyboards will work regardless of vsync state.

The CPUT framework we use for graphics samples continues to evolve, 8.1 support is underway. Thanks!

- Dave

Greg's picture

Very minor... When running with DX11 debug, a resource leak is reported from:
CPUT\CPUT_DX11.cpp
Line:
hr = mpD3dDevice->QueryInterface(__uuidof(IDXGIDevice2), (void **)&pDXGIDevice);
So add:
SAFE_RELEASE(pDXGIDevice);
after CreateSwapChainForHwnd.
Did not notice this before, could be the change to Windows 8.1 SDK.

Greg's picture

Thank you, great article, fantastic to use the supplied on screen keyboard instead of resorting to custom UI.

Is it possible to show the OSK with vsync off?

Add a Comment

Have a technical question? Visit our forums. Have site or software product issues? Contact support.