Adding Multi-Touch Support in Unity Windows Apps.
Someone asked me recently how to take advantage of multi touch in Unity apps running on Windows 7 & Windows 8 desktop. I remember saying things like “it should be really easy”, and “Someone must have done that” and “go look for a plugin”. He came back and told me he couldn’t find one, so I looked myself and sure enough lots of people wanted one but nobody had one that they were willing to admit. So, time to start rooting about in code.
My first idea was to write a plugin which maintains an overlay window on top of the main Unity window which would capture input, use the touch messages, and pass everything else to Unity. Not too hard to put an initial window there and catch the WM_TOUCH messages, “Great”, I thought, “this is gona be easyJ”. Hmm, things went downhill from there. I don’t know if you’ve tried this but it’s a nightmare trying to keep an overlay on top of another window, not to mention a direct 3D one, and trying to keep focus when you have minimizing, different windows themes, and let’s not forget recovering from hibernation and stuff like that. I was beginning to feel like “I don’t really want to write this now”, when the final straw came with the realization that when Unity goes full screen, it won’t work. If another window has focus, Unity will minimize. Bah!
Ok so I started to play around in the plug-in and it suddenly occurred to me that you don’t need an overlay window at all. All you need to do is attach a Windows Hook to the Unity window to capture messages, register the Unity window for touch, and sit back and process the touch messages. I love those moments – they’re what make programming so rewarding!
So, what do we need to achieve it?
1. I need to ensure that I have the correct window handle.
I do this by passing the name of the app from a Mono script to an initialize function in the plugin. Once we have the name, the function can iterate through the desktop windows until we find the one we’re looking for. This means we have to call EnumDesktopWindows() and have a callback like below.
The return -1 should be handled in the Mono script – you will see below that mine doesn’t, so please don’t mail and tell me. That kind of code will be application specific, so I’ve left it out. So, now we have the Window handle, or we returned an error.
2. Now we come to attaching the hook. To attach a hook, you need to know the handle of the DLL module, and the thread ID from the Unity window. The DLL handle is easy, that arrives via the DLLMain() function:
And, the Unity window thread ID you can get from generously named GetWindowThreadProcessID() . Once you have the Module handle and the Thread ID then you just call SetWindowsHookEx() to attach a message function of your own. Like this:
You will notice I attach 2 hooks here. This is because (as I found while writing this) that on Windows 7 the WM_TOUCH messages are posted to the window message queue. These you can trap with the first hook which has the type WH_GETMESSAGE. However, on Windows 8 desktop, the WM_TOUCH messages are sent directly to the window procedure, so you need to catch them by attaching a WH_CALLWNDPROC.
The 2 hook procs are pretty straight forward, this is them:
3. Finally we need to enable the Unity window for touch messages. This doesn’t seem to break anything in Unity, although as a caveat I’ll say I haven’t tried it on a very large or complex app: I don’t have one. Should be ok as far as I can see.
Those UnhookWindowsHookEx() calls are very important. Hooks are a shared resource so you have to make sure you play fair and give them back when you’re done with them. Your game will have to try to ensure this is the case when you exit under normal conditions as well.
First thing you’ll notice is that I don’t initialize the plugin in Start(). I thought it better to wait until the app is completely established and running before attaching the hooks etc. In the onetime initialization code in OnGUI(), I pass the window name to the DLL so the DLL can do its thing on the established window. All the DllImport stuff at the top is well known to plugin writers so I won’t go in to all that.
I hope this makes sense to everyone, it’s a fairly easy way to get Multi touch input into a Unity app on Windows 7 and Windows 8.