Adding Multi-Touch Support in Unity* Windows* Apps

Someone asked me recently how to take advantage of multi touch in Unity apps running on Windows 7 & Windows 8 desktop.  I remember saying things like “it should be really easy”, and “Someone must have done that” and “go look for a plugin”.  He came back and told me he couldn’t find one, so I looked myself and sure enough lots of people wanted one but nobody had one that they were willing to admit.  So, time to start rooting about in code.

My first idea was to write a plugin which maintains an overlay window on top of the main Unity window which would capture input, use the touch messages, and pass everything else to Unity.  Not too hard to put an initial window there and catch the WM_TOUCH messages, “Great”, I thought, “this is gona be easyJ”.  Hmm, things went downhill from there.   I don’t know if you’ve tried this but it’s a nightmare trying to keep an overlay on top of another window, not to mention a direct 3D one, and trying to keep focus when you have minimizing, different windows themes, and let’s not forget recovering from hibernation and stuff like that.  I was beginning to feel like “I don’t really want to write this now”, when the final straw came with the realization that when Unity goes full screen, it won’t work.  If another window has focus, Unity will minimize.  Bah!

Ok so I started to play around in the plug-in and it suddenly occurred to me that you don’t need an overlay window at all.  All you need to do is attach a Windows Hook to the Unity window to capture messages, register the Unity window for touch, and sit back and process the touch messages.   I love those moments – they’re what make programming so rewarding!

So, what do we need to achieve it?

1.       I need to ensure that I have the correct window handle.

I do this by passing the name of the app from a Mono script to an initialize function in the plugin.  Once we have the name, the function can iterate through the desktop windows until we find the one we’re looking for.  This means we have to call EnumDesktopWindows() and have a callback like below.

 

 

EnumDesktopWindows() takes a EnumWindowsProc() as a parameter and repeatedly calls the EnumWindowsProc(), once for each desktop window.

Here is a snippet from the initialize function in the plug-in, which obtains the correct window handle for the Unity window using EnumDesktopWindows() and the above CALLBACK.   

The return -1 should be handled in the Mono script – you will see below that mine doesn’t, so please don’t mail and tell me.  That kind of code will be application specific, so I’ve left it out. So, now we have the Window handle, or we returned an error.

 

2.       Now we come to attaching the hook.  To attach a hook, you need to know the handle of the DLL module, and the thread ID from the Unity window.  The DLL handle is easy, that arrives via the DLLMain() function:

 

 

And, the Unity window thread ID you can get from generously named GetWindowThreadProcessID() . Once you have the Module handle and the Thread ID then you just call SetWindowsHookEx() to attach a message function of your own.  Like this:

 

 

You will notice I attach 2 hooks here.  This is because (as I found while writing this) that on Windows 7 the WM_TOUCH messages are posted to the window message queue.  These you can trap with the first hook which has the type WH_GETMESSAGE.  However, on Windows 8 desktop, the WM_TOUCH messages are sent directly to the window procedure, so you need to catch them by attaching a WH_CALLWNDPROC. 

 

The 2 hook procs are pretty straight forward, this is them:

 

 

For the OnTouch() function, I used the code from one of the Microsoft samples (one example here) .  This will call GetTouchInputInfo() yada yada yada, to get the touch data.

 

3.       Finally we need to enable the Unity window for touch messages.  This doesn’t seem to break anything in Unity, although as a caveat I’ll say I haven’t tried it on a very large or complex app: I don’t have one.  Should be ok as far as I can see.

 

Those UnhookWindowsHookEx() calls are very important.  Hooks are a shared resource so you have to make sure you play fair and give them back when you’re done with them.  Your game will have to try to ensure this is the case when you exit under normal conditions as well.

 

4.       On the Unity side, we just need a basic interface to the plug-in from a Mono script. 

 

First thing you’ll notice is that I don’t initialize the plugin in Start().  I thought it better to wait until the app is completely established and running before attaching the hooks etc.  In the onetime initialization code in OnGUI(), I pass the window name to the DLL so the DLL can do its thing on the established window.  All the DllImport stuff at the top is well known to plugin writers so I won’t go in to all that. 

I hope this makes sense to everyone, it’s a fairly easy way to get Multi touch input into a Unity app on Windows 7 and Windows 8. 

Happy coding!

 

SteveH

 

For more complete information about compiler optimizations, see our Optimization Notice.

20 comments

Top
Matthieu O.'s picture

Great, I solved my problem !

Abhishek 81's picture

Great Information Steve.Even I implemented touch,tap,rotate and basic gesture for my Windows 8 prototype for Intel Perceptual Challenge.This article will be helpful to me,thanks.

Matthieu O.'s picture

Hi Steve,
I can handle the touch events but the strange thing is that I can't retrieve the touch count or position. As Daniel said, maybe you could post or email us a working project sample.
Thanks a lot.
Regards,
Matthieu.

Daniel K.'s picture

Hi Steve,

I have less experience with Windows .NET DLL side of things, and was wondering if you would mind posting or emailing us a zipped up project file that works - it only has to handle being able to receive basic touch events inside Unity. We have been trying to find a solid solution to the Unity / Windows 7 Multi-Touch dilemma for quite some time now, but to no avail.

We had luck when we were using NextWindow's touch screen overlays as they were kind enough to provide an SDK which I managed to get working with Unity, but for some reason Windows 7 Multi-touch support under Unity has been very evasive.

Regards,

Daniel.

steve-hughes (Intel)'s picture

Hi Matthieu

Without seeing your code, its hard to say whats going wrong. I would check that 1. you definately have the correct window name so that you are counting the correct messages, and 2, watch those types over the interface - make sure you have exactly the same type on both sides.

Cheers

Steve

Matthieu O.'s picture

Hi, thanks a lot for your tutorial but I have an strange issue. OnTouch is well called but when I assign UINT cInputs to my global int g_TouchCount (returned by int GetTouchPointCount() to Unity), it's always equals to zero. Do you have any idea from where it comes from ?
Thanks.

Kashyap S Phukan's picture

thanks steve, I got the dll working with the help of your example and some research on msdn. Thanks for the example tough. Actually I m working in your software development partner program, developing a game for the new ultrabook, got as a development platform from intel.

Thanks.

anonymous's picture

Hi

Yes, I remember funbox :). I'm not familiar with UDK, but from reading the page it looks to me like you just write a standard DLL, and export the functions you need. If you dont know how to make a dll, the easiest way is with Visual Studio.
In VisualStudio, create a new project, and select dll to create the basic source you need. Then you can include the code from the webpage. Just add the extern "C" section containing the exported functions which have "__declspec(dllexport)" as a prefix to the function.

I hope this helps.

Steve

Kashyap S Phukan's picture

Hello Steve.

I hope you are the Steve, who gave us information about directX graphics and optimization via intel tools at dare to be digital, 2011. I am from team funbox. I am struggling with this problem in UDK's DLL binding. Can you please guide me through this as I am not that great at windows native programming. I understood about the touch as you mentioned in the reference links. The problem is the complete structure of the DLL program, that's suits UDK because I am weak on native C++.

As for your reference, here is a link to Unreal Documents over DLL Binding.
:: http://udn.epicgames.com/Three/DLLBind.html

I am trying this for a game on intel Ultrabook.

Thanks for your time.

Pages

Add a Comment

Have a technical question? Visit our forums. Have site or software product issues? Contact support.