Developing Multi-Touch Rotation and Letter Recognition Apps Using C# in Windows 8*

I have to say it: Windows 8* Style UI programming is a lot of fun.  One thing I like is how it's so easy to incorporate multi-touch support.  After writing some sample code the other day, I thought I would share my findings.

Please note: This blog assumes familiarity with Windows 8* Style UI programming using the C# programming language, XAML designer, and the Visual Studio 2012 IDE.  Code is current as of the Windows 8 RTM release. For a primer on general touch programming for Windows 8, take a look at my article here: http://software.intel.com/sites/default/files/m/9/0/5/0/5/44966-Enabling_Touch_in_Windows_8_with_C.pdf

When dealing with multi touch, it is commonplace to use the manipulation event APIs that are provided in the Windows 8 Style UI framework.  These APIs are complete in that you can handle anything from rotation, to flinging objects with a quick swipe, or even custom gesture behavior.  The aforementioned link includes the reference material needed for a primer on manipulation events.

This blog will focus strictly on rotation gestures.  Two programming examples are shown:

·       Two-finger rotation of an image

·       One finger rotation by drawing the letter "R"

Let's start with the first case.  Two-finger rotation detection is quite easy.  One thing to point out is that by default, the pivot point is null.  This means that unless you specify a pivot point otherwise, this gesture uses at least two fingers, with the first one making contact acting as the pivot.  The default manipulation event API framework actually makes this rotation detection simple to use; no fancy custom code is needed.  Yes, it’s that simple!

For the second case, Windows 8 provides some APIs for letter recognition through the use of handwriting.  Specifically, InkPen is used where letters can be recognized when the user uses the handwriting mode of the Windows keyboard.  This API isn't limited to letter detection, but can detect other symbols like numbers, etc.  While the API is quite simple to use, for the sake of demonstration, I will show how you can actually use manipulation events for single letter detection.  Read on to find out more!

Glance back here if needed: http://msdn.microsoft.com/en-us/library/windows/apps/xaml/hh465387#using_manipulation_events

 

If you notice, the link demonstrates how to use the ManipulationMode keyword in XAML.  Hereon, it's assumed that the reader is acquainted with defining the manipulation event handlers. 

Let’s start with two finger rotation!  For this example, I defined manipulation events as follows:

            //called when pressing finger down on image

            imageToRotate.ManipulationStarted += manip_start;

           

            //while dragging (rotating) the image

            imageToRotate.ManipulationDelta += manip_delta; 

 

Figure 1: Writing the Manipulation Events **

 

Let's take a look at the body of manip_start, when the gesture first begins.

      

                RotateTransform tran = new RotateTransform();

                tran.Angle = _curAngle;

 

                //rotate about the center of the image

                tran.CenterX = imageToRotate.ActualWidth / 2;

                tran.CenterY = imageToRotate.ActualHeight / 2;

 

                //update the on-screen image using the transform

                imageToRotate.RenderTransform = tran;

 

 

Figure 2: Handling Basic Rotation **

 

 

A few things are going on here.  First, I use the initial orientation angle of the image as the current angle for this manipulation event.  This is in the event that the user has previously rotated the image to some angle.  A transformation matrix is then created, specifying that the rotation will occur about the x and y midpoints (the center) of the image.  We then update the image's transform matrix.  Of course, for starting the manipulation, angle hasn't changed quite yet and we really didn't "need" to update the RenderTransform.  I just happen to show it for sake of discussion.

When the image is then dragged, we see this routine come into play: 

void manip_delta(object sender, ManipulationDeltaRoutedEventArgs e)

        {

            if (_currentSensorMode == SensorMode.TOUCH_ROTATE)

            {

                //time to rotate the image!

                RotateTransform tran = new RotateTransform();

 

                _curAngle += e.Delta.Rotation;

 

                tran.Angle = _curAngle;

 

                //rotate about the center of the image

                tran.CenterX = imageToRotate.ActualWidth / 2;

                tran.CenterY = imageToRotate.ActualHeight / 2;

 

                //update the on-screen image using the transform

                imageToRotate.RenderTransform = tran;

 

Figure 3: Dragging Images **

 

All that's really different from the first routine is that we simply update the current angle with e.Delta.Rotation.  How can it be that simple?   Well it is if you are simply just trying to do the basics with multi-touch.  Note that for the pivot point, you don't have to use the image center; as hinted at earlier, you could use the first point of contact (x and y can be derived from the event passed into the previous routine and then saved).

OK, so that wasn't too bad.  Now the fun: what if I told you that your code must support rotating the image if you used one finger to draw the letter "R"?  Specifically, let’s see if we can do this without the InkPen API.  Note:  we have to allow for slack in the x and y components as drawing a perfect "R" would be exceedingly difficult, right?

So, to tackle this exercise, we again start with the same ordeal.  The model still uses manipulation events.  However, given we are using one finger to draw a particular letter, we must now track coordinates to ensure the user is drawing what we want.

Now, this dilemma suggests that we probably should use a little trigonometry to account for the following:

·       We should consider min and max lengths.  For example, when drawing the left leg of the letter "R," we want a natural feel for drawing it and so we don't want to allow too short a leg length, right?  Also, we may want to limit the size too because it made me too laborious to draw a leg that spans the entire screen dimension!

 

·       Sin / cos: it may be more precise and a better design to use these trig functions to allow for an error in the angle adjoining two points of the letter "R" that the user draws.  For example, if drawing the left vertical leg, ideally, if the angle theta is considered to be the angle the leg makes with the bottom left corner of the left, we would see that cos theta = 0 as there should be no x component.  The amount of slack here is up to you.  In fact, you don't have to use trig functions specifically, but could instead use triangular lengths.  If we draw a triangle between any two drawn points (well, unless the line is horizontal or vertical of course), we could instead focus on x and y component lengths.  That's the approach I used in the example below for simplicity.

 

For the code to follow, use the following illustration to follow along:

 

  

In this example, x1 is the point of contact that starts the manipulation.  Then, the user will drag the finger (without releasing as this ends the manipulation), which constantly invokes the delta event, until we find that x2 meets our error and length constraints.  We follow with x3, and so on.

The question now is: OK, so how do we remember points?  This snippet for the manipulation start shows how.

 

                //user is beginning to draw the letter "R" via touch

                if (draw_state == 0)

                {

                    x1 = new Point(e.Position.X, e.Position.Y);

                    draw_state++; //mark that x1 has been last drawn

                } 

 

Figure 4: Letter Recognition as a State Machine**

 

Here, I use draw_state to indicate the last point xi the user has drawn, for 1 <= i <= 5 in this case (well, we don't really have to remember x5 ).  We can think of the variable as depicting a finite state machine that "accepts" the user input if the variable reaches a value of 5. 

When the user drags the finger, we must now constantly track the user's finger position to see if it's within bounds of the next target point.  Remember, drawing doesn't have to be perfect!

 

//user is beginning to draw the letter "R" via touch

if (draw_state > 0) //user must have started this special gesture

{

Point xi = new Point(e.Position.X, e.Position.Y);

 

if (draw_state == 1) //did user draw left "leg" of "R"

{

 

 

if ((xi.Y < x1.Y && Math.Abs(xi.X - x1.X)<x_tol) && distance(x1,xi) > min_dist)

{

//success: advance draw state, user drew left edge of "R"

x2 = xi;

draw_state++;

 

}

}

 

else if (draw_state == 2) //did user draw top right point?

{

if (xi.X > x2.X && (xi.X - x2.X) < x_tol && distance(x2, xi) > min_dist)

{

//success: advance draw state, user drew left edge of "R"

x3 = xi;

draw_state++;

 

}     

}

 

//did user draw to the left for the middle of the letter?

else if (draw_state == 3)

{

if ((xi.X < x3.X) && (xi.X - x3.X) < (2 * x_tol) && distance(x3, xi) > (min_dist/2))

{

//success: advance draw state, user drew left edge of "R"

x4 = xi;

draw_state++;

 

}

}

 

//did user draw to the left for the middle of the letter?

else if (draw_state == 4)

{

if ((xi.X > x4.X) && distance(x4, xi) > min_dist && (xi.Y > x4.Y))

{

//success: advance draw state,

//user drew left edge of "R"

x5 = xi;

draw_state = 0; //gesture completed!!

 

//now rotate the image!

RotateTransform tran = new RotateTransform();

_curAngle += 180.0f;

tran.Angle = _curAngle;

 

//rotate about the center of the image

tran.CenterX = imageToRotate.ActualWidth / 2;

tran.CenterY = imageToRotate.ActualHeight / 2;

 

//update the on-screen image using the transform

imageToRotate.RenderTransform = tran;

}               

 

Figure 5: Handling Draw States by Cases ***

 

Whoa!  Definitely more involved than simple MTI detection.  Let's break down the code into phases:

·       Retrieve the current drag point (1)

·       Check the last point drawn (2)

·       Use current point distance from last point and based on state, check if we move to next state (3)

 

Clearly, (1) is a given.  Now, (2) of course just depends on the current state of our machine.  Notice that I save each point.  You could get away with just saving the last point though if you wish.  In (3), I use a helper routine for calculating the distance formula.  Notice that I also have defined some variables that provide both the slack for drawing a line in terms of x and y components along with metrics that ensure what you drew is both not too short and not too long.

 

If the user has drawn the final point of the letter, I simply rotate the image 180 degrees to just give feedback that the user drew the right letter.

 

Well, that wraps up my discussion on quickly getting hands on with rotation code in Windows 8.  I hope you enjoyed reading this blog and feel free to comment with suggestions, questions or comments. 

 

**This sample source code includes XAML code automatically generated by Visual Studio IDE and is released under the Intel OBL Sample Source Code License (MS-LPL Compatible)

***This sample source code is released under the Microsoft Limited Public License (MS-LPL)

 

 

 

Для получения подробной информации о возможностях оптимизации компилятора обратитесь к нашему Уведомлению об оптимизации.