Gesture Recognition Data

Top  Previous  Next

The SDK finger tracking module can recognize gestures defined in a gesture pack. Each gesture pack defines a set of gestures to be recognized by the module. By default, the SDK preloads a set of predefined gestures, listed in Table 8. To load additional gesture pack, use the LoadGesturePack function.

Working with multiple gesture packs is a reserved feature. The SDK currently works with the default gesture pack only.

Illustration

Gesture Name

Description

spreadfingers

Open the hand completely (stretch all fingers). The hand should be facing the camera and with the index/middle/ring fingers pointing upwards.

fist

All fingers folded into a fist. The fist can be in different orientations as long as the palm is in the general direction of the camera.

tap

Keep your hand in a naturally relaxed pose and move it in the Z direction as if you are pressing a button.

thumb_down

Stretch down the thumb and fold the remaining fingers. This pose can be in different orientations as long as the orientation of the thumb is down.

thumb_up

Stretch up the thumb and fold the remaining fingers. This pose can be in different orientations as long as the orientation of the thumb is up.

two_fingers_pinch_open

Pinch with the index and thumb fingers. The hand orientation is vertical. The pinched fingers can be anywhere between pointing directly to the screen or in profile. Other fingers can be stretched or relaxed but not folded, and can be anywhere between completely spread or closed together and touching.

v_sign

Extend the index and middle fingers, and curl the other fingers. This pose is not sensitive in the hand orientation as long as the fingers are stretched up.

full_pinch

Pinch with all fingers extended and touching the thumb. The pinched fingers should be pointing to the screen.

Swipe left

Swipe right

swipe

Swipe to the left with the right hand or swipe to the right with the left hand. The swipe moves the hand from one side to the other when the palm is facing the side and fingers are more or less towards the camera. The hand swiping must be inside the camera field of view.

wave

Move the hand to the left and then the right. Repeat the action if necessary.

Table 8: Predefined Gestures

Programming Aspects

Use the EnableGesture/DisableGesture function to enable/disable a specific gesture, as illustrated in Example 60 or the EnableAllGestures function to activate recognition of all the known gestures.

C++ Example 60: Load a Gesture Pack and Enable a Gesture

// hcfg is a PXCHandConfiguration instance

// Load a gesture pack

hcfg->LoadGesturePack(L"navigation");

 

// Enable a gesture

hcfg->EnableGesture(L"swipe");

 

// Apply changes

hcfg->ApplyChanges();

C# Example 60: Load a Gesture Pack and Enable a Gesture

// hcfg is a PXCMHandConfiguration instance

// Load a gesture pack

hcfg.LoadGesturePack("navigation");

 

// Enable a gesture

hcfg.EnableGesture("swipe");

 

// Apply changes

hcfg.ApplyChanges();

Java Example 60: Load a Gesture Pack and Enable a Gesture

// hcfg is a PXCMHandConfiguration instance

// Load a gesture pack

hcfg.LoadGesturePack("navigation");

 

// Enable a gesture

hcfg.EnableGesture("swipe");

 

// Apply changes

hcfg.ApplyChanges();

Gesture Event

If a gesture is enabled, the module tracks the gesture formation and fires the gesture events accordingly. The gesture events can be continuous or discrete. For continuous gestures, the module fires the event throughout the gesture formation process. In the case of discrete gestures, the module fires the gesture event only at the start and stop moment of the gesture.

There are two ways for polling the gestures that are fired in a specific frame:

1.Query a specific gesture using the IsGestureFired function. The gesture parameters are returned in the GestureData structure, as illustrated in Example 61.
2.Query all the gestures in the frame using the GetFiredGesturesNumber function. Iterate the number of gestures, and then call the GetFiredGestureData function for each one.

C++ Example 61: Poll the Gesture Status and Data

// hdata is a PXCHandData instance

PXCHandData::GestureData data;

if (hdata->IsGestureFired(L"swipe", &data)) {

  ...

}

C# Example 61: Poll the Gesture Status and Data

// hdata is a PXCMHandData instance

PXCMHandData.GestureData data;

if (hdata.IsGestureFired("swipe", out data)) {

  ...

}

Java Example 61: Poll the Gesture Status and Data

// hdata is a PXCMHandData instance

PXCMHandData.GestureData data=new PXCMHandData.GestureData();

if (hdata.IsGestureFired("swipe", data)) {

  ...

}

You can also use the SubscribeGesture function to provide a callback function for event notification, as illustrated in Example 62.

C++ Example 62: Gesture Notification Throught Event Callback

// hcfg is a PXCHandConfiguration instance

 

class MyHandler:public PXCHandConfiguration::GestureHandler {

public:

 

   virtual void PXCAPI OnFiredGesture(const PXCHandData::GestureData &data) {

      ...

   }

};

 

// Subscribe to the gesture notification event

MyHandler handler;

hcfg->SubscribeGesture(&handler);

hcfg->ApplyChanges();

 

...

 

// Unsubscribe

hcfg->UnsubscribeGesture(&handler);

hcfg->ApplyChanges();

C# Example 62: Gesture Notification Through Event Callback

// hcfg is a PXCMHandConfiguration instance

 

void OnFiredGesture(PXCMHandData.GestureData data) {

   ...

}

 

// Subscribe to the gesture notification event

hcfg.SubscribeGesture(OnFiredGesture);

hcfg.ApplyChanges();

 

...

 

// Unsubscribe

hcfg.UnsubscribeGesture(OnFiredGesture);

hcfg.ApplyChanges();

Java Example 62: Gesture Notification Through Event Callback

// hcfg is a PXCMHandConfiguration instance

 

class MyHandler implements PXCMHandConfiguration.GestureHandler {

   public void OnFiredGesture(PXCMHandData.GestureData data) {

      ...

   }

};

 

// Subscribe to the gesture notification event

MyHandler handler=new MyHandler();

hcfg.SubscribeGesture(handler);

hcfg.ApplyChanges();

 

...

 

// Unsubscribe

hcfg.UnsubscribeGesture(&handler);

hcfg.ApplyChanges();