Archived - What could you do with Intel RealSense Cursor Mode?

The Intel® RealSense™ SDK has been discontinued. No ongoing support or updates will be available.

cursor mode

How do you navigate or control a presentation when you don't have a keyboard/mouse or can't touch the screen? How can you reduce fatigue and allow more natural interaction of hands and computer? What software will allow you to "in-air" draw and be tracked to this level of accuracy?

The Intel® RealSenseTM SDK 2016 R1 (for Windows) introduces a new mode for tracking moving hands called Cursor Mode, which uses a single depth sensor. Available for the Intel® RealSenseTM SR300 user facing short range camera on Windows* 10, Cursor Mode provides a very fast and accurate way to track a single point representing the general position of the hand. This point does not represent a specific joint – instead, it is dynamically determined, allowing a smooth and responsive cursor action. 

Cursor Mode can track twice as fast, while using about half the power, compared to using full hand tracking mode. Cursor mode does not require calibration and tracks 1 or 2 hands in 2D or 3D.  The module supplies 3 coordinate systems:

  • 2D image coordinates (in pixels),
  • 3D world coordinates (in meters) and
  • 3D adaptive coordinates (relative position of each axis normalized inside an adaptive bounding box.

The Cursor module also supports a small, yet robust, set of "motion" gestures (click, hand closing, hand opening, circle clockwise, and circle counterclockwise). The least movement occurs with click, where the hand itself shouldn't move, just the fingers. In contrast, hand tracking identifiies particular hand positions (also called gestures), which at most include minor motion such as a one direction single wave. 

A comparison of the Hand Tracking vs Cursor modules can be seen in this chart:

FEATUREHAND TRACKCURSOR MODE FEATUREHAND TRACKCURSOR MODE
Supported CamerasF200, SR300SR300 only # of Hands22
Track22 joints1 point of interest Gestures12increasing
Latencynoticeable0 @ 60 FPS SegmentationYesNo
CalibrationNeedednot needed Joint RotationYesNo
Confidencelowhigh Power UsageHighLow
Motion Speed<.75-1.5 m/sup to 2 m/s Main Usefull hand mapUI control
Range SR300
            F200
85 cm
60 cm
~ 110 cm
not supported
 OS SupportWin 10
Win 8.1F200
Win 10
N/A
Rez/FPS640x480x60640x480x60 Min/Max FPS50/6060

NOTE: In the 2016 R1 release (v8), you cannot use Cursor mode while using Full Hand tracking mode. This will be fixed in future versions. 

Like hand tracking, cursor mode is most effective when the user gets visual feedback on screen, especially for hands being in or out of Field of View (FOV).

Another advantage of cursor mode is that hands don't have to occlude the screen if Adaptive Point mode is implemented. All activity can be set to occur in a bounded box. The advantages of Adaptive Point mode include greater reaction for less motion making it more responsive in some usages. But in doing so, it has less granularity (fine detail) than Cursor Point mode.  Cursor Point mode allows more detail and is better for drawing. .

Engagement Mode is the way to control when the user's hand movement is picked up as an interaction with the camera as opposed to the user just using his hands while speaking. Configure when to pay attention to the hand motion using EnableEngagement  with the option to use SetEngagementTime to show a progress bar (0-100%)

While improving in each SDK release, the module is only at beta level in the 2016 R1 release and still undergoing some changes. Limitations that currently exist include :
> A minimum palm width of 5.5cm (commonly about age 5 and up)
> Frame rate of 50-60 FPS (lower than 50 will have blank areas)
> Range of 20-110cm from the camera
> 2016R1 detection distance is only 70cm with a max range of ~110cm
> User cannot be holding any object in their hand(s)

Further improvement to gestures and range is planned as well as adding Drag and Drop plus 1 and 2 hand rotation of 3D objects. At some point in the future, back of hand support will be added. 
 

The Intel® RealSense™ Developer Kit featuring SR300 is now available from http://click.intel.com/realsense.html

For more complete information about compiler optimizations, see our Optimization Notice.

2 comments

Top

Like You talked about the confidence levels and etc. but it would be good if you tell us about the OS being operated on 

Hi can you please tell more about the hand track ?

Add a Comment

Have a technical question? Visit our forums. Have site or software product issues? Contact support.