The Intel® RealSense™ SDK has been discontinued. No ongoing support or updates will be available.
How do you navigate or control a presentation when you don't have a keyboard/mouse or can't touch the screen? How can you reduce fatigue and allow more natural interaction of hands and computer? What software will allow you to "in-air" draw and be tracked to this level of accuracy?
The Intel® RealSenseTM SDK 2016 R1 (for Windows) introduces a new mode for tracking moving hands called Cursor Mode, which uses a single depth sensor. Available for the Intel® RealSenseTM SR300 user facing short range camera on Windows* 10, Cursor Mode provides a very fast and accurate way to track a single point representing the general position of the hand. This point does not represent a specific joint – instead, it is dynamically determined, allowing a smooth and responsive cursor action.
Cursor Mode can track twice as fast, while using about half the power, compared to using full hand tracking mode. Cursor mode does not require calibration and tracks 1 or 2 hands in 2D or 3D. The module supplies 3 coordinate systems:
- 2D image coordinates (in pixels),
- 3D world coordinates (in meters) and
- 3D adaptive coordinates (relative position of each axis normalized inside an adaptive bounding box.
The Cursor module also supports a small, yet robust, set of "motion" gestures (click, hand closing, hand opening, circle clockwise, and circle counterclockwise). The least movement occurs with click, where the hand itself shouldn't move, just the fingers. In contrast, hand tracking identifiies particular hand positions (also called gestures), which at most include minor motion such as a one direction single wave.
A comparison of the Hand Tracking vs Cursor modules can be seen in this chart:
|FEATURE||HAND TRACK||CURSOR MODE||FEATURE||HAND TRACK||CURSOR MODE|
|Supported Cameras||F200, SR300||SR300 only||# of Hands||2||2|
|Track||22 joints||1 point of interest||Gestures||12||increasing|
|Latency||noticeable||0 @ 60 FPS||Segmentation||Yes||No|
|Calibration||Needed||not needed||Joint Rotation||Yes||No|
|Motion Speed||<.75-1.5 m/s||up to 2 m/s||Main Use||full hand map||UI control|
|~ 110 cm|
|OS Support||Win 10|
NOTE: In the 2016 R1 release (v8), you cannot use Cursor mode while using Full Hand tracking mode. This will be fixed in future versions.
Like hand tracking, cursor mode is most effective when the user gets visual feedback on screen, especially for hands being in or out of Field of View (FOV).
Another advantage of cursor mode is that hands don't have to occlude the screen if Adaptive Point mode is implemented. All activity can be set to occur in a bounded box. The advantages of Adaptive Point mode include greater reaction for less motion making it more responsive in some usages. But in doing so, it has less granularity (fine detail) than Cursor Point mode. Cursor Point mode allows more detail and is better for drawing. .
Engagement Mode is the way to control when the user's hand movement is picked up as an interaction with the camera as opposed to the user just using his hands while speaking. Configure when to pay attention to the hand motion using EnableEngagement with the option to use SetEngagementTime to show a progress bar (0-100%)
While improving in each SDK release, the module is only at beta level in the 2016 R1 release and still undergoing some changes. Limitations that currently exist include :
> A minimum palm width of 5.5cm (commonly about age 5 and up)
> Frame rate of 50-60 FPS (lower than 50 will have blank areas)
> Range of 20-110cm from the camera
> 2016R1 detection distance is only 70cm with a max range of ~110cm
> User cannot be holding any object in their hand(s)
The Intel® RealSense™ Developer Kit featuring SR300 is now available from http://click.intel.com/realsense.html