The Intel® RealSense™ SDK has been discontinued. No ongoing support or updates will be available.
This code sample demonstrates the use of the Intel® RealSense™ SDK for Windows* in a C#/WPF desktop application. This simple app, named BlockHead, utilizes three interesting features of the Intel RealSense SDK:
- Captures and displays the color stream from the RGB camera
- Retrieves face location and head pose estimation data
- Retrieves and evaluates facial expression data
(Note: The full functionality of this sample app requires a front-facing 3D camera.)
Introduction to Blockhead
As shown in Figure 1, the app displays the color stream in a WPF Image control and superimposes a cartoon image over the user’s face in real time.
Figure 1.Cartoon Image Superimposed on User’s Face
The cartoon image is programmatically manipulated in real time, based on data acquired from the SDK:
- Scales in size to match the user’s face (i.e., becomes smaller or larger as the user moves away from or toward the camera) based on face rectangle information
- Rotates left and right in response to the user’s head orientation (roll)
- Swaps image control content based on expression data acquisition and scoring (see Figure 2)
Figure 2.Smile, Tongue-Out, Kissing and Open-Mouth Expressions Detected in Real Time
For this simple demo app, the graphics were created in a drawing program and captured as portable network graphics (.png) files. These images could easily be substituted with artistically rendered transparencies, or even screen captures of friends, caricatures, etc. for a more compelling visual effect.
Different transforms (e.g., ScaleTransform, RotateTransform) are applied to the image object to position it in response to head tracking inputs from the Intel RealSense SDK. These inputs include face location, pose estimation, and expression recognition data.
The SDK can capture around 20 different expressions that can then be evaluated in an application. This particular app focuses on the mouth-oriented expressions: EXPRESSION_KISS, EXPRESSION_MOUTH_OPEN, EXPRESSION_SMILE, and EXPRESSION_TONGUE_OUT. However, it could easily be extended to use the head, eye, and eyebrow expression information as well.
Check It Out
To learn more about this app, pick through the code, and extend it to more interesting use cases leveraging the Intel RealSense SDK, please download it here.
To get started and learn more about the Intel RealSense SDK for Windows, go to https://software.intel.com/en-us/realsense/intel-realsense-sdk-for-windows.
About the Author
Bryan Brown is a software applications engineer in the Developer Relations Division at Intel. His professional experience includes a mix of software, electronic, and systems design engineering. His technical interests focus on applications of natural interaction and brain-computer interface technologies, with active participation in several alpha developer programs for various emerging technologies in these areas.