Archived - What's New in the Intel® RealSense™ SDK R3 (v.5.0)

The Intel® RealSense™ SDK has been discontinued. No ongoing support or updates will be available.

 The third release (R3) of the gold Intel® RealSenseTM SDK, version 5.0.3.187777, is now available for the Intel RealSense Camera F200. This release is also the Beta release for the Intel RealSense world facing camera R200. Please note that the camera driver/service (DCM) must be installed as well as the SDK. Please use DCM 1.3.20 for the F200 camera (or 1.2) and DCM 2.0.3 for the R200 camera. They are available on the same download page as the SDK.

As with previous releases, the system must have Windows* 8.1 64 bit (August update) or later with a 4th generation Intel processor (or later), an available root USB3 port, 8 GB RAM, and 6 GB free space.

Installation requires that you REMOVE any previous versions of the RealSense SDK and REBOOT. Then install the appropriate DCM and the 5.0.3 SDK and REBOOT.  Note: The installer won't start when there is not enough free disk space. It may also pause for a long period when selecting language packs (to get from the web) or changing camera types.
Note: 
 For building an installer for your app, please read  “Deploying Your Application” in the SDKDevGuide.pdf.

The R200 shares Blob and Face Tracking with the F200,  but only the R200 supports Scene Perception, Enhanced Photography, and 3D capture of Head and Body, Also, unlike the F200, the R200's graphics driver must support OpenCL* 1.2. 
A quick way to check which modes are supported by which cameras, check the Samples which are divided into front, rear, and both.

Improvements to F200 features since the R2 release:
- Face detection and face recognition are significantly improved with better 2D and 3D landmark accuracy
- Hand tracking is improved for back of hand (fixed skeleton), side of hand tracking, and fingers that point toward the camera. Two hand foldedness values are fixed and the 'gesture end' status is triggered when the hand leaves the FOV;  
- Object tracking now includes an extensible learning mode, map creation interfaces including instant SLAM map creation, edge based 3D object tracking and
RGB+Depth marker-less tracking for 2D objects. Additionally the toolbox includes camera calibration, object model creation and 3D configuration files for feature or edge based tracking.
- Speech support is available for US and British English, Latin American Spanish and Brazilian Portugese, German, French, Japanese, and Mandarin. (Each module is available as a separate installer from the website.)
- Unity Toolkit: More scripts and prefabs with samples are available..  (See also Tips and Tricks to use Unity with the SDK)
- Utilities: The Euler angle conversion in a non-default Euler order is fixed.
- BGS: More usage cues (too close, too far, and entry events _support optional callback. Improved hair and finger/hand segmentation. Lessened power consumption. (see also Background Segmentation Tips & Tricks)
- Touchless Controller has a new viewer. 
- 3D capture (only at Alpha) is still face only with F200,) See R200 for other usages. Now supports OBJ, PLY, and  STL output. Added optional hole filling (solidification) or texture generation (helps color).  See also Intel RealSense 3D Scanning Tips and Trick
 - Emotient* Emotion Detection is still only a Preview feature. (Anger, contempt, disgust, fear, joy, sadness, surprise AND positive, negative, and neutral sentiments). Consider using landmarks as alternative emotion indicators.
- Java* is still Preview only. 
-  The JavaScript web solution has been removed. It will be redesigned to work with browsers with increased security. The JS package is currently available separately - see this blog: How to get JavaScript up and running for R3        
- The Processing* sample must have the sketch folder location set in Preferences to  $(RSSDK_DIR)/framework/Processing. 
- Unity*: Default libraries are 32bit (Unity 4). Unity 5 requires the 64 bit libraries be added to the plugin directory. Add the .signature libraries to use the free versions of Unity. 

R200 major features (see also Introducing the Intel RealSense R200 world facing camera

Face detection for the R200 has been improved and 2D/3D landmarks enabled. The R200 can also now  do Face and Blob tracking.
3D capture (stationary) and reconstruct object, face, head, and body with standard OBJ, PLY, and STL output formats Optional solidification (hole filling) or texture generation  
Enhanced Photography 
added Segmentation and object tracking functions. Has depth resizing, depth hole filling, two layer segmentation, object insertion, sticker pasting, image refocus based on depth, euclidean disance measurement, object segmentation and layer tracking.
Scene Perception does real time tracking and dense reconstruction as a volume and as a mesh with export to an OBJ file.. Beta added the users ability to set a region of interest and export surface voxels and mesh data and their color. Relocalization and tracking recovery have also been added. Also camera pose can be set from an external tracking source and user can query the internal camera intrinsics corresponding to the returned voxel and normal images. Note: The meshing region of interest is changed from a sphere to a box. 

This release adds a Unity interface and samples.
For supported color/depth configurations see the SDK manual

Known Issues (both F200 and R200)

  • Expect high CPU use / low FPS when running multiple applications simultaneously.
  • Face_tracking cannot use 2D mode.
  • When recording to a file, the first 2 frames may be recorded in the wrong order Simply remove these 2 frames with the ClipEditor tool.
  • Mirror mode is NOT supported for Face or Pose Detection nor  Landmark Tracking.
  • Only set camera configurations to listed profiles. Using multiple apps with different profiles may cause unexpected results.
  • Don't call an UnsubscribeAlert during the callback OnFiredAlert or it will cause a crash
  • BGS issues - see https://software.intel.com/en-us/articles/realsense-bgs but note 3D Background Segmentation should not be used with any other module.
  • CAD edge based 3D tracking applications must ignore the initial pose coordinate system if it is not desired. (The initial pose coordinate system is always on and considered as a detected object until the target object in the scene is detected. After that point the initial rendered coordinate system is replaced by the new detected coordinate system.) 
  • Unity Toolkit : Need to Edit each rule separately as Editing Rules on multiple objects is currently not working. 
  • PointCloud mesh is 9th of the maximum depth resolution due to Unity’s mesh limitation N/A.
  • Working on AR mode (with "Sense AR" prefab) requires the game aspect ratio be set manually to 4:3.

F200 only Known Issues

  • Face recognition may cause high CPU utilization. Be sure to disable any mode when not required.
  • Continuous tracking option causes left/right hand confusion sometimes Need to be used with care.
  • The Hand module cannot by instantiated twice in a single app. Be sure to destroy each instance before creating another.
  • Nuance Speech Recognition and Synthesis does not include redistributable components, due to size. Get from website.  
  • Touchless controller intentionally ignores the pinch gesture in the lower part of the FOV. Only use in top section.


R200 only Known Issues

  • Microsoft Visual Studio 2013 runtimes cannot be installed when the R200 camera is selected,  so install separately
  • Depth loading is not yet available 
  • Face Tracking: 50cm to 3m from the camera. Camera rotating up to 10 degree per second up to  1 meter per second. Fast movement invokes faulty mesh and inaccurate pose.
  • 60 fps and YUY2 are not supported 
  • SP may return random error during large mesh save (due to win32 memory limitation) 
  • Tracking is inaccurate or fails when scanning monotonous areas, reflective surfaces, flat walls, large black areas or scenes with very few visible details/geometry structure. 
  • Pose might drift or the reconstruction might sink when the camera stays in a static position and orientation for a long period of time. 
  • VoxelResolution::HIGH (voxel)_RESOLUTION reduces tracking robustness. Only use it with SLOW camera movement in a small area (1meter3)
  • Enhanced Photography will measure inaccurately when point is on the edge or in areas with inaccurate depth data. Choose points that are 2-3 pixel away from the object edge and that have good depth data. 
  • Paste 'on plane' only works on a planar area with depth data.
  • EP depth and color resolutions must use the same aspect ratio (Google* depth 2.0 standard). Use color resolution = 640x480 and depth resolution = 480x360, NOT 628x468.  Color pop and depth blend will only work on a point that has depth data. 
  • EnhanceDepth real time hole filling  may result in poor depth map. Using HIGH option has no affect. 
  • Layer tracking runtime is not optimal so run capture and tracking on separate threads. 
  • The DepthBlend sample will not work (may crash) when loading from Google Depth 2.0 file. Also  only use preview and snapshot modes. 

 Download the SDK

For more complete information about compiler optimizations, see our Optimization Notice.