Archived - Introducing the Intel® RealSense™ R200 Camera (world facing)

The Intel® RealSense™ SDK has been discontinued. No ongoing support or updates will be available.


The latest Intel® RealSense™ camera to be released is the R200 - world facing camera. The Windows* 10 64-bit (and Windows 8.1 64 bit) SDK is already released, at beta, in the R3 (v5.0.3) Intel® RealSense™ SDK . An Android v5 SDK is likely to be released later this year. 

At approximately 102x9.5x3.8mm, the R200 will be integrated into select tablets and 2 in 1 systems, which should be available for the 2015 holiday season. The peripheral USB3 pluggable version of the R200 (130x20x7mm) and developers can be notified of availability to order at http://click.intel.com.

The R200 actually has 3 cameras providing RGB (color) and stereoscopic IR to produce depth. With the help of a laser projector, the camera does 3D scanning for scene perception and enhanced photography. The inside range is approximately 0.5-3.5 meters and an outside range up to 10 meters. Note: Range is very dependent upon the module and the lighting. For operating ranges of the various algorithms for both F200 and R200 cameras, see the visit the online documentation.

Unlike the Intel® RealSense™ F200 camera, the R200 is focused on the world, not the user. As such the RealSense SDK focuses on the following use cases for the R200.

  1. Capture the World in 3D and then edit, share, and print the objects in 3D.
  2. Enhance your Photography. The Intel RealSense R200 includes 3D filters which allow re-lighting, re-focusing, and background segmentation (removal/substitution of the background).
  3. Add virtual content into 3D captures of the physical world. In a feature called Scene Perception, you can add virtual objects into the captured real world scene since the R200 camera understands surfaces as well as objects and motion and can create scenes from estimations of the camera's position/orientation in the scene.

These use cases are based on two functional areas of the R200 cameras.

  • Tracking/Localization : Real-time estimation of the Camera’s position and orientation (pose) using depth, RGB and IMU data.
  • 3D Volume/Surface Reconstruction : Building real-time digital representation of the 3D Scene observed by the camera

Some of the features found in the Intel RealSense SDK Windows for the F200 will be introduced over time, remember the R3 SDK for the F200 is still just the beta SDK for the R200. The alpha R200 SDK (R2 of the F200 SDK), provide face detection and pose. Face detection works at up to 2.5m, pose works at up to 1.5m. Beta R3 adds blob tracking and face landmarks (1 meter). Android features will have a separate SDK and are not yet published.

While the RealSense SDK for Windows is shared by the F200 and R200 cameras and contains documentation and samples for both, the R200 requires a different (separate) DCM and firmware package. The R200 uses the DCM 2.x series while the F200 uses DCM 1.x.

One of the most exciting things about the R200 camera is the larger scanning capability and it's new method for measuring depth. The R200 includes stereo cameras as well as the RGB camera. And since the camera is less dependent on IR, it can be used outdoors. The color camera provides the images for humans and the 2 depth cameras provide the data for algorithmic consumption. Additionally if the system has an IMU (inertia measurement unit) built into the system, the SDK can adjust for gravity's effect on objects added to a scene.

With stereoscopy photography, depth/3D is computed from the disparity (pixel shift) between the 2 'separated' cameras using triangulation to output a depth measurement. Note that this is based from the parallel plane (not the absolute range from the camera). There is a utility included in the SDK to help with plane detection. Making sure the camera sees a horizon during init can be critical to the orientation of the scanning target.

The R200 tracks the camera movement in 3D space with 6 DOF (degrees of freedom). 3 Degrees from the Front/Back/Up/Down/Left/Right and 3 degrees from Yaw/Pitch/Roll movements.

To get the best 3D Scanning results:

  • Use a 2m (cubed) FOV, with at least 5000 pixels (640x480). For proper detection, use this chart:
    Distance / min.   Rectangular object size
     30cm                    4.5cm x 3.5cm,
    100cm                 16 cm x 11 cm
    180cm                 28 cm x 21 cm
  • Don't occlude more than 20% of an object.
  • Move the camera but keep the real scene object as stationery as possible
  • Run at 30 FPS or 60 FPS (see chart below). Use higher FPS to get smaller inter-frame displacement
  • Don't use plain, non structured surfaces. The IR projector sends random, non-uniform light pattern to add texture to the scene and runs the data thru the filters in the IR band. Additionally RGB input is added to the stereo depth calculations.
  • Move the camera at a moderate - slow speed, remember that filming at 60 FPS means 18M depth calculations per second.
  • Allow the time for the camera to init (shown on the bottom left of the screen) which includes centering the target within the green lines.

The color cam is capable of doing 32 bit RGBA at 1080p @60FPS using fixed focus and a 16:3 aspect ratio. The RGB cam has a slightly larger FOV than the dual cams but is not meant to be used as a standalone camera.
The dual depth cams use a fixed focus 4:3 aspect ratio with a 70x59x46 degree field of view.
The IR is a class 1 laser in the 850nm range,

Available Resolutions
@60FPS, depth at 320x240, color can be 640x480
@60FPS, depth at 480x360,  color can be 320x240 or 640x480
@30FPS, depth at 320x240, color can be 640x480 or 1920x1080
@30FPS, depth at 480x360, color can be 320x240, 640x480, or 1920x1080

The RealSense SDK provides interfaces to project depth to color and vice versa.

The power draw of the R200 ranges from 0 to 100mw (idle) to 1.0-1.6 watt when active (depending upon the modules used). It has several power saving features including the use of USB3 data bursts. 

Unity support for the R200 is included in the SDK (Note Unity 4.x is 32 bit and Unity 5.x is 64 bit, so add the appropriate libraries to your project. Also Unity personal (non Pro) can be used as long as you include the signed libraries in the project).

About the Author

Colleen Culbertson is an AE with Intel Developer Relations Division. 

Learn more about Intel® RealSense Technology:
www.intel.com/software/realsense

 

For more complete information about compiler optimizations, see our Optimization Notice.

6 comments

Top
carlos r.'s picture

Excuse Me Colleen,

In the article you've spoken about minimun distance of 180 cm to a capture área of 28x21 cm

Will the camera track Slim objects? (i.e. 28 x 8 cm) is there any thumb rule for the aceptable dimensions?

regards

 

heather m.'s picture

Hi

How does one get a camera to test out in real life scenario. 

Dongseong Hwang (Intel)'s picture

Hi, the article says "R200 has IMU sensor". Is it true? If not, please clarify it.

Elizabeth K.'s picture

Do resolutions & frame rate limits (@60FPS depth at 480x360, color can be 320x240 or 640x480; 
@30FPS depth at 480x360, color can be 320x240, 640x480, 1280x720, or 1920x1080) reflect a hardware limitation? Or might it be possible with driver / firmware updates that in the future we can get higher resolution at 60 fps on the color cameras?

Also, is the 16:3 aspect ratio you described, for the color camera's max bit rate (32-bit RGBA at 1080p at 60fps) a typo? That's a really weird aspect ratio. Did you mean 16:9? Is it possible to access the color signal at this frame rate and resolution?

Darren W.'s picture

Hi Collean,

The images you show at the top of the article could be a little misleading as the "IR" image you show is clearly a grey-scale copy of the RGB image. To explain, first there is no parallax between the RGB image and the grey-scale image you claim to be the IR image that there should be if the sensors capturing the images are physically separate. Second, near infra-red (NIR) radiation is more Lambertian in its reflection from surfaces, especially skin, so the lighting would be more diffuse in a NIR image compared to the same image taken in the visible spectrum. Thirdly, it is likely the shirt would show up in a different contrast as compared to the image taken in the visible spectrum as the dye is designed to reflect blue light but is likely to behave differently when reflecting NIR. Same goes for the coffee mug and the shirt buttons. Finally, from the following sentence:  "The IR projector sends random, non-uniform light pattern to add texture to the scene ... " I assume the R200 projects a random dot pattern in the NIR. That projected pattern is missing from the claimed NIR image.

Additionally, the depth image shown does not correspond with the other images, its close but no cigar.

I'm guessing that your subject was posing in front of the R200 to get the depth image while somebody else took his picture with a conventional camera (probably a smart phone) slightly displaced from the location of the R200. From this I assume extracting the RGB and NIR images from the cameras is either not possible or difficult to do, and for the purposes of this article it was just easier to employ a little subterfuge, which is fair enough.

Anyway, my pedantry and misleading images aside, the depth image looks great having picked a good deal of facial detail at a range of just under a metre (based on the vertical field-of-view). How much post-processing (hole filling, smoothing, etc.) has gone into produce the depth image shown?  

Did the subject have to be relatively still as you captured the depth image in order to get the facial detailsIf so would you get the same accuracy and precision if the person were moving about? 

In the article you write:

"The R200 includes stereo cameras as well as the RGB camera. And since the camera is less dependent on IR, it can be used outdoors."

This suggests that the F200 uses either time-of-flight or pattern distortion to measure depth, i.e. its essential the IR sensor can detect the projected pattern. However the projected pattern in the R200 is just an aid to depth sensing and is thus not essential. By how much does not having the NIR pattern projected on the scene affect the depth sensing performance of the R200? (This probably depends on how much texture objects in the scene have naturally.)

Are the left and right NIR images taken simultaneously or sequentially? 

Are the left and right NIR sensors of a larger pixel resolution than the output suggests? In other words,  do you take the left and right images at high resolution then scale them down to perform the depth computations in a timely manner and be able to stream the data at 30/60 fps? 

How do you go about calibrating the stereo cameras so that they are pixel-row aligned and how do you maintain that calibration as the camera is moved around an object? Does the camera have to remain level to perform optimally?

I guess if I want answers to these questions I should really buy one and try it out :)

 

 

 

Chavo M.'s picture

Hi Colleen,

Thanks a lot for the detailed introduction.I have used the F200 extensively and now moving to the R200.

I recently bought two R200 cameras for my company for the purpose of taking measurements and I have really searched online for a tutorial on how to create an application for the R200 to achieve this. I have not been able to find one that is even a bit decent and clear on this.I ran the RF_MeasurementSP demo in the bin folder of the RSSDK but it doesn't seem to work as expected.I also had a look at the code and it is not very elaborate.

Could you kindly please do a tutorial on  taking measurements using the R200-preferrably on C# :). I would really appreciate this and be grateful as we have a project that relies heavily on the R200.

Thanks Colleen.

seth p.'s picture

Thanks Colleen.  I was wondering who the bus deb contact would be for real sense?  I have a friend at TechCrunch interested in reaching out

Add a Comment

Have a technical question? Visit our forums. Have site or software product issues? Contact support.