Utility for changing laser camera parameters (IVCAM v0.5)

Utility for changing laser camera parameters (IVCAM v0.5)

IVCAM v0.5 is a free utility that I made that allows you to do the following with your Intel RealSense F200 camera:

- Read and write camera and laser projector settings.
- Display depth, XYZ (color coded), or IR streams in real time while you are changing those settings.

Requirements are either:

a) Laptop with a built-in RealSense F200 camera

or

b) Computer with the following:
- 32 or 64 bit Windows 8.1 or Windows 10.
- DCM installed (to use the USB version of the F200 camera).

This is how it currently looks:

You can download it from here:
http://www.samontab.com/web/files/ivcam.zip

samontab.com
49 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.

Thank you Sebastian!  Great add to the community!

(In a few days I'll move it up to a post note to stay near the top).

The RealSense camera has a laser in it?  I thought the depth sense data stream was infrared using the time-of-flight algorithm?

Robert

OK, here is how this camera works:

The Intel RealSense camera has an IR laser projector that emits IR light into the scene. There is an IR camera that reads the emitted pattern. Based on the displacement of the pattern due to objetcs in the scene, it can calculate the distance of the objects from the camera. This method of calculating depth in general is know as structured light, and this is the way other 3D cameras, like the kinect, work.

In the following image you can see the IR light being emitted by the camera into a wall that was a few cms away. (I expected a pincushion distortion in both axes, but it seems that only the vertical axis is distorted).

Now, about the IVCAM settings:

The LaserPower argument basically allows controlling the intensity of this laser projection, from powered off at 0 up to 16 at maximum power. Usually you will be using this setting at 16.

MotionRangeTradeOff: The difference here is that the pattern used in this camera is not a fixed set of points as is the case with the kinect, but instead is a moving pattern, from top to bottom. It seems that the rate at how the patttern is projected is controlled by this setting. The longer the number, the longer the vertical pattern gets stretched, so less patterns per second are being emitted and detected. This allows to gather more light, and therefore to detect objects farther away, but when they move, you don't detect the pattern correctly so no depth info for fast moving objects far away. The reverse is true for smaller numbers. You can think of this parameter as exposure time in normal camera terms.

Here is an image of the pattern. Because the image was exposed for too long (normal camera rates, like 30fps for example), the pattern looks blurred in the vertical direction, but some information can be seen in the horizontal axis.:

If you decrease the exposure time, you can actually see the pattern that is being projected into the wall:

Note that the pattern is projected so that it is seen straight, even though the illuminated area  is not.

Accuracy: This parameter changes between 3 types of projected patterns, from coarse up to fine grained.

FilterOption: I think this parameter is only for the post processing and creation of the depth map. Given the initial points, with this option you select the method to interpolate the rest of the frame. Here there are a few different option. Look at the SDK manual for more details on this.

I hope these parameters make a bit more sense now.

samontab.com

Samontab,

My understanding is that the Real Sense camera uses time-of-flight not structured light.  Here is a description of the camera www.eetasia.com posted on January 9th:

" It comprises a conventional HD RGB sensor, a laser that emits 16 different modulated signals and an IR sensor to detect the time-of-flight of the transmitted laser signals and compute distance. "

This article also mentions that ASUS and Lenovo are marketing laptops with the camera in the Netherlands.

However,  a user comment to an article in PC Perspective (www.pcper.com) in October 2014, stated that Real Sense uses structured light depth sensing as did Kinect v1, whereas the first version used time-of-flight.  Perhaps this is the source of confusion.

By the way, Kinect v2 now uses time-of-flight.

Maybe someone from Intel can clear up the debate.

Regards,

Jack

Thank you Sebastian, this has helped me a lot.

Regards,

Sam

The previous "Perceptual" camera was a rebranded SoftKinetic DepthSense device, which was Time of Flight camera.
I'd be really surprised if they would have totally changed the device between the beta and the gold release. The resolutions, range, etc. are mostly the same, so I'm 90% sure it's still Time of Flight.
 

samontab: thanks for this brilliant tool, can you extend it with SetDepthConfidenceThreshold? It's strangely missing.

 

Looking at Samontab's excellent pictures of patterned (structured) light would seem to answer the question.  
The Perceptual camera was totally different than the RealSense F200. (And similarly future Realsense cameras may use different tecnology than F200). 

@samontab and others.  Thank you for the information!

Robert

Very useful tool and topic.

Thanks!

Hi guys,

I just updated this tool:

IVCAM v0.2
- Updated camera frame access for better performance.
- Added DepthConfidenceThreshold to the controls.
- Added individual buttons to set a feature to its default value.
- Added tool tips explaining the feature values.
- Added IR Camera information details.

Here is a screenshot of it, showing a longer range capability by tweaking the parameters:

This version should work better in general than the previous one. So if you had any issues before, give it a try again.

You can still get it at the same place:

http://www.samontab.com/web/files/ivcam.zip

samontab.com

The longer range is definitely something I'm interested in, thanks!

Regards,

Sam

Nice post. Thank you all!

Another update to this tool:

IVCAM v0.3
- Added XYZ camera stream (Lab color space coded image based on XYZ position of each pixel, X and Y for color, Z for lightness)
- Updated depth stream view to display the entire depth range correctly (You should now see more gray scale tones, not mostly just black and white as it was before)
- Minor bug fixes

You can still get it at the same place:

http://www.samontab.com/web/files/ivcam.zip

samontab.com

I wrote similar software for Linux, and recorded a youtube demo of adjusting the controls.

Nice to hear that Daniel!

I'm also developing a Linux version of the SDK for the Intel RealSense camera, as I usually prefer to use Linux. It looks promising because as you could see, the OS detects the camera without an issue. Actually the app that I published here is cross platform, so once I finish the Linux access to the camera, this app will be able to run in Linux as well.

Regarding your depth pixel format mistery, the camera actually has 6 depth formats, as seen in the documentation:

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/d...

Here are the 6 depth formats explained:

 

PIXEL_FORMAT_Y8

The 8-bit gray format. Also used for the 8-bit IR data. See fourcc.org for the description and memory layout.

 

PIXEL_FORMAT_Y8_IR_RELATIVE

The IVCAM specific 8-bit IR data containing the I1-I0 IR image. I1 is the IR image with the coded pattern on and I0 is the IR image with the coded pattern off.

 

PIXEL_FORMAT_Y16

The 16-bit gray format. Also used for the 16-bit IR data. See fourcc.org for the description and memory layout.

 

PIXEL_FORMAT_DEPTH

The depth map data in 16-bit unsigned integer. The values indicate the distance from an object to the camera's XY plane or the Cartesian depth.The value precision is in millimeters.

A special depth value is used to indicate low-confidence pixels. The application can get the value via the QueryDepthLowConfidenceValue function.

 

PIXEL_FORMAT_DEPTH_RAW

The depth map data in 16-bit unsigned integer. The value precision is device specific. The application can get the device precision via the QueryDepthUnit function;

 

PIXEL_FORMAT_DEPTH_F32

The depth map data in 32-bit floating point. The value precision is in millimeters.

A special depth value is used to indicate low-confidence pixels. The application can get the value via the QueryDepthLowConfidenceValue function.

I guess the mistery would be what does the YUV stream looks like, as it seems that it does not belong there.

Best of luck, and keep up the good work!

PS: I wish Intel could have some of those awesome hacking events here in Sydney, Australia...

samontab.com

Hi Daniel, great work!, thanks for sharing it here.

I am also working on a Linux version of the SDK for the camera, as I prefer to use Linux. It seems promising because, as you saw, the camera seems to be working fine out of the box. Once I'm finished with that, this app would be cross platform.

Here is some help with your format mistery. The camera actually has 6 pixel depth formats documented:

PIXEL_FORMAT_Y8
The 8-bit gray format. Also used for the 8-bit IR data. See fourcc.org for the description and memory layout.

 

PIXEL_FORMAT_Y8_IR_RELATIVE
The IVCAM specific 8-bit IR data containing the I1-I0 IR image. I1 is the IR image with the coded pattern on and I0 is the IR image with the coded pattern off.

 

PIXEL_FORMAT_Y16
The 16-bit gray format. Also used for the 16-bit IR data. See fourcc.org for the description and memory layout.

 

PIXEL_FORMAT_DEPTH
The depth map data in 16-bit unsigned integer. The values indicate the distance from an object to the camera's XY plane or the Cartesian depth.The value precision is in millimeters.

A special depth value is used to indicate low-confidence pixels. The application can get the value via the QueryDepthLowConfidenceValue function.

 

PIXEL_FORMAT_DEPTH_RAW
The depth map data in 16-bit unsigned integer. The value precision is device specific. The application can get the device precision via the QueryDepthUnit function;

 

PIXEL_FORMAT_DEPTH_F32
The depth map data in 32-bit floating point. The value precision is in millimeters.

A special depth value is used to indicate low-confidence pixels. The application can get the value via the QueryDepthLowConfidenceValue function.

 

The only mistery that I see here is the YUV pixel format. It doesn't really make much sense to be there. I wonder what that image looks like...

 

I hope it helps!

 

PS: I wish Intel would have those awesome hacking events here in Sydney as well!

samontab.com

I saw the pixel formats in the both the documentation, and in the USB descriptors, but when I select anything I get 16 bit per pixel data. I looked for interleaved frames, and multiplanar formats, and didn't find anything there either. The YUV format shows up as effectively 16 bit little endian depth values too. I'm going to try to figure out how to get the IR stream tomorrow.

mmm, that sounds like maybe the stream selection is not working. I am currently requesting the pixels with PIXEL_FORMAT_Y8, and using only 8 bits per pixel to create the depth image, and it is working fine. This means that I am not getting the 16 bit per pixel data stream that you are receiving regardless of the stream you use.

samontab.com

New SDK breaks this utility

Yes Vlad, this was expected.

I was reading the details in the Release Notes of the updated library and there are a few changes to the API, so probably one of the methods I'm using is not supported in this release.

Thanks for testing it!.

When I upgrade the library, I will port the application to the new API. For the moment, just the R1 version is supported.

samontab.com

Hi guys,

I just updated this utility with the latest version of the RealSense SDK (R2).

IVCAM v0.4
- Compiled using latest RealSense SDK (R2).

This is a screenshot of it:

samontab.com

Noise handling to be improved still

Updated the application to use the latest Intel RealSense SDK R3 (v5).

samontab.com

Hi friends any new discussion?

is there any advances under this topic?

Hi PONRAM,

Do you have any questions on the topic?

samontab.com

Hi guys,

I just updated this utility with the latest version of the RealSense SDK.

IVCAM v0.5
- Compiled using latest RealSense SDK, which at the moment is R4 (v6.0).
- Included RealSense SDK redistributable in the installer.
- Added current camera firmware version in the display.

This is a screenshot of it running in Windows 10:

samontab.com

Hi Sebastian, I tried to install this version in Windows 8.1 with R200 camera and it is not working, It says that IVcam.exe stopped working.

 

Thanks

Hi Gerardo,

I haven't tested this application with a R200 since I don't have one (it was developed for the F200), but in theory it should still work.

Do you have the DCM for the R200 installed correctly?

That would be the most likely cause of the problem.

If you have the DCM correctly, and you can run any other examples from the RealSense SDK, then maybe I would need to adapt the code a bit to make it work with the R200 as well.

samontab.com

Looking through the documentation it looks like the IVCam device functions are for the F200 only.

You are right Sam M.

IVCam parameters, which are the ones exposed in this app, are only for F200:

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/d...

R200 uses a different type of projector it seems, as it operates with a different set of parameters. Interesting. I just assumed that the R200 used the same emitter, but it seems that it allows changing the projected pattern in a more detailed manner. Probably they are using a totally different pattern completely. Here are the DS parameters:

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/d...

Nice!, I would really like to get my hands around an R200 to support it as well, it seems much more advanced than the F200.

samontab.com

You are right Sam M.

IVCam parameters, which are the ones exposed in this application, are only available for F200:

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/d...

R200 uses a different type of IR emitter, but it can also be controlled. It just has a different interface, DS parameters:

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/d...

If I get my hands around an R200 I will add support for the R200 in this application.

samontab.com

You are right Sam M.

IVCam parameters, which are the ones exposed in this application, are only available in the F200.

R200 uses a different IR emitter, with a different set of parametes (DS), and also what is being projected is different.

So, they are totally different things.

On the F200 you can control timings of the projection, whereas in the R200 you control position of the projection.

if I get my hands around an R200, I will add support for it in this application.

samontab.com

Can the noise be reduced by increasing laser power? 

Usually depth information contains "holes" in it, so you need to post process it to work with a smooth continuous data stream.

Increasing the laser power is basically putting more light into the scene, and this may help in some cases, but if the object is too close for example, you end up with an over exposed depth image, similar to a normal camera, so that's not good as well.

Getting the right amount of laser power is an important step. You could monitor the depth values constantly and change the laser power accordingly to maximize it's accuracy, but the noise that you mention will still be there because of problems in the detection for example, difficult materials, edges, etc...

samontab.com

I tried with the SR300 on Windows 10 64bit but it seems that doesn't work with this.
After installing the utility, it fails to start.

So far I have noticed that the SDK compatibility with the F200 works only with samples from Intel, while other programs cannot even recognize the last SDK!

Scigor, you might want to try setting it for win8 compatibility and run as admin if you have not already.

Quote:

Douglas K. wrote:

Scigor, you might want to try setting it for win8 compatibility and run as admin if you have not already.


That wouldn't make any difference.

The problem is that the new camera has a slightly different API, so the tool will not work with it as is.
It only works for native F200.

samontab.com

samontab,

not sure how to say this other than to be blunt.  you are wrong. IVCAM works well on my sr300. i have it set to win8 compatibility, run as administrator. otherwise it would not run properly.  build 10586.104 windows 10.  i have a laptop dedicated to realsense cameras so if need be i can do a fresh install of any build i can determine i need. i happen to be running a fresh build for the sr300 which arrived last friday. [1/26] ivcam

Thank you guys,

samotab is right on the API, in fact other old applications still don't recognize the SR300.

But I followed the advice from Douglas K., set it to win8 compatibility and at last IVCAM now works.

Quote:

Scigor wrote:

Thank you guys,

samotab is right on the API, in fact other old applications still don't recognize the SR300.

But I followed the advice from Douglas K., set it to win8 compatibility and at last IVCAM now works.

This is interesting. Thanks for sharing guys.

Maybe it loads an earlier version of the SDK when using win8 compatibility?

I don't have an SR300 at hand to test it, but good to see that there is a workaround for it.

samontab.com

Quote:

samontab wrote:

Maybe it loads an earlier version of the SDK when using win8 compatibility?

Could be a possibility.
In fact I didn't have an F200 but before ordering the SR300 I installed the SDK R5.

And when I got the SR300 I have updated to the latest R6 version.
 

Hm,
this looks all a bit strange, since I can use IVCAM v0.5 together with the SR300 without starting it in compatibility mode. It works as well for me as it did with the F200. It does not crash (as long other programs are not using the SDK at the same time) and the changes to the parameters get accepted and are immediately noticeable.

Some days ago I deinstalled all runtimes and all sdk installations and installed the latest sdk and runtime (8.0.24.6528).

Thank You Sebastian ! 

Hey samontab,

thanks for sharing this great tool!

However will there be an update with the latest Realsense SDK which is atm R6 (v8.0).

Cheers peter

Very Useful tool and topic ! 
Thanks 

Regards,
Amir

Hey samontab,

anything similar to IVCAM v0.3 for librealsense R200 in Linux?

 

Thanks!

Hello

 

This Information is very useful.

Also I just started with the SR300 cameras and had a small issue.

  • Even though I am using it indoors even at the slight onset of sunlight through the Windows the SR300 does not detect depth Images beyond 30 cms and is completely blacked out? I was looking to improve this.
  • Also is there a indetail description for using libRealSense as I am trying to run multiple SR300 cameras or onef200 and one SR300 camera on one System along with Windows 10 and unity.So please could you give me some suggestions on how I could proceed on this?
  • I am attempting Hand tracking and would prefer outdoor tracking but the SR300 and f200 work only indoor.Is there some way I could use the R200 for Hand tracking or could in someway manage the SR300 and F200 outdoors?
  • Also a very Basic question but what is the best distinctive Point which makes the R200 work outdoor as a Long range camera and SR300 and F200 short range cameras not work outdoor or in a very bright room?

Thank you for all the Support.

Help would be highly appreciated as I am a Student who has just entered this Domain.

Cheers

Leave a Comment

Please sign in to add a comment. Not a member? Join today