Raising gesture recognition resolution

Raising gesture recognition resolution

Hi,

Although help mentions that "SDK gesture recognition module takes RGB, depth, or IR streams as input", it seems to me that it really takes into account just depth stream with low 320x240 resolution.

We've tried:

    EnableGesture();
    EnableImage(PXCImage::COLOR_FORMAT_RGB24,1280,720);
    
But it seems this has only to do with rendering recognized gestures in camera window, not with gesture recognition itself.

Is 320x240 the best possible resolution for gesture recognition? Does it use depth sensor only?

Many thanks,
Ivan

5 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.

Hi Ivan

Yes, the depth sensor is used for gesture recognition,  and the resolution of that camera is 320x240. 
Realize this is just one of hopefully many devices we will be able to utilize with the SDK in the future.   Hopefully, the additional devices will support higher depth sensing  resolutions. 

 Thanks for pointing out the confusion with our docs,   we will try and improve the clarity for the next release

-Eric 

I think Eric's comment is a little incomplete since "more resolution" needs to take into account total system considerations, for example:

- USB bandwidth -  Right now you have 3D+2D HD, plus audio pretty much maxing out the data pipe at maximum frame rate.  And remember increasing 3D resolution *cubes* the data being captured.  So more 3D data also has considerations for CPU bandwith, latency, etc.  Of course in the future the assumption is that PerC is integrated into the PC itself and there will be more bandwith to work with. 

- Frame rate - With the current USB design the system will do 2D HD at 30fps plus 3D capture at 60fps.  If you are willing to go with lower frame rates this would be one consideration to look at, but frame rate also ties into latency.

- Power consumption - The max power over USB is 2.5 watts.  More resolution means more power.  See note above about when this gets integrated into the PC.

- Field of View - Narrowing the field-of-view with the same resolution sensor will increase the resolution in the FOV "cone".  The current design was done with a FOV assuming a user's hands and torso in front of a laptop keyboard, but if you were willing to live with a narrower FOV you could increase resolution with the same sensor.  

...and a few others.

The PerC camera as-is is actually a very nice engineering balancing act taking all these vectors into account, of which resolution was one consideration.  But USB is a basic bottleneck preventing higher resolution without making some other tradeoffs as long as we are talking about an after-market peripheral.

 

 

Hi Eric and Mitch,

Many thanks for detailed explanation.

When iterating between GeoNode changes (QueryGesture()->QueryNodeData), any hints how to make positionImage.x and positionImage.y coordinates less jumpy?

Tried using some threshold and confidence param, but that is not enough.

Also tried changing device properties, but seems everything is read only, but PROPERTY_DEPTH_SMOOTHING.

Warm regards,
Ivan

On Creative camera application can change/set the following properties:

PROPERTY_COLOR_EXPOSURE

PROPERTY_COLOR_BRIGHTNESS

PROPERTY_COLOR_CONTRAST

PROPERTY_COLOR_SATURATION

PROPERTY_COLOR_HUE

PROPERTY_COLOR_GAMMA                                             

PROPERTY_COLOR_WHITE_BALANCE

PROPERTY_COLOR_SHARPNESS

PROPERTY_DEPTH_CONFIDENCE_THRESHOLD

PROPERTY_DEPTH_SMOOTHING

Leave a Comment

Please sign in to add a comment. Not a member? Join today