Hand Tracking and Voice Reco can not be done by different processes simultaneously!

Hand Tracking and Voice Reco can not be done by different processes simultaneously!


UPDATE: It doesn't have to be from two different processes.  I added voice recognition support to the C++ app that does the (full) hand tracking and got the same crash with one small difference.  I was able, sometimes not always, to get 1 to 3 successful recognitions before getting the crash.  Other times the crash happened right after initialization.

Has anybody been able to get full hand tracking working with speech recognition active at the same time!

UPDATE 2: I have no isolated the operation that causes the speech recognition engine to fail.  It's right after I initialize the PXCSenseManager session via it's Init() call.  If I don't make that call (and obviously don't do any subsequent SDK calls), then the speech recognition works fine.  The instant I do make that call, the speech recognition engine gets the REC_ABORT and REC_END calls as indicated below.  So just initializing the session is enough to cause the problem.  Note, I'm sure it's not my code because you can run the test below with the un-modified Intel Hands Viewer and Speech Recognition samples to reproduce the problem.

---- Original Message ----

I have a two apps that cooperate to implement my total RealSense enabled app.  One app handles hand tracking and the other does voice recognition.  However that doesn't work because the speech recognition thread in the app doing speech recognition always fails quickly with a REC_ABORT followed by a REC_END message.  I originally thought the problem might be a CPU overload issue as I wrote here:


However that is not the case.  You simply don't seem to be able to have one processes talking to the camera while another process does speech recognition.  Here's a real simple test:

- Run the Hands Viewer sample from the Intel RSSDK code samples.  Start the sample until you see the depth video stream.

- Run the Speech Recognition sample next.

When you start the Speech Recognition sample, soon after you see the "Init OK" message you will see the REC_ABORT and REC_END messages.  If you stop the Hands Viewer sample and shut it down, the Speech Recognition sample works again.

So is it not possible at all to have one process access the SDK/camera for hand tracking and another process do speech recognition?  If it is possible, how do I make that happen?

8 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.

You could try using a multi threaded approach instead of multi processes.


Already tried that.  Doesn't help.  Still get the problem except sometimes I'm able to get 1 to 3 max successful recognitions before it breaks, which may be due to the fact the test app is C++ instead of C#.

Hi @samontab,

I had the opposite problem using C#.

If I try to use full hand tracking and voice recognition on the same process, it crashes, even in another thread.

Using different processes worked for me.

I will try to create a code sample that reproduces the error. and post here.


A little extra information on the speech recognition problem. The speech recognition only crashes if I try to use the RealSense camera as the microphone. If I use another microphone, then the speech recognition doesn't crash.

So two points Robert,

1. The commercial integrated camera will not have a mic, it will use the system mic - so programming for a separate mic now will hold up well in the long term.

2. Bryan (creator of Blockhead and other code samples on IDZ) showed both running on his HP elitebook. 


It appears to be a problem that only occurs on my development machine.  I had two people run the exact same test.  One with a PC that used the external camera like mine, another with an integrated laptop.  Neither of them had the problem.

At this point I believe it is a problem with the SDK that only occurs in a minority of PC environments like mine.  Seeing as integrated device sales will (in my opinion) far outstrip users of the external camera, my peculiar system configuration that is somehow hostile to the SDK won't be shared by many others (if any).  Because of this, on my development station I'm simply not going to use the camera's microphone and will use a different one instead.

Additional data:  I can run two apps simultaneously that share the camera.  It's only when I try to run two apps where one needs the camera and the other the microphone that the problem manifests itself.  In fact, the moment I run an app that uses the camera and then initializes the sense manager, if I then restart the speech recognition app, the VFW800 disappears from the available audio device list (Source).  So on my PC there is something really strange going on that causes this.  Total speculation, but I wonder if in my case something happens when the SDK/sense manager initializes that makes the microphone actually disappears from the available and usable audio input device list. In any case, I am moving on with my RealSense development using an external mic.  If anyone manages to solve this in the future, please post your solution.

Hello all,

I eventually found the problem that was causing my development system to be unable to do hand tracking and speech recognition at the same.  All of this was due to an old version of libpxc_d.lib that was hanging around from the last SDK (version 5).  Astonishingly my app could still run using that old LIB file, which made finding and debugging the issue a lot murkier.  I didn't discover what was the root cause until I loaded ProcMon and saw that when PXCSession_Create() was called, the SDK was trying to open up the registry key for the SDK DLLs that had "/v5" in the sub-key path instead of /v6.  That's when I realized it had to be an old library file.  

Everything works fine now. Sorry for the trouble.

-- roschler



Leave a Comment

Please sign in to add a comment. Not a member? Join today