The incredible power of voice

By Matteo Valoriani, Published: 10/10/2018, Last Updated: 10/10/2018

The incredible power of voice

Here we are with a new post and with this week we finish the description of the main features of our project TraIND40. In the next and last post of this Challenge we will be ready to tell you about our project in its totality. 

This week we want to address another aspect that can greatly enhance the user experience within a VR/MR app. 
During the post about UX and UI we talked about the "learnability" of UX, that is how long the user  discover and remember the main functions and mechanisms of interaction, and “usability”, how easy it is for the user to interact with the system. Very often in VR apps these aspects can be very critical.  

In the post number 3 we dealt with the theme of Mixed Reality device controllers like HPXXX and we saw that they are very feature-rich and can be used for various purposes.  

All these possibilities can be difficult for the user to manage and voice commands can be an easy to remember shortcut to speed up many operations. Voice input is a natural way to interact with an object in mixed reality. It can help users to reduce the time and minimize effort. Windows Mixed Reality shell provides system level commands such as “Select”, “Close”, “Move This” and “Face Me”, but with MRTK, we can easily add voice commands to our experiences.  

The steps to be taken are the following: 

  1. Enable microphone capability
  2. Add SpeechInputSource
  3. Define new Keyword and relative keyboard shortcut 
  4. Create a script that implement interface ISpeechHandler 
  5. Handle the event OnSpeechKeywordRecognize  
public class SpeechCommandManager : MonoBehaviour, ISpeechHandler {
    public void OnSpeechKeywordRecognized(SpeechEventData eventData)
        // .....
        string word = eventData.RecognizedText.ToLower();
        switch (word)
            case "close menu":

            case "remove":
           // ....    

 And this is the result.


See you next week with the final video and final post.


Product and Performance Information


Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.

Notice revision #20110804