The incredible power of voice

The incredible power of voice

Here we are with a new post and with this week we finish the description of the main features of our project TraIND40. In the next and last post of this Challenge we will be ready to tell you about our project in its totality. 

This week we want to address another aspect that can greatly enhance the user experience within a VR/MR app. 
During the post about UX and UI we talked about the "learnability" of UX, that is how long the user  discover and remember the main functions and mechanisms of interaction, and “usability”, how easy it is for the user to interact with the system. Very often in VR apps these aspects can be very critical.  

In the post number 3 we dealt with the theme of Mixed Reality device controllers like HPXXX and we saw that they are very feature-rich and can be used for various purposes.  

All these possibilities can be difficult for the user to manage and voice commands can be an easy to remember shortcut to speed up many operations. Voice input is a natural way to interact with an object in mixed reality. It can help users to reduce the time and minimize effort. Windows Mixed Reality shell provides system level commands such as “Select”, “Close”, “Move This” and “Face Me”, but with MRTK, we can easily add voice commands to our experiences.  

The steps to be taken are the following: 

  1. Enable microphone capability
  2. Add SpeechInputSource
  3. Define new Keyword and relative keyboard shortcut 
     SpeechInputSource 
  4. Create a script that implement interface ISpeechHandler 
    SpeechCommandManager
  5. Handle the event OnSpeechKeywordRecognize  
public class SpeechCommandManager : MonoBehaviour, ISpeechHandler {
    public void OnSpeechKeywordRecognized(SpeechEventData eventData)
    {
        // .....
        string word = eventData.RecognizedText.ToLower();
        switch (word)
        {
            case "close menu":
                menuScript.UpdateState();
                break;

            case "remove":
                menuScript.Remove();
                break;
           // ....    
            default:
                break;
        }
    }
}

 And this is the result.

 

See you next week with the final video and final post.

 

有关编译器优化的更完整信息,请参阅优化通知