Iridium Studios: Voice Commands, FPS, and Intel® RealSense™

Note:  If you're interested in Intel® RealSense™, then you'll want to check out the 2014 Intel® RealSenseApp Challengea contest intended to encourage innovative apps that take advantage of everything that perceptual computing has to offer. Using the Intel® 2014 Software Development Kit (SDK) and the brand new 3D gesture camera, developers will be able to show off their ideas, spark future imagination, and maybe even take a few prizes home from the $1 million dollars’ worth of cash and promotions offered. Interested? Check out the official 2014 Intel® RealSenseApp Challenge page to get started!

Jason Wishnov, CEO, Founder and Lead Designer at Iridium Studios, recently sat down for a brief video interview in which he talked about Iridium’s new tactical war game titled “There Came An Echo” and how it uses the new Intel® RealSense™ Technology  [formerly Perceptual Computing] speech recognition capabilities to direct the combat and game play with recognizable voice commands, much like on a real battlefield. Click the image below to watch what Iridium has been up to lately:

First Person Shooter games, otherwise known as FPS, are very linear: you’ve got a disembodied voice in your ear telling you to go here or shoot there and there’s not a sense of really being part of a bigger story. In other words, you’re not in control of the action – someone else is. There’s not a true narrative. Iridium Studios figured out that it would be an interesting experience to turn that around that is on the other side of the game: the user gives commands, and the user controls the story.

Ideas are easy – it’s implementation that’s the tricky part. How do you make the user truly in charge of the gameplay? Buttons? Controls? What if you could give them voice command control, much like on a real battlefield, with accuracy? That’s what Iridium Studios set out to do, and they’re reporting that they’re getting up to the mid-90’s range as far as accurate voice commands carried out.

Human and computer interaction is evolving beyond the traditional input controls that we’re familiar with. RealSense technology makes it possible for our digital worlds to interact with our physical, organic worlds in meaningful ways. Many of the projects that developers are creating step across boundaries that just a few years ago would have been impossible to imagine.

If you can visualize controlling your computer merely by using your voice or a wave of your hand, rather than a mouse, a keyboard, or even a touchscreen, then you can see just the beginnings of what this technology is capable of. RealSense computing focuses on natural human interactions with machines in addition to those familiar control apparatuses many of us have literally grown up with: facial recognition, voice commands, gesture swiping, etc.  Responsive computing that is individually tailored to an individual’s unique needs.

Are voice commands better than actual controls? Jason reports that from an accuracy-focused point of view, the traditional controls probably will win out (for now), but voice controls are quickly gaining ground. Plus, voice controls give you a much more intuitive sense of immersion in the game, as you can see from the video above. Instead of being told to run here or there, you’re actually able to give commands, and use your own battle tactics to solve problems as they come up. That’s a heady mixture that is sure to take off as the technology grows more sophisticated. RealSense is all about natural human interactions with machines in addition to those tried and true control mechanisms: facial recognition, voice commands, gesture swiping, etc., and this lends itself especially well to FPS games.

RealSense technology gives developers more of an opportunity to create a greater experience for game players. Iridium Studios utilized the new RealSense SDK to construct their newest projects, and report that between updated support for voice commands and Unity integration, they were really able to accomplish a lot. Their goal? Put out great games, of course, but also to convince gamers that alternative forms of controls like voice and gesture are viable forms of control in FPS games.

What kinds of innovations are supported by this SDK? There are several:

·         Speech Recognition:
Speech recognition is a common way to add perceptual computing interactivity to an application. Examples of this might be direct PC command or dictation. For speech recognition, when a user speaks to their computing device, the speech recognition algorithm in the SDK interprets the speech, recognizes that the user has spoken a command pre-programmed into the application, and passes the command on to the application

·         Hand and Finger Tracking:
Close-range tracking is the overall term for a sub-category of perceptual computing interactivity that includes recognition and tracking of hand poses, such as the thumbs up, hand and finger tracking, and hand gestures. This usage is made possible through the Creative Interactive Gesture Camera’s 3D capability.

·         Facial Analysis:
Face tracking can be used as a perceptual computing component in games or other interactive applications. The Intel Perceptual Computing SDK supports facial recognition, facial tracking, and attribution detection such as smile.

·         Augmented Reality:
2D/3D object tracking provides the user with an augmented reality experience in which real-time input from the Creative*Interactive Gesture Camera (RGB video, depth map and audio) is combined with other graphics sources or video.

·         Background Subtraction:
Using information from the depth cloud, we've developed exciting technology that allows developers to separate objects/people in the foreground from the background. This technology works in real time, meaning users can eliminate irrelevant or cumbersome background information from video to immerse themselves in video chat, online collaboration, and more. 

This SDK is absolutely free to download, and enables developers to be on the cutting edge of creation with their apps, integrating facial analysis, speech recognition, hand and finger tracking, and 2D/3D object tracking. Developers are encouraged to purchase the Creative Interactive Gesture Camera Developer Kit, a depth-sensor camera that is designed for use with the SDK for better object tracking. You’ll also want to check out the Perceptual Computing Forums, a message board dedicated to developers and the challenges faced in developing apps using this new technology. Topics on the board right now include “How to get the dictation in command and control mode?”, “Problem while detecting more than one face Attributes”, “Details about markerless object tracking”, and “Gesture Detector problem with custom depth image data”.

If you’re a developer, what ideas have you come up for RealSense and “real life” integration that might work? Give us your thoughts in the comments!





For more complete information about compiler optimizations, see our Optimization Notice.