In recent years, virtual reality (VR) technology has progressed exponentially to enable immersive environments in which users feel a heightened sense of realism—that “you’re really there” feeling in the created environment. Across the board, CPU performance, GPU performance, VR headsets’ visual fidelity, and VR-enabled software have all advanced tremendously.
Games are the most obvious beneficiaries of VR technology and are already beginning to make the most of it. Other software genres can benefit from VR’s immersive capabilities as well, including education, training, and therapeutic usages.
However, as with many new technologies, it’s easy to implement VR that looks cool on the surface but has fatal flaws that pull you out of the immersive experience or ultimately make you wonder why someone went to the trouble of creating the software. Developers run the risk of having an initial “Oh, wow!” quickly become “What’s the point?”
Fortunately, that risk is avoidable. VR has been studied carefully, and a body of research on physiology and end-user responses to many types of VR software has revealed a clear set of guidelines for creating successful VR experiences that make the most of the technology.
In addition, Intel completed a significant effort to extend VR research. In that effort, researchers observed end users’ initial and continuing experiences with a variety of VR activities, followed by detailed debriefing sessions and questionnaires to discover the specific factors that made the experiences enjoyable. One key finding was a high statistical correlation between enjoyment and the level of immersion. The research also revealed several aspects of the games and environments that closely correlated with immersion, and therefore are key to extending that feeling.
The combined findings from existing and new research led to these guidelines for creating immersive VR experiences. These guidelines build on each other, with each level making it possible to advance to the next. The guidelines fall naturally into three categories:
Makes immersion possible in a virtual world by using technology that keeps the VR user safe from injury while wearing a headset; comfortably free from soreness due to hardware ergonomics and free from motion sickness; and undistracted by unrelated sights and sounds leaking in from the outside world.
Makes the virtual world seem real by providing smooth 3-D video, realistic sound, intuitive controls for manipulating the environment, and natural responses to the user’s actions in the virtual world.
Keeps the immersion alive and engaging, rather than being merely impressive, by enabling interaction with nearly everything in the virtual world. It also offers good content or gameplay that’s independent of technology, making VR interactions core to the experience, and easing the user quickly and smoothly into the virtual world.
This white paper describes guidelines in all three of these categories that will allow VR software titles to live up to the immersive potential now offered by the technology. This paper also references other sources of detailed technical guidelines where available.
Note: This white paper is intended for a wide range of VR software developers and hardware manufacturers whose needs vary. You may want to skip over sections that aren't relevant to your part of the ecosystem.
Elements with this symbol indicate guidelines that can be improved with increased CPU power.
Physical foundation guidelines are essential requirements to establish any immersion in a VR experience. For a player to begin trusting a system enough to become immersed, that player must feel safe, comfortable, and free from outside distractions. Many of the technical guidelines developed by hardware providers like Oculus*, Microsoft*, and HTC* fall into this category.
Be sure the entire system supports the user’s ability to move safely in the physical world while experiencing the virtual world. Responsible developers should:
- Inform users of the possible risks before they start any VR experience.
- Provide clear guidance about the physical requirements for the system (the motion envelope supported), even before the user puts on the headset.
- In the virtual view, provide “boundary” feedback before the user gets close enough to bump into the boundary1. The feedback could be visual, audible, haptic, or a combination.
- Provide a calibration step to accurately define the reach envelope, so the boundary is reasonably accurate for players of all sizes.
Make sure users are aware of the social consequences of their actions in the virtual world and are not put in social situations that are demeaning or dangerous. Responsible developers should:
- Require the user’s consent before entering an online social environment. If the social environment includes both visual and audio presence, make sure the user knows that both modes will be active prior to entering.
- Give the user control over their online avatar’s appearance and identity. For example, don’t assume users want their Steam account name to be their screen name in a social virtual world; allow them to confirm (or change) their screen name before entering the social situation.
- Never condone demeaning or dangerous personal interactions between users in an online social environment.
- Provide ways for users to encourage good social etiquette. For example, providing visible ways to flag bad actors can sometimes be enough to prevent the bad actions in the first place.
As soon as users start to feel uncomfortable wearing a headset or holding the controller, they stop feeling immersed. Responsible developers should create adjustable hardware that can be fitted appropriately to a player of any size. For example:
- A good headset fit means the headset does not rest heavily on the bridge of the nose or cheekbones, where the weight causes soreness over time.
- A good controller should fit into a range of hand sizes. For example, it should fit the hand of a 5th-percentile female (breadth = 7.34 cm or 2.89 in), yet not risk causing accidental button presses when used by a 95th-percentile male (breadth = 9.76 cm or 3.84 in).
- Design for a range of human dimensions by referring to anthropometric data tables like those published by the military2.
In the past, VR-induced motion sickness (simulator sickness, cybersickness, etc.) was a huge concern. One study published in 1999 found that of 148 participants, 80% reported some experience of VR-induced symptoms and 5% suffered serious effects3. Modern VR systems have come a long way, but the risk of cybersickness is still a major concern.
Of course, some people are more susceptible to cybersickness than others: research has shown heightened susceptibility in women, children, and those suffering mild illness or sleep loss4. But if the goal of a VR system is to be enjoyed by anyone who wants to use it, preventing such symptoms should be a high priority for any development effort. There are many excellent resources on this topic, including Oculus “best practices”5, Google Cardboard guidelines6, and a textbook called “Virtual Reality” by Steven LaValle of the University of Illinois7.
Here are just a few examples of the guidelines promoted by those experts for preventing cybersickness. See the references for more details.
- Respond faithfully to the user’s movements at all times, preferably at or near typical human locomotion speeds. Strive for zero latency, especially in head tracking.
- Run code at frame rates equal to or greater than the hardware refresh rate, to avoid judder.
- Make sure the eye viewpoints are located correctly, considering stereo offsets.
- Avoid having moving objects that take up a large portion of the user’s field of view, to prevent feelings of self-motion.
- Make acceleration infrequent and short, preferably instantaneous.
- In teleporting, provide adequate visual cues to retain bearings and preserve original orientation if possible.
Occluding the Real World
Feeling immersed in the virtual world requires that the physical system sufficiently prevents the real world from leaking into the virtual experience and causing distraction.
- Provide sufficient padding around the headset so that no light leaks through.
- Provide an audio soundtrack consistent with the virtual world to set context and prevent audible distractions from the user’s external physical location.
After assuring safety and health, the next step in creating an immersive experience is crafting a convincing world for the user to explore. Obvious mismatches between the user’s expectations and the virtual world will, at best, subconsciously block them from being fully immersed. At worst, they become an obvious distraction that spoils the illusion akin to a film editor failing to crop out a microphone boom or camera rig in a movie scene.
The graphics of the virtual world will be the primary way that users will experience it. While highly realistic graphics are desirable, consistency and smoothness of the graphics are even more important benchmarks than realism. Imperfections such as heavy pixelation, tearing, or inconsistent levels of detail are all visual artifacts that will draw users out of the VR world.
- Make sure the world is rendered correctly, with no technical cheats if there is a possibility they will be spotted by the user5.
- Consistent frame rate, even during moments of high actions and movement, is critical to maintaining player focus on the virtual world1.
- The best environments have interesting horizons and detailed skies, but with calm or dark floors8.
- Take into account how the virtual grid representing the player’s real world environment will be folded into the visual experience. For example, when the image below the floor grid boundary is full of details, it makes the grid appear to be levitated off the ground8.
- Make sure any background panoramas are seamless, so there are no hard transitions where the left and right sides of the image meet8.
- Increased graphical clarity of human characters or the players’ avatars has diminishing returns because of the “uncanny valley.” This theory hypothesizes that as human replicas approach a convincing level of reality but aren’t quite there, they elicit greater feelings of revulsion from observers than mere cartoons.
Being able to tie a sound to its location is an important cue for users navigating the virtual world. A lack of spatial sound or a mismatch in the perceived source of the sound will draw the user’s attention away from the virtual world. For that reason, most or all sounds should be spatialized.
- Assist users in making sure their headphones and sound settings are configured correctly to optimize the sound experience.
- Even music should be spatialized, particularly if it is tied to a menu or UI element, as opposed to being ambient9.
- It is better to embrace sound expectations rather than subvert them. For example, when a user hears birds, they are likely to look up because of their experience in the real world, even if that noise has been spatialized below them9.
- Sounds should get louder as the player gets closer to the source. This includes the leaning of their head, not just the movement of their avatar5.
- The orientation of the user should also affect the quality and magnitude of the sound (e.g., facing toward the sound or facing away).
- Include the Doppler effect, as it provides valuable motion cues7.
- Avoid invisible or unidentifiable sources of sound, unless the goal is to create a sense of confusion and deliberately unnerve the player9.
Another important role of sound is to set the stage for the scene and bring the player into the virtual world both perceptually and emotionally. For example, the ambient creaks and howls of a haunted house enrich the environment while the spooky music sets the mood and communicates tone to the user.
- Low levels of ambient noise can also serve to mask ambient noises from the user’s actual environment, decreasing distractions from the virtual world.
- However, it’s important that ambient sound does not drown out the spatial sound. If the ambient noises are too dominant, they will make it difficult for users to locate sounds from other characters or events.
Ubiquitous sound effects
Just as users expect everything in the environment to respond visually to their interactions, they also expect everything to have a realistic audio response. Whenever an object is grasped, dropped, thrown, or manipulated, users will expect a sound effect that matches.
- Specificity of the sound effects is important. Different objects make different noises when they are grasped or collide, and users will be attuned to any mismatches between their expectation and the sound they hear.
- Real sounds are better than synthesized or unnatural sounds. The more familiar users are with that sound, the more real it will feel and the more easily they will be able to identify it within the environment and maintain the illusion of reality9.
Cues in the environment
If VR users are truly immersed in the virtual world, they will expect to interact with the environment. Every action should be met with the appropriate responses aurally, visually, and haptically.
When starting out with VR, users will expect nearly everything in the environment to be interactive. If only certain things in the environment are interactive, be sure to explain clearly to the player so they can accurately predict which parts of the world they will be able to interact with. This avoids the discouraging experience of guessing and then being disappointed by objects that look real in the VR world but are inert. This communication and “training” should start early as users will begin to form a mental model around interactivity from the beginning of their experience.
Currently, haptic feedback in VR lacks fidelity and specificity, but still adds to the feeling of immersion by making the experience more tangible. Each VR world will need to develop its haptic language to communicate with different patterns. For example, a quick, light vibration might represent the user picking up an object, while a violent pulsing vibration could represent a game player taking damage.
No discernible lag
It is critical to maintain high accuracy and zero latency tracking of the user’s head and hand movements to maintain the user’s immersion in their virtual body. Mismatches or delays in timing are not only jarring, but can lead to motion sickness and breakdown of the user’s physical comfort.
It should be easy to do things like select, grasp, manipulate, carry, throw, and place objects or other environmental features. The level of interaction does not always need to be completely realistic, as going too far can have the effect of making small tasks very tedious and difficult.
In fact, it is important that the precision required in the real world is not required in the virtual world, because the player has much less practice and feedback for these manipulations than they do in the real world. The realism will come from the unique ways in which all these objects respond to interaction, the unique sound effects they produce, and the way that objects can be combined.
Utilizing features such as a “basin of attraction” can give players more leeway in how they manipulate objects so they don’t get distracted by repeated failures to do simple tasks. In this context, a basin of attraction represents a set of user inputs that will result in a desired action. For example, if a player is attempting to place a bottle upright on a table, rather than applying an extremely realistic physics engine to see if the bottle stays upright, the VR application should accept a larger range of orientations that will result in the bottle staying upright7.
Limit large movements
Avoid requiring users to make lots of large movements with their arms or you risk the possibility of players getting “gorilla arms.” This is a phenomenon seen in some VR titles where players are required to play with their arms always extended, making large gestures, causing fatigue to quickly set in.
It’s easy to believe that the good equipment and immersive VR capabilities described above are all that a software title needs to be successful. While it’s true that such a title has significant advantages, it’s not enough. The analogy of 3-D in a movie theater is instructive. No matter how cool the effects of falling snowflakes or rockets coming towards the audience, those novelties soon wear off if the movie isn’t providing an experience that’s worth watching—that is, good characters, a story arc that maintains interest, and dialogue that an audience wants to listen to.
In the world of VR that analogy holds strongly. There are VR-specific principles for creating an experience that users will enjoy even beyond the attraction of interacting with a virtual world. This section describes those principles as well.
First comes good content. The most exciting interaction in the world will soon grow tedious if it does not serve worthwhile content. This principle isn’t specific to VR, but that is why the principle is important: VR is a wonderful means to a goal—a great gaming or educational or training experience—but is not the goal itself.
This will naturally vary by genre. For games, good content means fun gameplay, progressively tougher challenges, reward systems, engaging graphics, an interesting storyline, and the other fundamentals of good games that have been studied thoroughly for many years. For educational software, good content means accurate information, varied ways of encountering the information, some means of assessing learning, and so on. For training and therapeutic applications, success factors are similarly well known.
VR developers will be most successful when they remember that the technology is a tool for interacting with good content and not a crutch that makes up for mediocre content.
Because a virtual world is at the same time very different from the real world but nearly as believable, many of the most successful VR titles take steps to smoothly transition to the virtual world. Here are two methods of doing so, either or both of which can be effective:
Good tutorials take place within the virtual environment, not in a purely instructional 2-D space, to rapidly give the user a sense of visual familiarity once the game or activity begins. Tutorials should cover the use of controller hardware (e.g., wands for hands), specialized actions such as picking up or throwing objects, and motion. Motion is important because, although advanced VR systems allow you to move forward and laterally, those movements are limited by the size of the room while the VR environment may be miles across. As a result, most games will have some system for teleporting or moving farther than the user moves in the physical room. Such gestures require tutorial help in the actual VR environment. Integrated tutorials are already a best-known method in standard games, but they’re even more important with VR.
Gentle but rapid transitions
To reduce the shock of entering a new environment, software can help the user transition gradually to the virtual world. For example, on startup the VR title could avoid an abrupt and confusing lurch by fading in the ambient soundscape, then the images. The SteamVR tutorial uses this approach, letting the user watch while it modifies the environment. Only after that brief but meaningful transition does the tutorial ask the user to begin interacting with the virtual world.
In a VR environment, users assume that everything is there for a purpose and, therefore, expect to interact with everything around them. In 2-D PC video games it’s common to have some objects be “live” and interactive while others are static and can’t be touched or moved. Developers may assume the same is true of VR games, but it is not.
Agency to interact
Studies with end users have shown that the heightened reality of a VR environment leads them to expect to touch, pick up, or examine the objects around them for the simple reason that everything seems real. Users rapidly come to expect that their agency (their ability to choose what to interact with and how to do so) will be respected. When several objects are interactive and then one is “dead” and can’t be manipulated, it’s an intrusive reminder that this is an artificial world. The unfortunate effect of such gaps is that users may pull out of the immersive experience. By contrast, the more that objects are interactive in the virtual environment, the greater the chance that users continue their engagement.
Enough content to sustain interest
Part of ubiquitous interactivity is the idea of sufficiency. Not only should the content be consistently interactive, if even in small ways, but there must be enough content to keep the user’s interest. Because VR is a highly sensory experience, successful titles will strike a balance, providing enough content to maintain that sensory experience but not so much that it becomes overwhelming.
Primacy of VR Interactions
Good VR software is designed to make the most of VR capabilities. It’s easy to imagine activities that might use VR in somewhat interesting but still superficial ways. One example might be a VR reading app that allows the user to walk through rows of library books, hearing footfalls, and selecting a book before sitting down to read it. In this case the VR interactions might be novel, but over time they can start to feel like window-dressing that just gets in the way of the main activity.
By contrast, software that emphasizes the unique capabilities of VR and puts them to use in the primary interactions will be much more effective in sustaining the user’s interest and continuing usage.
Jeremy Bailenson, director of Stanford’s Virtual Human Interaction Lab, said that experiences worth developing in VR can be described by one or more of the following adjectives: rare, impossible, dangerous, expensive10. A further rule of thumb for a good VR fit then is to determine if the answers to any of the following questions are yes:
- Rare. Does the activity happen so infrequently (e.g., seeing a supernova or a little-known sea creature) that it’s impractical to do in real life?
- Impossible. Is it imaginary (e.g., a world of monsters) or does it violate the laws of physics (e.g., humans flying like hummingbirds) in such a way that it simply cannot be done?
- Dangerous. Is injury or worse likely to happen if you carry out the activity? Examples might be surgical practice for a beginner, skiing at high speed ahead of an avalanche, or dropping grenades while racing through city streets.
- Expensive. Does it cost too much money (e.g., traveling around the world or rebuilding the Taj Mahal) for most people to reasonably do it?
If at least one answer is yes, the activity could be a good candidate for software that creates a VR experience. Such situations are ideal for creating a sense of reality to simulate activities that most people would never be able to experience otherwise.
Virtual reality’s capabilities have come so far that a remarkably convincing sense of immersion is now possible. Achieving that in a sustainable way requires VR developers to pay careful attention to several categories of possible distractions: foundational issues of safety and comfort, basic sensory realism, and an implementation that goes beyond the mere novelty of VR to really sustain a user’s engagement.
Following these guidelines to create VR software titles that give users worthwhile immersive experiences will allow developers to achieve real, and not just virtual, success with this new technology.
Many thanks to Elise Lind and Colin Bay of Thug Design for their help with this article.
1. HTC Corporation (2016). Develop for Vive and SteamVR. HTC Vive Developer Portal, available at https://developer.viveport.com/us/develop_portal/
2. Military Anthropometric Data Tables, excerpted from Gordon, Claire C. et. al. 1988 Anthropometric Survey of U.S. Personnel: Summary Statistics Interim Report. March 1989. Available at http://www.mcieast.marines.mil/Portals/33/Documents/Safety/OSH/Anthropometry%20Data%20Tables.pdf
3. Sharples, S., Cobb, S., Moody, A., & Wilson, J.R. (2008). Virtual Reality-Induced Symptoms and Effects (VRISE): Comparison of head mounted displays (HMD), desktop, and projection display systems. Displays 29, pp. 58-69. (Elsevier: 2008) Available at https://pdfs.semanticscholar.org/2a88/a274795c76b41583a595a7cc6181aaf064b3.pdf
4. LaViola, J.J. (2000). A Discussion of Cybersickness in Virtual Environments. SIGCHI Bulletin Volume 32/1, January 2000, pp. 47-56. Available at http://graphics.cs.brown.edu/~jjl/pubs/cybersick.pdf
5. Oculus VR, LLC (2016). Oculus Best Practices. Available at https://developer3.oculus.com/documentation/intro-vr/latest/concepts/bp_intro/
6. Google (2016). Designing for Google Cardboard. From Google Design Center, available at https://www.google.com/design/spec-vr/designing-for-google-cardboard/a-new-dimension.html
7. LaValle, S. M. (2016). Virtual Reality (textbook). To be published by Cambridge University Press, downloadable at http://msl.cs.uiuc.edu/vr/vrbook.pdf. © Steven M. LaValle, 2015, 2016.
8. Valve Corporation (2016). How to make comfortable SteamVR backgrounds (member contributed guide). Available at http://steamcommunity.com/app/250820/guides/
9. Microsoft Corporation (2017). Windows Holographic Documentation. Windows Dev Center, available at https://developer.microsoft.com/en-us/windows/mixed-reality/design
10. Oremus, Will (August 2016). What Virtual Reality Is Good For, Slate, accessed November 2016 at: http://www.slate.com/articles/technology/future_tense/2016/08/the_only_good_reasons_to_use_virtual_reality_and_the_current_vr_renaissance.html