Hello guys, welcome to another dev diary post from our Minerva Lab project! New here? Don’t know what I am talking about? Check these previous posts! We are approaching the final line and we have some nice things to show you, take a look into it.
Today we are going to talk about probably the most important subject (or problem) you can find when developing for Virtual Reality, performance. If you are reading about this you perhaps have the knowledge that one of the main issues with comfortable VR experiences is the cyber-sickness.
But why are you talking about cyber-sickness and performance? The first step to improve the User Experience is to keep a high framerate, in fact this was one of the great evolutions from the previous VR HMD’s generations, screens that used to have a 30 Hz refresh rate now can be about 120 Hz. This improvement made the synchronization between eye and head movement more fluid and decreased the cyber-sickness.
So, in order to maintain a high framerate, we must process the render cycle much faster, meaning that to have great fps we have to handle with game updates and render pipelines quicker than when build a common application (which has around 30 frames per second). Nowadays developers have been more comfortable in developing games, and rarely have problems with performance (since consoles, PCs and even mobiles got a lot of power to render graphics), but in VR we must turn ourselves to much careful development as developers used to be at the first gaming generations.
Luckily, we do have amazing tools to help us in this endeavor, both main game engines that I have been talking in our posts (Unreal and Unity) have amazing profiling tools that can be easily accessed. So, I believe my job here is done, right? WRONG. To have access to a tool is something totally different from know how to handle it, and most of the information you can extract from profiling profiles will just make you confused about where to go, as sometimes they will not solve your problem.
First I want to share a slide that was presented in Oculus Connect (which I had the pleasure to talk a little bit in this article), that summarizes how to look for problems and how to act to fix the issues. In short, you can have two main kind of bottlenecks in VR, CPU Bound or GPU Bound. The first being more common for mobile VR experiences (since they deal with less powerful processors) and the latter more related to dedicated graphical cards.
Most of the times, at least from our experience, the bottleneck is the GPU. But do not despair, there are great tips to increase the performance with little impact on the quality of your experience. First of all is how to handle lights in your scene.
In Unreal Engine there are 3 types of lights: dynamic, static and stationary. Dynamic lights are the more expensive (regarding processing) and would be better to avoid to use them at all when developing for VR, they generate real-time shadows and can be moved in runtime to other positions.
Static lighting is a type of light that needs to be configured in previously of the execution of the experience, meaning that the are able to change intensity, color or position. In fact, these lights are so fixed that you can “compile” them. In Unreal Engine you can build these lights and it will generate a lightmap over the textures of the environment and you will have no cost in runtime regarding of how many lights you put in your scene or how big is your map. The trade off is that you need to process all this lighting and saving it takes some memory that will need to be loaded when changing the maps.
The last, but not least, is the stationary lights. They are much like static lights, but they can change intensity and color in runtime. They also generate real-time shadows, but are much less demanding the dynamic shadows. Unfortunately, at least in Unreal Engine, you must never overlap the influence of more than 4 lights in the same place.
Another great feature that can be used to create lighting effects with almost zero cost for the render loop is to have multiple lightmaps of the same environment. In this example, there is a clear explanation of how to change only the light setup of a map, which can be done in runtime as well. We used this feature to turn out the lights of the room on our crime scene, making possible to use specific tools that only act in low-light conditions.
Nevertheless, is always great to stress out how good work Epic has been doing in providing examples to teach through practice Unreal Engine, for instance this Realistic Render example, show how to combine Materials, Light Functions and different types of lights to have the best photo realistic result in a scene.
It is worth to note that this is not an exclusive feature of Unreal Engine, since the launching of Unity 4 this other engine has make available the same feature to have built lights. To be honest, in most of the discussions I have with colleague this feature does not always work as good as in Unreal Engine, but I am sure that the guys from Unity3D are always improving it and it is just a matter of time (if is not already ready) to have a similar solution.
There are other Performance features on Unreal Engine that are “a must” when developing for VR, it is noteworthy to talk about Multi-view (or Instanced Stereo), that save a lot of processing not rendering the same view twice (which would be needed since we have 2 perspectives, one for each eye).
The other is the Forward Rendering technique, in opposition of the default Deferred Renderer that might not be right for all VR experiences, and has been more commonly used for mobile rendering cycles.
But last, and not least there is always the rendered resolution that can be adjusted. For instance, when developing for Oculus Rift you can take use of Dynamic Resolution which auto adjust the screen percentage to keep a frame rate more constant. Nevertheless, when dealing with very complicated scenes that you can always play with lowering the pixel density only for a time in your experience, or even to increase the density when detecting the player to have a more robust system.
That is it for today! Thanks for reading and I hope you all have enjoyed it! Please be tuned to the next post!