The Intel Power Checker 2.0 now supports measurement both on battery and with the system plugged into an external AC power source. External power measurement is only supported on Intel® Second Generation Core processors and if the Intel® Power Gadget software has been installed.

For this article, I took a very compute-intensive parallel application that I wrote to solve instances of the logic puzzle Akari. The code uses a backtracking algorithm to explore how to place light bulbs onto a grid under constraints dictated by the rules of the puzzle and the layout of the puzzle instance. Potentially millions of independent tasks can be generated by the code as the solution space is searched by threads executing those tasks. This solution method is eminently scalable to a large number of threads and is able to keep many cores running at peak speed for a sustained amount of time.

How to Use Intel Power Checker

The Intel Power Checker provides a GUI wizard that leads you through the four steps of power analysis. These four steps in the checker are described below. Before starting the assessment, be sure to know which section of your application (a workload) you want to be measured, as the Power Checker will only measure a 30 second execution interval. (If you want to measure the entire execution workload, you should try some other tool, like Intel Power Gadget.) Your workload could be a compute-intensive portion or an I/O-intense section or just some point in execution that typifies the majority of expected usage.

Step 1: Specifying the Power Meter device

If you have an external power meter attached to your test system, you can select the model being used on the first screen of the wizard. The default is that no external device is being used. For this default case, Intel Power Checker will determine if the system is capable of providing power consumption data and if the correct power driver, EzPwr.sys, is installed. (The driver is part of the default installation of Intel Power Gadget.)

Step 2: Measure System Baseline

The first measurement that the Intel Power Checker will perform is on the next screen within the wizard. This is to measure the baseline power consumption of the hardware without your application running. Prior to this measurement phase any unnecessary processes such as operating system updates, Windows Indexing Service, virus scans, media players, and internet browsers should have been shut down. In other words, to get the most accurate results you should make your test system as idle as possible and ensure that nothing will become a foreground process during your measurement runs.

Once you have a quiescent system, click the “Start” button to begin this phase of the testing. The Intel Power Checker waits 15 seconds to allow the system to come to an idle state before starting the measurements. You need to be sure to position your mouse and the keyboard out of reach, or keep your hands away from them, to avoid any stray contact that might trigger some response from the platform. After the pause, the checker will observe the system for 30 seconds in this idle state. A progress bar will show the time remaining in each part of this phase. Once the baseline data collection is complete, click the “Next” button to proceed to the next phase.

Step 3: Measure Active Application

Before you are taken to the next screen in the wizard, you are instructed to start the application you are interested in measuring. Start up your application and click the “OK” button to advance the GUI to the next screen. Once you have reached the Step 3 screen, use the scroll bar to locate your application in the process list and click on that line to select it. If your application is not listed, click the “Refresh List” button so that your application’s process will be available to select. In addition, you can use the “Apply Filter” button to narrow down the list in order to find your application’s process quickly. .After selecting your application from the list, click “Next” to move on to the data collection for this phase. Before starting the assessment, be sure your application has reached the desired point of measurement. If there are some initial setup computations that are not of interest, you will need to get past this point before letting Intel Power Checker begin measurement. For my Akari application, there is very little setup time. It was typically in the thick of computation by the time I had gotten to the point of selecting the process from the list.

As soon as I could, I clicked the “Start” button to begin capturing measurement data. Since this is one of the crucial power measurements for your application, always begin capturing data after the workload or critical section has begun and make sure this active execution will run longer than the 30 seconds needed to complete the measurement time.

Step 4: Measure Idle Application

The final phase is to measure your application’s idle power consumption. This is another important phase of energy efficiency measurement of an application since your application must not only do efficient computation, but also not waste energy when sitting idle.

This step doesn’t make much sense within my compute-intensive application since there is no idle state of the application. Once you start the application on a given puzzle instance, it simply computes all legal solutions in parallel and then ends. As (multiple) solutions are found, they are printed out by the thread that found it. If there are no solutions, a message is printed just before the application terminates. This latter case describes the workload I used for my tests. Because you must have your application running in “idle” mode for this step, I left the application running at full speed and simply allowed Power Checker to take its measurements.

If your application does have an idle state, perhaps waiting for interaction from the user, the checker will give the system 15 seconds to calm down fully before taking a final 30 second measurement.

Upon completion of this last data collection phase, you will be able to proceed to the results screen within the Intel Power Checker wizard. After all three measurement phases have been completed; a Tool Report File will be generated containing all of the results for later analysis.

What data is presented

The View Results screen of the Intel Power Checker wizard provides basic information about the software assessment. The type of processor in your system and the type and model of the power source that was used are given. Four numerical values for each of the three measurement phases are presented. These values are:

• Elapsed Time: The exact number of seconds that each of the phases lasted.
• Energy Consumption: The rate that the battery was discharged during each of the three phases.
• Average C3 State Residency: The percentage of time that the system was in the C3 state during the data collection period.
• Platform Timer Period: The number of milliseconds that the platform timer collected

Typical results would hopefully show a larger percentage of time spent in the C3 State Residency for the application idle time measurement (the middle of the three columns on the View Results screen). As my puzzle solving application was still computing as much as it did in the active execution measurement step, this was not the case for my results. This is atypical for the intended type of applications Intel Power Checker assumes will be measured. Thus, the C3 State Residency values provided by the tool for the idle application were not valid for my particular application.

The name of the report file and the directory to which it will be found are listed on the View Results screen.

Some Caveats

Below are some things you should consider before and during a measurement run using Intel Power Checker.

• Before you start using Intel Power Checker, be sure your chosen workload will run for at least 30 seconds from the point you wish to measure power consumption. In my case, I required a data set that would force the application to run for at least 75 seconds (30 for active measurement, 15 for idle setup, and 30 for idle measurement) plus the time I needed to click boxes and find my application in the process list. Since I ran the application on several different numbers of threads, I needed to be sure that the fastest execution time was still large enough to get all the timings steps completed during a Intel Power Checker run.
• Upon starting Intel Power Checker, the checker may first report that the platform timer period is invalid. In this case, some currently running (background) process has changed the default and it will be up to the user to determine which currently running application has changed the value. Once you have identified the culprit you must stop this process or service before restarting Intel Power Checker. If you are unsure about which active process is preventing Intel Power Checker from starting, you will need to turn off processes one at a time and try Intel Power Checker until the error message doesn’t come up.
• Instructions on the Step 3 screen ask you not to touch the keyboard or mouse. If you are measuring an interactive application or you must interact with the application to generate activity for the full 30 seconds, you will need to touch the keyboard and/or mouse. If possible, a workload that can forego interactivity and still compute for the 30 seconds of measurement time would be best. However, if interaction by the user is part of how the application is utilized, interfacing through peripherals will give you a more accurate measure of the overall energy consumption for typical application usage.
• A data file is created during each phase of the Intel Power Checker assessment to hold the current information. If you cancel the assessment in any of the three phases then a data file will not be created for that phase. After all three phases have been completed, a Tool Report File, in XML format, will be generated containing all of the results. You can find the name of the report file and where it is located on the View Results screen.
• The “Submit Results” button on the View Results screen is optional and only intended for members of the Intel® Software Partner Program to submit their measurement results to the program. If you are not a member, do not submit your results. Simply click on the “Close” button after you have examined the results compiled by Intel Power Checker.

Some Results

The purpose of this article is not to determine the best scenario for running my Akari solver application in the most energy efficient way. You will want to do this for your application, though, and this article has given you the background on Intel Power Checker to determine if this checker can help you quantify the current power consumption of your application. Also, as you make modifications to the application you will be able to determine if those changes improve the energy efficiency or cause your application to suck more power than before.

In addition to the average C3 State Residency percentage, the checker delivers the total number of Joules expended during the 30 seconds of execution time measured. From this I can compute the average Watts for execution parts of the application. I have found that a better metric for comparing different applications or different runs of the same application is milliwatt hours (mWh). You need the total execution time of the execution portion of the application to compute this value. Since Intel Power Checker only measures activity in 30 second segments, you will need to have some timing data available, which I happened to have for the different runs I made of my Akari application.

I found significant differences when running with and without Hyper-Threading Technology (HT) turned on. Also, if the platform was running on battery (DC) power or from the wall socket (AC) power, a difference in execution time and power usage was evident. For example, when running with HT on and a full complement of four threads on the 4 logical cores in my system, I saw the AC power run 1.19X faster that when running the same workload on DC power. However, the former run took 1.15X more power.

Comparing results between runs on DC power versus AC power is a not a good comparison, especially in this case. The power source is detected by the system and the processor is allowed to run with Intel® Turbo Boost Technology at a higher frequency if the platform is using external power. Even so, you may need to be concerned about power consumption of your application in both power source circumstances and you will need to run measurement experiments within each setup to gauge how well your application modifications affect overall power consumption.

System Requirements

You can use Intel Power Checker on a laptop or netbook based on Intel® Core™ processor or Intel® Atom™ processor technology. A desktop with an external power meter or a desktop that is capable of providing the power consumption information can also be analyzed. A Java* Runtime Environment (JRE) (version 6 update 11 or higher) is also required to run the checker. Supported operating systems are Microsoft Windows* XP (Service Pack 3), Microsoft Windows Vista* (Service Pack 2), Microsoft Windows* 7 (Service Pack 1 [32-bit and 64-bit]), and Microsoft Windows* Server 2008 R2.

Download link

To download the Intel Power Checker installation package, go to the following link:

http://software.intel.com/partner/app/software-assessment/. Click on the Intel Power Checker tab to move down to the download link.

Other supporting links

There is a video demonstration of using Intel Power Checker, “A Look at Intel Power Checker,” at the link: http://software.intel.com/en-us/videos/channel/intel-software-partner-program/a-look-at-the-intel-power-checker/1127786023001. Dave Valdovinos and Taylor Kidd, both from Intel, show off the GUI wizard as it measures the power performance of a game-like application.

By Julien Hamaide

Download Article

Download Load Balancing Games (PDF 800KB)

With the increase in processing unit count, load balancing has become a challenge. Constantly loading all units and using all the available processing power requires specifically designed code. Many resources tell you how to organize or optimize your code and data, but few talk about scheduling and load balancing. This article explains how to effectively dispatch units of work to the available CPUs and adjust the game content to the available computing power.

SCHEDULING

Scheduling threads is the operating system’s responsibility. But operating system threads are heavy, and spawning and deleting a thread for each task is overkill. To limit this overhead, games use job systems, which consist of several worker threads that process jobs. The threads are created at the start of the system and are used during the entire game. Jobs can be made of C++ classes or C functions with arguments. Because you don’t use the operating system’s scheduler, however, you must implement a custom scheduler whose role is to dispatch the jobs to all available worker threads. The advantage is that you can set up custom algorithms and adapt them to the game’s needs. The following discussion assumes that the job system implements a dependency manager.

Games are real-time software. For each frame, every system must be updated. Every task must be completed on time or the frame rate will decrease. But not all tasks have the same importance. Tasks can be divided into two categories:

• Required tasks in a frame. These tasks are required to complete the current frame. Physics, animation, rendering, and input processing tasks are necessary for correctly displaying the current frame. The constraint on this type of task is a hard constraint: Any delay directly introduces a delay in the frame.
• Frame-delayed tasks. These tasks are important for the game, but their results are not needed immediately. Pathfinding is a perfect example. If this type of task is delayed, it may not be a problem immediately, because the following frames could absorb the delay. Nonetheless, you must ensure that the task is processed sufficiently early.

The simplest type of scheduling, first-in, first-out (FIFO) schedulers dispatch jobs in the order they were added. Although not sufficient for an entire game, FIFO scheduling can act as a first implementation and reference. This algorithm introduces little overhead: It just picks the last element of a list. That said, it doesn’t try to optimize poor scheduling order, either. Nor does it make any difference between frame-needed jobs and frame-delayed jobs.

One important aspect of a scheduler is its behavior over variable processor count. On gaming consoles, the number of available CPUs is known and doesn’t change over time. On a computer, the story is completely different. The number of CPUs is determined at game startup, and different architectures are available.

If there is only one thread, the scheduler executes tasks one after the other, as if the code were a single thread. Because the overhead is low, performance is similar to a single-threaded version. Because the number of CPUs in computers and consoles is growing, however, the scheduler will have random execution times, depending on the order in which the jobs are pushed and how the critical path is processed.

When the number of processors increases, the scheduler tends to reach the optimal time if it does not introduce latency. Jobs will be processed as soon as they are pushed and their dependencies are resolved.

This scheduler type uses the priority you define to order the jobs. Although based on user input, priority-based scheduling can also be updated to increase performance. User input is kept at minimum granularity, so there is no need to give thousands of priority values. A good set contains these priorities:

• Immediate. Run as soon as a worker thread is ready, or preempt a low-priority thread. This priority is useful if other jobs need this job’s output. For example, an animation could be needed to update the position of a grabbed object.
• Needed for the end of the frame. This priority is set on jobs that are required but not used before the end of the frame—typically particle system or sound system updates.
• Low-priority game updates. This priority is for the kind of task you want to execute as quickly as possible but without limiting higher-priority tasks. Examples include pathfinding and saving data to disk.
• Idle jobs. Those jobs will use idle CPU time. Such jobs can be used to perform background tasks, such as sending statistics to a server or cleaning unused resources.

Unfortunately, some problems will appear. Low-priority and idle jobs might never be triggered if the game overwhelms the CPUs. If the target is 60 frames per second (FPS) and the current CPUs are capable of handling 50 FPS, a new frame will start just after the required jobs are handled. New immediate jobs are directly pushed as the new frame begins, preventing lower-priority jobs from being trigged. To avoid this situation, use a dynamic priority system. To supplement user-level priorities, a second priority value is introduced, the value of which is handled by the scheduler itself. For the low-priority scheduling problem, this value can be increased every time a job misses a launch opportunity. Its priority will then become superior to the priority of immediate jobs. The same problem might appear if an important job depends on a low-priority job. In this case, the priority is transferred to the job on which it depends.

This alternate priority value allows you to sort jobs within a priority category and boost some jobs into the upper category. You can compute the value using different parameters, and you can adapt the scheduler to your needs by tweaking that value. Here are a few of the features that you can use to modify job priority:

• The number of dependent jobs. If a job blocks lot of other jobs, boosting its priority can be a huge win.
• The length of the job. The scheduler can keep statistics about job duration. Longer jobs can be scheduled sooner. If they are scheduled too late, the game might wait for its end with no other jobs to execute.
• Is the job part of the critical path? The critical path is the longer sequence of jobs in the dependency graph. Those jobs must be scheduled with a high priority.

ARCHITECTURE-SPECIFIC PROBLEMS

Architectures can vary among computers. The architecture type, the number of CPUs, the number of cores inside each CPU—everything can affect the low-level behavior.

Typical architectures are symmetric multiprocessor (SMP) and nonuniform memory access (NUMA). In SMP, each CPU has equal access to memory. The memory bus is then most likely the bottleneck. For NUMA, each CPU has its own memory attached. Therefore, it accesses its own memory quickly with its own bus, but accessing other CPUs’ memory can be slow, and latency can be high. Each operating system has its own application programming interface (API) to query system setup. Nothing specific can be done for SMP architectures, but things are different in a NUMA architecture: The scheduler must try to assign jobs on processors that have the data in their local memory. In Windows* operating systems, you can query information about your system using functions such as QueryWorkingSetEx to discover which NUMA node memory is allocated to. Figure 1 shows a four-processor NUMA and SMP architecture.

Figure 1. Typical topology of NUMA and SMP architecture

When a CPU is multicore, each core uses a unique arithmetic logic unit (ALU) and Single Instruction, Multiple Data (SIMD) unit. If two computation-expensive jobs are scheduled on each core of a single CPU, the performance will degrade, so the scheduler should avoid scheduling such jobs on the same CPU. Users can tag jobs to be computation expensive, but those jobs are usually easy to spot. Animations or physics jobs are good examples of computation-expensive jobs.

Code is just data, and as such must be cached by the CPU. For better performance, the scheduler must schedule jobs using the same code on the same processor.

BALANCING THE SYSTEM: ADAPTING TO VARIABLE CPU COUNTS

If your game targets a wide range of machines, prepare it for multiple resolutions. For example, particle systems can be up-scaled, spawning more particles. Pathfinding can output a smoother path. Animation can use more bones for skeletons. Your game can increase its quality, using all the extra power it can. However, it should also be able to work on low-end machines. Players are used to tweaking the 3D settings—resolution, texture size, shadow definition, and so on—but it would be awkward for a player to tweak the artificial intelligence (AI) update rate or the pathfinder precision. The game must be able to adapt itself to the available power. But how do you prevent the game from running out of resources? What do you sacrifice first?

Think of the AI system in three parts: seeing, thinking, and acting.

For an AI-driven character, seeing is acquiring all data needed to make a decision, thinking is combining all this data and making decisions, and acting is applying the decision. These three parts might not be updated at the same frequency. The acting layer is frame-required: The character must do something every frame, the new animation pose must be computed, and so on. But for the other two layers, the requirement is lower. There’s no need to update the sensory data each frame, nor does the current decision need to be challenged with the current data. But how do you determine the system’s update frequency?

One solution is to let the scheduler decide. The sensory system creates a first job with a low priority. If processing power is available, the scheduler will proceed with the job in the current frame. If not, the priority will be increased over time. Some frames later, the job will be finally executed. The trick is to schedule the next update within the current job (with some delay to prevent double-updating in the same frame). Using this technique, the system is updated as often as possible but no more. If the update rate is too low, the system can use extrapolation, such as is done in network play.

To implement such a system, two techniques are available:

• Blackboards. Variables such as current decision and first visible enemy are continuous values. They could be updated at each frame. A blackboard allows you to keep the latest value of such variables. When an update job is executed, it writes its result to the blackboard but not instantaneously. The blackboard accumulates all changes during the frame and applies them at the end. The other jobs then never read inconsistent values, as variables don’t change during a frame.
• Future objects. Requests such as pathfinding are less frequent and usually take more time to process. The use of future objects can help in such cases. The future object allows you to send a request and wait for the result. If the result is not yet available, the code falls back to another behavior. Future objects can contain intermediate results, such as first path segments.

The main advantage of the component system is its modularity. Each entity is made of a component, each with its own data and executing its own task. Cutting the entities’ update code into pieces eases calling them at different frequencies. If your component system is data-oriented, different tables store each type of component, which are then updated in a row. For a given component type, only half of the table can be updated, thus dividing the frequency by two.

Take particular care about the time step: It should be accumulated over the skipped frame. To compute this step, the system keeps the last time steps, and each component stores the frame index it was last updated. Using the current frame index, the system can add time steps from skipped frames.

For the sound system, the story is completely different. The update is strict: The game must send its audio data on time or a glitch will be heard. You cannot balance by changing the update frequency. Instead, the idea here is to limit the number of sounds that can be played simultaneously. It’s easy for the scheduler to know what percentage of CPU the sound jobs are using, so the user can decide to limit this percentage to a fixed value. The sound system can then adapt its simultaneous play count using this value. If you use Digital Signal Processing (DSP) effects, the system can limit its simultaneous instances.

As most physics systems’ algorithms are based on an integration technique, the time step must be sufficiently small and constant. Some systems require you to break the time step into smaller pieces, and the update runs several times a frame. Users can't decrease the update rate, so the only solution is to decrease the need for CPU time. Several solutions exist:

• If your gameplay does not depend on the physics system’s behavior, you can change your collision resolution algorithm. A discrete collision algorithm is less precise than continuous collision detection, but it is also less expensive. Changing this algorithm at runtime can be tricky depending on your framework. Generally, this is a start time decision.
• Lower the distance at which objects are removed from the physical world.
• Lower the distance at which objects do not interact with other objects but only with static geometry. Most physics systems assign two 32-bit integers to their objects. Those bits are considered layers. The first layer defines which layers of an object belong to the collision group; the second layer determines which layers an object intersects with in the collision mask. Use this system to limit collisions between objects when they are far from the camera. The test is fast and easy to set up, and you can update the bits depending on the object’s distance from the camera.

Because the first technique is only available at start time, use a heuristic to determine whether to activate it. The other solutions can be enabled at runtime.

The streaming system is responsible for loading data from disk. If the data is compressed, the system will consume some CPU cycles, but you must pay attention to the impact this system can have on the frame rate. A good solution is to use a low priority for the streaming thread. But as in priority-based scheduling, “starving” may occur.

A huge difference between game consoles and computers is the existence of foreign processes. When users launch a game, they rarely close all running applications or shut down the operating system’s services. Some processes might wake up and perform some background work. When load balancing your game, you should always keep a bit of security in your timing measurements. On Windows* machines, GetProcessTimes and GetSystemTimes return the times (idle, user, and kernel) of the game process and the whole system. The application can sample those values and extract the time spent in other processes. The mean value of the recent peak is used as a security. On a six-processor machine with a game running at 60 FPS, the total available CPU time during a frame is 6 CPU * 1/60 FPS = 100 ms. If other processes have peaks around 20 ms per frame, the game should limit itself to 80 ms, even if in 90 percent of frames the other processes use less than 5 ms. As explained later, the load balancer integrates this constraint in its design.

THE LOAD BALANCER

You’ve seen different techniques for lowering the required CPU power. But when should the game enable them and in which order? The load balancer is responsible for activating each system in turn. The idea is to degrade or progressively improve each system in turn. Unfortunately, there is no unique solution: The answer really depends on the importance of each system in your game experience. A mood game will surely put more importance on music than a first-person shooter will. So instead of explaining a finished and tuned system, let’s explore the concept and how to tune your game to your needs.

The main load balancing criterion is the frame length (equivalently, the frame rate). The goal is to keep the highest constant frame rate. Even if the game can run at 40 FPS 90 percent of the time and 30 FPS for the last 10 percent, it might be more interesting to keep the game at 30 FPS. Changes in frame rate are more noticeable than a slightly lower frame rate. And keeping a slightly lower frame rate frees some CPU time for streaming and the unexpected.

At the end of each frame, the load balancer receives information about the last frame length and job lengths, then compares this time with previous frame lengths. Table 1 contains the frame length for a given scenario. In this example, the load balancer requires that the update rate be around 50 Hz. The load balancer maintains the list of level of detail (LOD) it can enable or disable (see Table 2). The time each LOD takes is computed at the start of the application and is used as an approximation of the time the tasks required to execute. Every time an LOD is enabled, its time is updated (using a moving mean). When the load balancer detects that the frame length has decreased several frames in a row, it inspects its lists and activates the next LODs. You provide this list, as it depends on what constitutes the more important details you want to keep. The load balancer will always activate those details in order. But when should the game start decreasing the frame rate?

The load balancer can access other information in the list, as well: the minimum frame rate required to disable a feature. In this example, the developer decided that it’s more important to have precise pathfinding than to have a game at 50 Hz. But if the frame rate were to decrease below 30 FPS, the feature could be disabled. At startup, the load balancer targets a high frame rate but runs at the lowest LOD. The system adds a new LOD each frame until the system finds its equilibrium, preventing the system from stalling the first few frames.

This technique allows a precise definition of the degradation of the game. Once the system is tuned, it is fully automatic, maintaining the influence of each LOD on the frame length. The frame rate is also linked to LOD selection. Although the design and the code are simple, the results are good. To tweak the system, an edit-and-continue rule system can come in handy. The ability to dynamically simulate lower-end machines (by limiting the number of CPUs) is also important. With those tools in hand, the rule list can be created and tuned in hours. This system was used in an unreleased game and is currently used in a game that will be released in 2012.

CONCLUSION

The scheduler is a complex system. As the conductor, it should be adapted to the score your game is playing. There is no unique solution, but you should select the best algorithm based on the code and data statistics. The algorithm should also be able to adapt to different architecture types. The key to success is analyzing, adapting, and profiling. But a scheduler alone won’t solve the problem. The load balancer is there to finish the work. It serves as an LOD coordinator, linking definition and quality to frame rate. You then have everything you need to shape the best experience for a wide range of machines.

About the Author

Julien Hamaide graduated as a multimedia electrical engineer at the Faculté Polytechnique de Mons (Belgium) at the age of 21. After two years of working on speech and image processing at TCTS/Multitel and three years leading a team on next-generation consoles at Elsewhere Entertainment, Julien started his own studio, called Fishing Cactus (www.fishingcactus.com). He has published several articles and spoken about such topics as 10Tacle's movement and interaction system and multithreading applied to AI. He now leads development at Fishing Cactus.

RESOURCES

Amdahl, G. (1967). “Validity of the Single Processor Approach to Achieving Large-Scale Computing Capabilities.” AFIPS Conference Proceedings (30):483–485.

Alexandrescu, A. (2001). Modern C++ design: generic programming and design patterns applied.

Hamaide, J. (2008). “Multithread Job and Dependency System.” Game Programming Gems 7.

Leiserson, C. What the $#@! is Parallelism, Anyhow? http://software.intel.com/en-us/articles/what-the-is-parallelism-anyhow-1.

Leiserson, C. Are Determinacy-Race Bugs Lurking in YOUR Multicore Application? http://software.intel.com/en-us/articles/are-determinacy-race-bugs-lurking-in-your-multicore-application.

Multiple Processors. http://msdn.microsoft.com/en-us/library/ms684251.aspx.

Introduction

Efficient antialiasing techniques are an important tool of high-quality, real-time rendering. MSAA (Multisample Antialiasing) is the standard technique in use today, but comes with some serious disadvantages:

• Incompatibility with deferred lighting, which is used more and more in real-time rendering;
• High memory and processing cost, which makes its use prohibitive on some widely available platforms (such as the Sony Playstation* PS3* [Perthuis 2010]). This cost is also directly linked to the complexity of the scene rendered;
• Inability to smooth non-geometric edges unless used in conjunction with alpha to coverage.

A new technique developed by Intel Labs called Morphological Antialiasing (MLAA) [Reshetov 2009] addresses these limitations. MLAA is an image-based, post-process filtering technique which identifies discontinuity patterns and blends colors in the neighborhood of these patterns to perform effective antialiasing. It is the precursor of a new generation of real-time antialiasing techniques that rival MSAA [Jimenez et al., 2011] [SIGGRAPH 2011].

This sample is based on the original, CPU-based MLAA implementation provided by Reshetov, with improvements to greatly increase performance. The improvements are:

• Integration of a new, efficient, easy-to-use tasking system implemented on top of Intel® Threading Building Blocks (Intel® TBB).
• Integration of a new, efficient, easy to use pipelining system for CPU onloading of graphics tasks.
• Improvement of data access patterns through a new transposing pass.
• Increased use of Intel® SSE instructions to optimize discontinuities detection and color blending.

The MLAA algorithm

In this section we present an overview of how the MLAA algorithm works; cf. [Reshetov 2009] for the fully detailed explanation. Conceptually, MLAA processes the image buffer in three steps:

1. Find discontinuities between pixels in a given image.
2. Identify U-shaped, Z-shaped, L-shaped patterns.
3. Blend colors in the neighborhood of these patterns.

The first step (find discontinuities) is implemented by comparing each pixel to its neighbors. Horizontal discontinuities are identified by comparing a pixel to its bottom neighbor, and vertical discontinuities by comparing with the neighbor on the right. In our implementation we compare color values, but any other approach that suits the application’s specificities is perfectly valid.

At the end of the first step, each pixel is marked with the horizontal discontinuity flag and/or the vertical discontinuity flag, if such discontinuities are detected. In the next step, we “walk” the marked pixels to identify discontinuity lines (sequences of consecutive pixels marked with the same discontinuity flag), and determine how they combine to form L-shaped patterns, as illustrated below:

Figure 1: MLAA processing of an image, with Z, U, and L-shapes shown on the original image on the left

The third and final step is to perform blending for each of the identified L-shaped patterns.

The general idea is to connect the middle point of the primary segment of the L-shape (horizontal green line in the figure below) to the middle point of the secondary segment (vertical green line—the connection line is in red). The connection line splits each pixel into two trapezoids; for each pixel the area of the corresponding trapezoid determines the blending weights. For example, in the figure below, we see that the trapezoid’s area for pixel c5 is 1/3; so the new color of c5 will be computed as 2/3 * (color of c5) + 1/3 * (color of c5’s bottom neighbor).

Figure 2: Computing blending weights

In practice, to ensure a smooth silhouette look, we need to minimize the color differences at the stitching positions of consecutive L-shapes. To achieve this, we slightly adjust the connection points on the L-shape segments around the middle position based on the colors of the pixels around the stitching point.

Sample usage

The camera can be moved around the scene using drag-and-click, and the mouse wheel can be used to zoom in or out. In addition, there are three blocks of controls on the right side of the sample’s window:

Figure 3: A screenshot of the sample in action

The first block controls the rendering of the scene:

• Pause Scene toggles the scene’s animations on/off;
• Show zoom box toggles the zoom box feature on/off, which allows the user to take a closer look at a pixel area to better compare the antialiasing techniques. The area of interest can be changed by right-clicking to the new area to examine;
• Scene complexity simulates the effect of increasing scene complexity by using overdraw. The value (between 1 and 100, adjusted by the slider) indicates how many times the scene is rendered per frame (with overdraw).

The second block selects which antialiasing technique to apply: MLAA, MSAA (4x) or no antialiasing. This is of course to allow comparison of the techniques’ performance and quality (the default choice is “MLAA”);

The last block of controls is only available if the antialiasing technique used is MLAA, and controls how the algorithm should be run:

• Copy/Map/Unmap only copies the color buffer from GPU memory to CPU memory and back, but won’t perform any MLAA processing between the two copy operations. This allows measurement of the impact of the copy operations on the overall performance of the entire algorithm;
• CPU tasks pipelining turns the pipelining system on/off for CPU onloading of graphics tasks (the default is on) so that it is easy to see the benefit of pipelining;
• Show found edges runs the first part of the algorithm (find discontinuities between pixels), but the blending passes are replaced by a debug pass, where a pixel is:
  • changed to solid green if a horizontal discontinuity has been found with its neighbor;
  • to solid blue if a vertical discontinuity has been found with its neighbor;
  • changed to solid red if both horizontal and vertical discontinuities have been found with its neighbors;
  • unchanged if no discontinuities have been found.

Figure 4. Gear tower scene with MLAA and the ”show found edges” option enabled

Sample Architecture

Without the pipelining optimizations, the sequence of events for each frame is:

1. ANIMATE AND RENDER TEST SCENE
2. MLAA STAGE (if MLAA is enabled)

   • Copy the color buffer where the scene was rendered to a staging buffer
   • Map the staging buffer for CPU-side access.
   • MLAA post-processing (the staging buffer is both input and output)
   • Unmap the staging buffer
   • Copy staging buffer back to the GPU-side color buffer
   • Render the zoombox (if applicable)
3. RENDER THE SAMPLE’s UI, PRESENT FRAME

Except for the “Perform the MLAA post-processing work” step (and again not considering the pipeline for now), all of these steps are implemented using standard Microsoft DirectX* methods.

The Tasking API

By nature the MLAA algorithm is easy to parallelize. For both the discontinuities detection and blending passes, we can process the color buffer in independent chunks (blocks of contiguous rows or columns). MLAA can be implemented using a task-based solution that automatically makes full use of all available CPU cores, while keeping the code core count agnostic

This sample uses a simple C-based tasking API that is implemented on top of the Intel® Threading Building Blocks (Intel® TBB) scheduler. This wrapper API was created to simplify the integration of the technique into existing codebases which already expose a similar tasking API (e.g. cross-platform game engines). An added benefit is the increased readability of the main source file MLAA.cpp.

The two important functions of the wrapper APIs are:

    BOOL CreateTaskSet(
        TASKSETFUNC                 pFunc,      //  Function pointer to the 
                                                //  Taskset callback function
        VOID*                       pArg,       //  App data pointer (can be NULL)
        UINT                        uTaskCount, //  Number of tasks to create 
        TASKSETHANDLE*              pDepends,   //  Array of TASKSETHANDLEs that 
                                                //  this taskset depends on.  The 
                                                //  taskset will not be scheduled
                                                //  until all tasksets in this list
                                                //  complete.
        UINT                        uDepends,   //  Count of the depends list
        OPTIONAL LPCSTR             szSetName,  //  [Optional] name of the taskset
                                                //  the name is used for profiling
        OUT TASKSETHANDLE*          pOutHandle  //  [Out] Handle to the new taskset
        );

And:

    //  WaitForSet will yield the main thread to the tasking system and return
    //  only when the taskset specified has completed execution.
    VOID WaitForSet(
        TASKSETHANDLE               hSet        // Taskset to wait for completion
        );

The callback function has the following signature:

typedef VOID (*TASKSETFUNC)(VOID* TaskInfo, INT iContext, UINT uTaskId, UINT uTaskCount);

MLAA requires a dependency graph of three consecutive tasksets:

• The first taskset finds the discontinuities between pixels in the color buffer;
• The second taskset performs the horizontal blending pass, and depends on the completion of the first taskset for the discontinuities information;
• The third taskset performs the vertical blending pass, which depends on the completion of the first taskset for the discontinuities information, but also on the completion of the second taskset because of the transpose optimization (cf. corresponding section for details).

This dependency graph is expressed in MLAA.cpp as a sequence of three calls to CreateTaskSet; the taskset callback functions (implemented in MLAAPostProcess.cpp) being respectively MLAAFindDiscontinuitiesTask, MLAABlendHTask, MLAABlendVTask.

If pipelining is not enabled, we wait for the last taskset to complete by calling WaitForSet with the handle of the last taskset. When the call returns, the MLAA work for the frame is complete. Things are just a little bit more complicated when using the pipeline.

The CPU/GPU pipeline

To get the maximum performance from our implementation, we have to keep both CPU and GPU sides fully utilized. Due to the data dependencies between the tasksets, full utilization can be achieved by interleaving the processing of multiple frames, as shown in the diagram below:

Figure 5: Frames moving through the pipeline. The red blocks on the CPU main thread timeline illustrate that the main thread will take MLAA work if the application gets CPU-bound.

In other words, we have to build a pipelining system, where a pipeline is a sequence of CPU stages (workloads) and GPU stages, and able to run multiple instances of said pipeline at the same time.

In our case, each instance of the pipeline corresponds to the processing of one frame. We have three steps:

Step 1 (GPU stage):

• Animate and render test scene;
• Copy color buffer to staging buffer (using asynchronous GPU-side CopyResource).

Step 2 (CPU stage):

• Map the staging buffer;
• Perform the MLAA post-processing work (staging buffer is both input and output).

Step 3 (GPU stage):

• Unmap staging buffer and copy it back to the GPU-side color buffer;
• Finish frame rendering (zoombox, UI) and present.

To implement this concept, we designed a simple Pipeline class. Each stage is represented by a PipelineFunction structure that specifies the function to be called, and the stage type. The callback function must have the following signature:

typedef void (*PPIPELINEFUNC)( UINT uInstance, ID3D11DeviceContext* pContext )

Where uInstance indicates which instance of the pipeline the function is being called from. A GPU stage uses a DirectX* query (of type D3D11_QUERY_EVENT) to signal its completion, but a CPU stage has to explicitly call the Pipeline’s class CompleteCPUWait method to signal completion:

    //  CompleteCPUWait signals an instance that expects an CPU wait in order
    //  to complete. When the Pipeline reaches that instance, it will be complete
    //  for that stage in the pipeline.
    VOID
    CompleteCPUWait(
        UINT                    uInstance );

Creating the pipeline instances requires a single call to the Init method. In our case, the code is:

	PipelineFunction Func[ 2 ];
	Func[0].Type = PIPELINE_EVENT_COMPLETE;
	Func[0].pFunc = AnimateAndRenderScene;
	Func[1].Type = PIPELINE_CPU_COMPLETE;
	Func[1].pFunc = MLAAPostProcessing;

	g_Pipeline.Init(g_NumPipelineInstances, ARRAYSIZE( Func ), Func, g_pPipelineQueries );

Where g_NumPipelineInstances is the number of pipeline instances to create (3 in this case).

To run the pipelines, we call the Start method each frame:

    //  Start is used to execute the Pipeline.  Processing will continue until
    //  an instance in the pipeline completes.  The index of that instance is
    //  return.  PIPELINE_INVALID is returned if the Pipeline was set to flush
    //  and there are no more instances left in the Pipeline.
    UINT
    Start(
        ID3D11DeviceContext*    pContext,       //  Context to issue Event query on.
        
        BOOL                    bFlush = FALSE  //  If bFlush is TRUE, the Pipeline
                                                //  will not create new instances.
                                                //  Specify TRUE if you want to flush
                                                //  the Pipeline.
        );

Because the Start method returns the index of a completed instance of the pipeline, the third and last step doesn’t have to be called through the Pipeline class. The code of the last step executes right after the call to the Start method, indexing the data structures with the index returned by Start.

Pipelining does not rely on the tasking API WaitForSet call, since it is blocking and so does not allow pipelining to occur. The solution is to use a completion taskset—that is, a task that depends on the completion of all the MLAA tasks, and whose only work will be to call the CompleteCPUWait method.

Intel® SSE optimizations

The first pass of the MLAA algorithm identifies discontinuities between pixels. Conceptually, each pixel checks its color and compares it with its bottom neighbor (when looking for horizontal discontinuities) or right neighbor (when looking for vertical discontinuities) [Reshetov09, section 2.2.1].

In this sample, we have kept the simple discontinuity detection kernel of the reference implementation. A discontinuity exists between two pixels if at least one of their RGB color components differs by at least 16 (on the 0-255 scale of the RGBA8 format).

This definition works well in the sample and allows a very compact and efficient SIMD implementation. However, more complex approaches are possible, and sometimes necessary, to get the best possible results. For example, a pixel’s luminance could be used instead of directly comparing color values, and a variable threshold recomputed each frame to take into account the scene’s overall luminance and contrast [Luminance]. Depth values could be used to assist with edge detection or any kind of custom data to exclude specific zones of the color buffer from being processed.

Because this step is independent from the rest of the algorithm, and we are running on the CPU, any data from the program can be used directly as input data. The only limits to the detection algorithm are:

• The fact that the algorithm must output a bit for each pixel indicating whether a discontinuity is detected;
• The tradeoff between performance and quality/complexity;
• The programmer’s imagination.

As with the original reference implementation [Reshetov09, section 2.4], we work with a RGBA8 render target format and the “vertical discontinuity” and “horizontal discontinuity” bit flags computed by MLAA are stored in-place in the two high bits of the pixel data (i.e., the two high bits of the alpha component which is unaffected by the MLAA blending operations). This keeps the implementation simple while helping with memory footprint, and allowing optimizations in the next steps of the algorithms (cf. “the transposing optimization” section below).

Because each pixel is 32 bits of data, and each pixel can be processed independently of the others in this step, we can process 4 pixels at a time using Intel® SSE intrinsics. The detection code is very short.

	
	//---------------------------------------------------------------------------------------
	// Given a row of pixels, check for color discontinuities between a pixel and its
	// neighbor. Intel SSE implementation, so we process all 4 pixels in the row at a time.
	// If a discontinuity is detected, the corresponding flag is set in the alpha component of the pixel.
	//---------------------------------------------------------------------------------------
	inline void ComparePixelsSSE(__m128i& vPixels0, __m128i vPixels1, const INT EdgeFlag)
	{
	// Each byte of vDiff is the absolute difference between a pixel's color channel value, and the one from its neighbor.
		__m128i vDiff = _mm_sub_epi8(_mm_max_epu8(vPixels0, vPixels1), 
									 _mm_min_epu8(vPixels0, vPixels1));
	// We are only interested if the diff. is >16, and not interested in alpha differences.
		const INT Threshold = 0x00F0F0F0;
		__m128i vThresholdMask = _mm_set1_epi32(Threshold);
		vDiff = _mm_and_si128(vDiff, vThresholdMask);
	// Each of the four lanes of the selector is 0 if no discontinuity detected RGB, 0xFFFFFFFF else.
		__m128i vSelector = _mm_cmpeq_epi32(vDiff, _mm_setzero_si128());

The reference implementation then proceeds to update the alpha component of each pixel, but used inefficient serial code to do so. We can optimize this sequence by using the following Intel® SSE code:

	
	__m128i vEdgeFlag = _mm_set1_epi32(EdgeFlag);
	vEdgeFlag = _mm_andnot_si128(vSelector, vEdgeFlag);
	// vEdgeFlag now contains EdgeFlag for the pixels where a discontinuity was detected, 0 otherwise.
	vPixels0 = _mm_or_si128(vPixels0, vEdgeFlag);

We also replaced the MixColors function (which computes a linear interpolation of two colors) to use a full Intel® SSE implementation.

The Transposing Optimization

The next task of the algorithm is to find “discontinuity lines”, i.e., sequences of consecutive pixels which are marked with the same discontinuity flag (horizontal flag for horizontal blending pass, vertical flag for vertical blending pass) while walking rows and columns of our color buffer [Reshetov09, section 2.1].

Because discontinuity lines tend to be short, intuition suggests (and profiling data confirms) that the most expensive part of this operation is scanning between discontinuity lines, i.e., scanning the large areas of consecutive pixels where no discontinuity flag is set.

The good news is that we can check 4 pixels at a time using the _mm_movemask_ps Intel® SSE intrinsic when the following conditions are met:

1. We are scanning 4 pixels stored at consecutive addresses;
2. The address of the first pixel is “Intel® SSE-aligned” (16 bytes alignment);
3. The discontinuity flag is stored as the high bit of the 32 bits of pixel data.

During the horizontal blending pass, (1) is true (we scan horizontal rows of pixels in the color buffer represented as a 2D linear array of pixel data); (2) is true almost all the time (remember that 16 bytes alignment is equivalent to “the index of the starting pixel in the buffer is a multiple of 4” as the buffer is properly aligned); and (3) is true as we chose bit 31 to represent the horizontal discontinuity flag.

If all conditions are true, we compute the flags:

	__m128 PixelFlags = *(reinterpret_cast<__m128*>(WorkBuffer + OffsetCurrentPixel));
	// Creates a 4-bit mask from the most significant bits
	int HFlags = _mm_movemask_ps(PixelFlags);	

And five outcomes are possible depending on the value of HFlags:

• 0 (the most common case by far): no discontinuity flag set in this group of 4 pixels, move to the next group of 4 pixels.
• Bit 0 is set: discontinuity line starts at first pixel of the group.
• Bit 1 is set: discontinuity line starts at second pixel of the group.
• Bit 2 is set: discontinuity line starts at third pixel of the group.
• Bit 3 is set: discontinuity line starts at fourth pixel of the group.

This optimization is part of the reference implementation and works well for the horizontal blending pass, but was impossible to apply for the vertical blending pass. As the code scans the buffer vertically, (1) is false (adjacent pixels in a column are not stored at consecutive addresses), and (3) is false as well (the vertical discontinuity flag is stored at bit 30 of the pixel data).

The problem with (3) is easy to work around by introducing a simple “shift left by one bit” operation if we are processing the vertical flags, transforming the code above to:

	int Shift = (EdgeFlag == EdgeFlagV) ? 1 : 0;
	__m128 PixelFlags = *(reinterpret_cast<__m128*>(WorkBuffer + OffsetCurrentPixel));
	PixelFlags = _mm_castsi128_ps( _mm_slli_epi32(_mm_castps_si128(PixelFlags), Shift) );
	int HFlags = _mm_movemask_ps(PixelFlags);

But (1) is still a problem. In addition, vertical scanning is extremely cache-unfriendly. Because of both of these issues, the vertical blending pass is about 3 times (300%!) as expensive as the horizontal blending pass in the reference implementation (as shown by profiling data).

Our solution to this issue is to make the vertical blending pass use the cache and Intel® SSE-friendly data access patterns of the horizontal blending pass by considering the color buffer as a matrix of pixels and transposing it between passes:

• Perform horizontal blending pass
• Transpose the (horizontally blended) color buffer
• Perform vertical blending pass
• Transpose back color buffer

This way the code for both blending passes is exactly the same (which adds the advantage of simplicity/readability), the only difference being which flag to scan for, and we benefit from all the optimizations and cache-friendliness of the horizontal pass.

In practice, the transpose operations are not implemented as separate passes/tasksets, but as the last part of their respective blending pass. This allows us to benefit from the cache “warmness”.

The two transpose operations are not free. The cost is the extra code execution time, one extra work buffer and a synchronization point between the horizontal and vertical passes (we must wait for all horizontal tasks to be done before we can start any of the vertical tasks, because we must wait for the color buffer to be fully transposed before starting the vertical pass work).

Even with these extra costs, the overall performance is significantly better than the previous approach. As expected, profiling data shows that both blending passes have equivalent performance.

Performance Results

MLAA performance was measured on the following two configurations:

• Code name “Huron River” : Intel® Core™ i7-2820QM Processor (Intel® microarchitecture code name Sandy Bridge, 4 cores 8 threads @2.30 GHz) with GT2 processor graphics, 4 GB of RAM, Windows 7 Ultimate 64-bit Service Pack 1
• Code name “Kings Creek”: Intel® Core™ i5-660 Processor (codename “Clarkdale”, 2 cores 4 threads @ 3.33 Ghz), with GMA HD Graphics (codename “Ironlake”), 2 GB of RAM, Windows 7 Ultimate 64-bit Service Pack 1

We measured the average frame rendering time of our sample as a function of scene complexity for the different antialiasing settings. We also measured the rendering time for the Copy/Map/Umap only mode to highlight the impact of the color buffer copy operations on the overall performance of the algorithm.

Results

The data shows that for the Huron River machine, the extra cost of MSAA 4x goes up linearly with the scene complexity (in fact, for all resolutions, the frame time when using MSAA 4x to render our test scene appears to be approximately the double of the frame time when no antialiasing is used). In contrast, the cost of MLAA appears more or less constant (around 4 ms/frame at 1280x800). This is consistent with our expectations, as unlike MSAA 4x, MLAA is executed only once per frame, regardless of scene complexity / number of draw calls.

We also observe that except for very low complexity values (i.e. less than ~5) MLAA always outperforms MSAA 4x, regardless of resolution, and that the difference in performance grows with complexity (because as noted above, the cost of MSAA 4x grows linearly with complexity when the cost of MLAA does not).

In the case of the Kings Creek machine, we can’t compare the cost of MLAA vs. MSAA 4x as the latter is not provided by the Ironlake hardware. The goal is then to determine if MLAA allows us to provide software antialiasing as an alternative with acceptable performance. Our measurements at the 1280x800 resolution show that again the cost of MLAA is largely independent of scene complexity, and the average value is ~7.5 ms (discarding the outlier data point at complexity = 100).

Interestingly, if we compare this result to a hypothetical MSAA 4x implementation with approximately the same performance profile than the Huron River one (frame time with MSAA 4x ~ 2x frame time with no antialiasing), we notice that again MLAA would outperform MSAA 4x for almost all complexity values (≥4 in this case).

Table 1. Rendering times of our test scene on the King’s Creek machine.

Table 2. Rendering times of our test scene on the Huron River machine.

Figure 6: Rendering times of our test scene for each of the antialiasing techniques, as a function of scene complexity. The bottom, flat curve is the difference between the “MLAA with pipeline on” curve, and the “No antialiasing” curve, measuring the cost of using MLAA [Huron River, res. 1280x800]

Figure 7: Rendering times of our test scene for each of the antialiasing techniques, as a function of scene complexity. The bottom, flat curve is the difference between the “MLAA with pipeline on” curve, and the “No anti aliasing” curve, measuring the cost of using MLAA [Huron River, res. 1600x1200]

Figure 8: Rendering times of our test scene with MLAA on and off, as a function of scene complexity. The bottom, flat curve is the difference between the “MLAA with pipeline on” curve, and the “No antialiasing” curve, measuring the cost of using MLAA [Kings Creek, res. 1280x800]

References

[Reshetov 2009] RESHETOV, A. 2009. “Morphological Antialiasing”

[Perthuis 2010] PERTHUIS, C. 2010. MLAA in God of War 3. Sony Computer Entertainment America, PS3 Devcon, Santa Clara, July 2010.

[Jimenez et al., 2011] JIMENEZ, J., MASIA, B., ECHEVARRIA, J., NAVARRO, F. and GUTIERREZ, D. 2011. Practical Morphological Anti-Aliasing. In Wolfgang Engel, ed., GPU Pro 2. AK Peters Ltd.

[SIGGRAPH 2011] JIMENEZ, J., GUTIERREZ D., YANG, J., RESHETOV, A., DEMOREUILLE, P., BERGHOFF, T., PERTHUIS, C., YU, H., MCGUIRE, M., LOTTES, T., MALAN, H., PERSSON, E., ANDREEV, D. and SOUSA T. 2011. Filtering Approaches for Real-Time Anti-Aliasing. In ACM SIGGRAPH 2011 Courses.

[Luminance] Definition of luminance for CRT-like devices:

INTERNATIONAL COMMISSION ON ILLUMINATION. 1971. Recommendations on Uniform Color Spaces, Color Difference Equations, Psychometric Color Terms. Supplement No.2 to CIE publication No. 15 (E.-1.3.1), TC-1.3, 1971.

And for LCDs:

ITU-R Rec. BT.709-5. 2008. Page 18, items 1.3 and 1.4

Download MAXIS-mizing Darkspore Game Performance with Intel® GPA 4.0 (PDF 3.4MB)

Contents

• A New Gaming Experience Made Possible With Processor Graphics
• Darkspore* is Designed for Everyone
• Darkspore* Rendering
• Hot Loading Shaders in Frame Analyzer
• They Drew First (Deferred) Blood
• Other Optimizations for Darkspore*
• Conclusion
• References
• Authors

A New Gaming Experience Made Possible With Processor Graphics

Released in early 2011, the 2nd Generation Intel® Core™ processors have fundamentally changed the PC gaming landscape, with processor and graphics merged into a single piece of silicon. The tighter integration of graphics into the processor has revolutionized gaming performance and made your favorite games run and look great on mainstream machines. As a developer, the market for your games has opened up immensely.

Figure 1: PC Volumes Dwarf Consoles through 2014

Targeting the mainstream is possible and Intel provides you the tools to help achieve success in this market. Intel® Graphics Performance Analyzers (Intel® GPA) 4.0 was released at GDC 2011 with full support for this next generation of processors. Game developers including the Darkspore* team at Maxis* have taken full advantage of Intel® GPA 4.0 features to make sure their game fully utilizes the new processor graphics.

Darkspore* is Designed for Everyone

Darkspore* is best described as an online action RPG where you control a squad of three heroes to fight the Darkspore*, an evil infestation of creatures. The game includes a version of the award winning Spore* Editor where you can fully customize not just basic features of the characters like color, height, textures, but vertices in the characters’ meshes. With Darkspore*, Maxis* is targeting mainstream to high end graphics hardware. At the Game Developers Conference* 2011, David Lee Swenson, the Lead Engineer on Darkspore*, presented how he used Intel® GPA 4.0 to help achieve this goal.

Darkspore* Rendering

For Darkspore*, Maxis* implemented a variation of the light pre-pass renderer described by [Engel 2008]. One main advantage of using a light pre-pass renderer is to decouple lighting from the scene geometry. In a forward renderer, lighting is computed at the time the object is drawn. In a light pre-pass renderer, light is accumulated in an off screen buffer and either applied in a post pass or sampled in a final pass per object. The Darkspore* renderer is composed of three main passes: deferred pass, lighting pass, and final pass. The deferred pass saves the material data for opaque objects in the scene, the light pass saves all lighting calculations into a light buffer, and the final pass uses the results of the previous passes to create the final frame. The deferred pass uses 2 render targets. The first render target is composed of world normals with a gloss term: Normals (RGB) + Gloss (A).

Figure 2: First render target = world space normals (RGB) + gloss (A)

Depth ([R*256]G), Specular Power (B), and a Toon ID (A) for a toon-style character outline are rendered to the second target of the deferred pass.

Figure 3: Second render target = Depth ([R*256]G) + Specular Power (B) + ToonID (A)

Following the deferred pass, there’s a lighting pass that renders up to 6 parallel lights as well as cloud shadows and the main shadow. Then, each point or spot light are rendered with gobos and/or shadows to a 16F light buffer: Diffuse (RGB) + Specular (A).

Figure 4: Light buffer = Diffuse (RGB) + Specular (A)

During the final pass, each object is rendered again sampling the light buffer. Values for areas intended to glow are written to a second target. Then, the glow target is downsampled and blurred back together with particles and post-processing effects like fog, distortion, and the death effect.

Figure 5: Final pass = Color + Glow + Post FX + Particles + UI

At this point all it needs is the UI and the final frame is complete.

Figure 6: The final frame

Darkspore* and Intel® GPA

Before Intel® GPA, the Darkspore* team at Maxis* was using a long list of tests and settings to understand and debug game performance. With Intel® GPA, the game can be run with the applicable command line options. Once a frame is captured, it’s just a matter of setting the X and Y axis to the appropriate metrics to get a useful profile of performance.

Figure 7: Darkspore* with command line options

Since Darkspore* was known to be pixel shader bound, the Darkspore* team typically would set the X axis to the “PS Duration” metric and the Y axis to “GPU Duration”. With these metrics, the taller the bar, the more GPU time it is taking and the wider the bar the more pixel shader time.

Figure 8: Initial performance graph with four expensive calls

The four calls labeled above (A, B, C, D) are the most expensive in this frame capture. Calls A and C correspond to the deferred and final pass for the blood decals. Call B corresponds to the parallel lights, cloud shadows, and shadows. Call D runs the edge detection that was originally a bigger part of the game but has since been moved to the high spec configuration. Looking at the shaders for the blood decal calls (A and C), a few issues were found and fixed by the Darkspore* team:

• Tiling of decals was supported but never used
• Vectors were normalized that were used only for a cubemap lookup
• A Fresnel term was adding very little to the scene, given the fixed camera angle
• Alpha test was implemented with both a clip instruction and blending
• Normal calculation was overly complex with values that could be moved to the vertex shader

After optimizing the decal shader and moving character shading to the high spec, re-profiling Darkspore* only shows Call B that corresponds to parallel lights and shadows. Given the amount of work done in Call B, it is expected to be expensive but grouping all these tasks into one call amortizes the cost of picking up the normal and depth values and recovering the position.

Figure 9: Re-profiling Darkspore* after optimizations to the decal shader

Hot Loading Shaders in Frame Analyzer

The Darkspore* team used Frame Analyzer to verify the changes made to the decal shaders had the positive impact they expected as shown above. But, they also took advantage of the hot loading shaders feature of Frame Analyzer to test changes on the fly. Frame Analyzer allows for replacement of shaders in HLSL or shader assembly for selected calls. By hot loading shaders in Frame Analyzer, you can immediately see the performance difference of the changes without having to capture another frame and hopefully recreate enough of the same events in the scene to make it comparable.

Using this hot loading functionality, the Darkspore* team was able to rapidly test changes made to the decal shaders. After loading the modified decal pixel shader for calls A and C, the deferred and final pass draw calls, Frame Analyzer showed a 30% and 24% improvement respectively for these calls.

Figure 10: 30% improvement on Call A with modified decal shader

They Drew First (Deferred) Blood

Looking further at these two calls that draw the blood decals in Frame Analyzer, we can see their volumes if we set the render state to wireframe. The blood decals are effectively writing all of the pixels in the volume because there is no test in place to early out in the pixel pipeline. The stencil buffer can be used to kill pixels in the final pass.

Figure 11: Blood decals writing to too many pixels

After making the change to the final pass and doing a simple frame rate check at the beginning of the level, there was a noticeable frame rate improvement of about 2 FPS. In Frame Analyzer, this new stencil write test can be set on the decal calls to fully understand the effect:

1. Select both calls A and C (deferred and final pass) and setting STENCILENABLE to true.
2. Then for call A, set STENCILFUNC to D3DCMP_ALWAYS, the STENCILPASS to D3DSTENCILOP_REPLACE, and STENCILREF to 2.
3. For call C, set STENCILFUNC to D3DCMP_EQUAL and the STENCILREF to 2.

Figure 12: Blood decals now being stenciled correctly

Now the blood decals are only writing the appropriate pixels and the final pass draw call was improved by 65.1% as reported by Frame Analyzer. All of these rendering changes could be done live and in the same session within Frame Analyzer.

Other Optimizations for Darkspore*

The Darkspore* team made several miscellaneous optimizations to the trees, terrain, character detail, and render system. The forest levels in the game had lots of trees. Maxis* found all the trees' models had roots below the ground that were being rendered, adding unnecessary polygons that no one ever saw. These were promptly removed saving processing time.

Figure 13: Dense tree root geometry was drawn but not visible

In Darkspore*, the terrain mixes 4 textures together per pass. The artists have control of which textures are mixed per vertex. Large sections were found to only need one texture instead of mixing four. These triangles that had only one texture were instead rendered with a simpler material. This was a big win in some cases and smaller in others, but always a win.

Figure 14: Not all landscape triangles require blended textures

With the Spore* Editor, the player can customize their squad creatures with countless combinations of parts and equipment. This results in some very detailed but also very polygon heavy creatures. The wireframe of the high LOD (level of detail) for a player creature shows the high polygon density of the creatures' models. NPCs were found to be at least this dense. Surprisingly, when these creatures got reduced to the one triangle per pixel level, they were hurting both pixel and vertex shader performance because of the way most graphics parts allocate 4 pixels to a "quad" and a minimum of one quad per triangle. This resulted in a waste of 3/4ths of the pixel shader performance.

Figure 15: Character model with high level of detail

As part of the light pre-pass renderer, normals were originally saved in view space. The view space normals only required two channels but weren't worth the cost of pixel shader instructions to recover the normal. Also, a world space normal was needed for reflective/refractive objects which resulted in a transform in the pixel shader, which would be a really bad place for an extra matrix transform. In the end, it was worth an extra channel to keep the normal in world space and avoid adding the extra matrix transform to the pixel shader.

Figure 16: World space normal used in reflective/refractive objects

Conclusion

Using the various features available in Intel® GPA 4.0, the Darkspore* team was able to discover and fix bottlenecks in their graphics pipeline. At the end of the day, many of the optimizations made for mainstream graphics improved the overall gaming experience. With these optimizations in place, Darkspore* runs at well over 30 FPS on the 2nd Generation Intel® Core™ processors.

References

Engel, Wolfgang. Light Pre-Pass Renderer. March 16, 2008. http://diaryofagraphicsprogrammer.blogspot.com/2008/03/light-pre-pass-renderer.html

Authors

David Lee Swenson (Electronic Arts*) is a 20 year software industry veteran. He’s Maxis* lead engineer on new rendering architecture and art pipelines for Darkspore*. Previously, David was responsible for the Spore* environment rendering and terraforming systems. He has also worked on console and PC titles at LucasArts*, 3DO*, Sierra On-Line*, Orion*, and MediaFactory.

Omar A Rodriguez (Intel Corporation) a graphics software engineer in the Intel® Software and Services Group, where he supports Intel® graphics solutions in the Visual Computing Software Division. He holds a B.S. in Computer Science from Arizona State University. Omar is not the lead guitarist for the Mars Volta*.

For the purposes of this article - when I refer to Intel Inspector XE - I am referring to the memory error analysis within Intel Inspector XE and/or Intel Parallel Inspector

Useful Settings for memory error analysis in Intel® Inspector XE:

Settings which impact memory error analysis in Intel Inspector XE:

Settings not recommended for use with memory error analysis in Intel® Inspector XE:

Settings which have no impact on memory error analysis of Intel Inspector XE:

Notes:
1) Memory Error Analysis Level 1 (Memory Leak Detection) requires information in the executable and all shared libs in your application to properly walk the call stack:

a) Frame pointers: Use -fno-omit-frame-pointer.

b) Exception handling information: enabled via -fasynchronous-unwind-tables, -fexceptions, or -O0

2) Using Debug versions of the Microsoft C Runtime Libraries (/MDd and /MTd) enables the Microsoft* debug heap manager. see: Using the Microsoft* debug heap manager with memory error analysis of Intel® Parallel Inspector.

Note: There are other options which may add frame pointer or Exception handling to your binary as a side effect, Examples: -fexceptions (which is the default for C++).or -O0 . To make sure the executable (and shared libs) have this information, use the objdump -h <binary> command. You should see .eh_frame_hdr section there.

More Information:

This article addressed the most obvious switches that developers would have concerns over. Most switches will work with Intel® Parallel Inspector and/or Intel Inspector XE - but not every switch combination is tested. If you have information regarding other switches, please add a comment to this article. If you have question regarding a particular switch please submit an issue to the Intel Inspector XE Forum.

Versions:
Intel® Inspector XE 2011
Intel® C++ Compiler 11.X,12.X
GCC Compiler 3.4.6 – GCC 4.5.0
MS Visual Studio 2005, 2008, 2010

Objective

Abstract

Introduction

Walkthrough

    for(int i=0; i!=n ;i++)
        N[i] = V[i] / magnitude(V[i]);

      for(int i=0; i!=n ;i++)
  003238C8  mov         esi,dword ptr [ebp+8]
  003238CB  push        edi
  003238CC  mov         edi,dword ptr [dest]
  003238CF  add         esi,8
  003238D2  mov         ebx,4000h
        N[i] = V[i] / magnitude(V[i]);
  003238D7  fld         dword ptr [esi-4]
  003238DA  fld         dword ptr [esi-8]
  003238DD  fld         dword ptr [esi]
  003238DF  fld         st(1)
  003238E1  fmulp       st(2),st
  003238E3  fld         st(2)
  003238E5  fmulp       st(3),st
  003238E7  fxch        st(1)
  003238E9  faddp       st(2),st
  003238EB  fmul        st(0),st
  003238ED  faddp       st(1),st
  003238EF  fstp        dword ptr [ebp-4]
  003238F2  fld         dword ptr [ebp-4]
  003238F5  call        _CIsqrt (3255B0h)
  003238FA  fstp        dword ptr [ebp-4]
  ...

        01385E80  movss       xmm0,dword ptr [eax-4]
        01385E85  movss       xmm1,dword ptr [eax-8]
        01385E8A  movss       xmm2,dword ptr [eax]
        01385E8E  movaps      xmm4,xmm0
        01385E91  mulss       xmm4,xmm0
        01385E95  movaps      xmm3,xmm1
        01385E98  mulss       xmm3,xmm1
        01385E9C  addss       xmm3,xmm4
        01385EA0  movaps      xmm4,xmm2
        01385EA3  mulss       xmm4,xmm2
        01385EA7  addss       xmm3,xmm4
        01385EAB  sqrtss      xmm4,xmm3
        01385EAF  movaps      xmm3,xmm5
        01385EB2  divss       xmm3,xmm4
        01385EB6  mulss       xmm0,xmm3
        01385EBA  mulss       xmm1,xmm3
        01385EBE  mulss       xmm2,xmm3
        01385EC2  movss       dword ptr [p],xmm1
        01385EC7  movss       dword ptr [ebp-28h],xmm0
        01385ECC  movq        xmm0,mmword ptr [p]
        01385ED1  movss       dword ptr [ebp-24h],xmm2

Performance Results

Conclusion and Further Performance Improvements

Objective

Abstract

Introduction

         __m128 a = _mm_rsqrt_ss(b);  // a = 1.0f/sqrt(b) approx

Generated Assembly

  inline void saxpy_simd4(float* S,float _a,const float* X,const float *Y,int n)
  {
     __m128 a = _mm_set1_ps(_a); 
     for(int i=0; i!=n ;i+=4)  // process 4 elements at a time
     {
         __m128 x = _mm_load_ps(X+i);
         __m128 y = _mm_load_ps(Y+i);  
         __m128 s = _mm_add_ps(_mm_mul_ps(a,x),y);  // a*x + y 
         _mm_store_ps(S+i, s );
     }
  }

    (AVX assembly instructions from loop listed only):
 001B4FF0  vmulps      xmm1,xmm0,xmmword ptr xpoints (1B97A0h)[eax]  
 001B4FF8  vaddps      xmm1,xmm1,xmmword ptr ypoints (1B9390h)[eax]  
 001B5000  vmovaps     xmmword ptr dest (1C0440h)[eax],xmm1  
 001B5008  add         eax,10h  
 001B500B  cmp         eax,400h  
 001B5010  jl          saxpy_128+20h (1B4FF0h)  

 00B5F7C2  mov         eax,dword ptr [i]  
 00B5F7C5  add         eax,4  
 00B5F7C8  mov         dword ptr [i],eax  
 00B5F7CB  cmp         dword ptr [i],100h  
 00B5F7D2  jge         saxpy_128+122h (0B5F872h)  
 00B5F7D8  mov         eax,dword ptr [i]  
 00B5F7DB  vmovaps     xmm0,xmmword ptr xpoints (0B715A0h)[eax*4]  
 00B5F7E4  vmovaps     xmmword ptr [ebp-1D0h],xmm0  
 00B5F7EC  vmovaps     xmm0,xmmword ptr [ebp-1D0h]  
 00B5F7F4  vmovaps     xmmword ptr [x],xmm0  
 00B5F7F9  mov         eax,dword ptr [i]  
 00B5F7FC  vmovaps     xmm0,xmmword ptr ypoints (0B71180h)[eax*4]  
 00B5F805  vmovaps     xmmword ptr [ebp-1B0h],xmm0  
 00B5F80D  vmovaps     xmm0,xmmword ptr [ebp-1B0h]  
 00B5F815  vmovaps     xmmword ptr [y],xmm0  
 00B5F81A  vmovaps     xmm0,xmmword ptr [x]  
 00B5F81F  vmovaps     xmm1,xmmword ptr [a]  
 00B5F824  vmulps      xmm0,xmm1,xmm0  
 00B5F828  vmovaps     xmmword ptr [ebp-190h],xmm0  
 00B5F830  vmovaps     xmm0,xmmword ptr [y]  
 00B5F835  vmovaps     xmm1,xmmword ptr [ebp-190h]  
 00B5F83D  vaddps      xmm0,xmm1,xmm0  
 00B5F841  vmovaps     xmmword ptr [ebp-170h],xmm0  
 00B5F849  vmovaps     xmm0,xmmword ptr [ebp-170h]  
 00B5F851  vmovaps     xmmword ptr [s],xmm0  
 00B5F859  vmovaps     xmm0,xmmword ptr [s]  
 00B5F861  mov         eax,dword ptr [i]  
 00B5F864  vmovaps     xmmword ptr dest (0B78240h)[eax*4],xmm0  
 00B5F86D  jmp         saxpy_128+72h (0B5F7C2h)  

Register Shortage

  void springupdate(__m256 A[][3], __m256 B[][3],__m256 &restlen)
  {
    __m256 half = _mm256_set1_ps(0.5f)
    for(int i=0;i != N ; i++)  // 8*N constraints in total
    {
      // each a and b contain the xyz endpoints for 8 pseudo-springs
      __m256 *a=A[i];
      __m256 *b=B[i];
      __m256 vx  = _mm256_sub_ps(b[0],a[0]); // v.x=b.x-a.x
      __m256 vy  = _mm256_sub_ps(b[1],a[1]); // v.x=b.x-a.x
      __m256 vz  = _mm256_sub_ps(b[2],a[2]); // v.x=b.x-a.x
      __m256 dp  = vx*vx+vy*vy+vz*vz;        // assume operator overloads for add and mul 
      __m256 imag= _mm256_rsqrt_ps(dp);      // inverse magnitude
      // normalize v
      vx = _mm256_mul_ps(vx,imag); // vx *= inverse magnitude 
      vy = _mm256_mul_ps(vy,imag); // vy *= imag
      vz = _mm256_mul_ps(vz,imag); // vz *= imag 
      __m256 half_stretch = ( dp*imag - restlen) * half;
      // move endpoints a and b together 
      a[0]=a[0]+ vx * half_stretch;    
      a[1]=a[1]+ vy * half_stretch;    
      a[2]=a[2]+ vz * half_stretch;    
      b[0]=b[0]- vx * half_stretch;    
      b[1]=b[1]- vy * half_stretch;    
      b[2]=b[2]- vz * half_stretch;   
    }   
  }

Conclusion