Self-test: Collecting basic performance data with GPA

So you want to self-test your game, and collect some basic performance data. You're going to use Intel GPA to analyze your game on Windows, but you're not quite sure how to get started. Here are some simple steps to get you running quickly. (These directions are required for companies in Intel's self-test program, but a good idea for all developers.)

Getting started

You'll be collecting some real-time data as you play your game (watching and then capturing it with the GPA System Analyzer), and you'll then capture one or more game frames to study more deepy with GPA Frame Analyzer. This will give you data files you can analyze or share with somebody else if you wish to review your performance with them.

Pick some part of your game that represents typical gameplay. Be ready to play it while you capture data. If you have very different parts of gameplay, plan to study each. You'll want to profile every aspect of your game (loading screens, menus, etc.) eventually, but the single most important part is typical gameplay, so start there.

Configure your game as you prefer for performance testing. We recommend (and require for the self-test program) you use 1366x768 resolution or better, at Medium quality or better. Your game should also detect the graphics hardware and configure itself correctly "out of the box", so your players don't need to make any config changes. See the GPU Detect sample to see how to do this. That sample also includes the latest list of Intel device IDs.

Make sure your machines have the latest software; at the very least check for updates to GPA and your graphics driver, since those may influence your results.

A special note. At the moment, you won't be able to use GPA if this is an OpenGL game - GPA doesn't support OpenGL on Windows. Are you an OpenGL developer? Head to the GPA Forum and let us know what you think.

GPA also can't collect frame captures for Windows Store games or games running DirectX 8 or older.

For full details on what works and doesn't, visit the GPA release notes.

Install

Install GPA on your test system (sometimes called the test target), and also on another system (the analysis platform). You may find it convenient to have the installation create desktop icons for you. The configuration of the analysis system doesn't matter much, but it needs to be running Windows 7 or Windows 8.

Launch and connect

Start the GPA Monitor on both systems. You'll see it in the taskbar.

On the test system, hover your mouse over the GPA Monitor icon, and you'll see the IP address of that system. Make a note of this address; you'll be using it soon. At this point, you don't need anything else running on the test system.

On the analysis system, click on the GPA Monitor, and pick "System Analyzer". System Analyzer will start up, and ask where to connect. The first time you use it, it'll say <This Machine>

Clear that text, enter the IP address of your test system, and click Connect. After System Analyzer connects, you'll see a summary of the different ways you might study what's happening on your test target system. If you have any trouble connecting to the target system, check your network configuration.

NOTE: If you can't connect from the analysis system, please don't simply run System Analyzer on the test machine to collect your data. It has a feature-rich GUI which can slow down your game if it's running on the same system. If you must run on a single system, the GPA System Analyzer Heads Up Display (HUD) has much of the same functionality.

In the System Profiling category, you'll see System View. That's not quite the view you want for studying your game, since it shows only the generic summary of everything running on your system. To study your game directly, you'll need to pick something else.

Now, start your game on the test system. When you look at System Analyzer, you'll see a new category for User Applications, along with your game. It doesn't matter exactly what you're doing in your game yet, it just needs to be running.

In this case, we're running the sample game CityRacer. You should see your game show up in the User Applications, like CityRacer does here.

If your game doesn't appear in the list, click on the GPA Monitor icon on the test target system, choose Preferences, enable Auto-detect launched applications, then close and start your game again. The GPA Monitor will now display a green dot. If your game still isn't listed in System Analyzer, go to the GPA Forum to discuss. Because there's a small overhead for auto-detection, this feature will reset each time you start GPA Monitor. If it's required for your game, you'll need to turn it on every time you start the GPA Monitor.

Collect data and study

Now that your game is running, click on its name to get ready to study it in real time.

At the top of the window, System Analyzer shows you which executable it's monitoring. If it says "Monitoring System View", go back and try the above steps again.

You'll see a long list of metrics on the left, some strip charts on the right, and some State Overrides you can use to study the behavior of your game. At the top, there are several command buttons and some summary stats. The command buttons should all be enabled (except for "next frame"). If they aren't, check the steps above to make sure you've connected to your game, and check that your game uses one of GPA's supported versions of DirectX.

Now, set up the metrics you want to watch in real time. Find a metric on the left, and drag it into one of the charts on the right. You can view more metrics by dragging a new metric in between the others, or you can display multiple metrics in one chart with <ctrl>-Click.

To start, show these metrics:

  • Target App CPU Load (in the CPU section) - your game's CPU use
  • FPS (in the DirectX section) - the game's instantaneous frame rate, as well as averages over time
  • GPU Busy (in the GPU section) - how heavily your game is using the GPU
  • GPU Frequency (in the GPU section) - shows whether the GPU is running at its maximum frequency or is throttled back
Later, you can study other metrics.

Once you have these metrics displayed in System Analyzer, turn your attention back to your game. Ensure your configuration is the way you want (set resolution to 1366x768 or better, with Medium quality settings), to give a good performance baseline and show good playability.

Start playing your game and watch the System Analyzer display. Does the performance data look like you expect? GPU-bound games will tend to have fairly high GPU Busy values and GPU frequency will tend to be at its mid-point or higher. If you want to know details about your CPU with its integrated GPU (like the supported GPU frequencies), you can discover some hardware details with a tool like CPU-Z and then check those on http://ark.intel.com/.


When you feel like System Analyzer is showing the right data, continue playing the game, and press the CSV button at the top. Wait about 30 seconds and press it again. The exact time isn't important, but you should collect data long enough to give a good average across your typical gameplay. You've just captured a 30-second summary of the data shown in System Analyzer, and saved it into a comma-separated value file.


Now, press the camera icon at the top of System Analyzer. You've just taken a frame capture of your game.

If you have different parts of your gameplay to study, repeat those two steps (CSV and frame capture) as needed.

Reviewing your results
Now that you have your data, what should you do with it? Both file types are captured in your "My Documents/GPA_<release>" folder, but on different machines. The CSV file is stored on the analysis machine and the frame capture is stored on the test machine (it can be quite large). Each file is automatically named with the executable name plus the data and time it was recorded.

For a CSV file, open it (with a text editor or spreadsheet tool) and look at it, and check if it shows what you expected. You should see a column heading for each piece of data that you were showing in System Analyzer, and a row for every sample taken during your 30-second interval. Check that the values make sense. In rare cases, a few values are shown that are out of range. If that happens, uninstall and reinstall GPA, and check these directions.
For the frame capture, view it in GPA Frame Analyzer. If you prefer, you can open Frame Analyzer from the analysis machine, giving it the IP address of your test machine like you did with System Analyzer, but it may take a while to copy and open. Faster is to run Frame Analyzer on the test machine, and analyze it there. Either way, close your game before starting Frame Analyzer. Since Frame Analyzer renders the frame in realtime, its performance will be altered if the game is still running.

To share your test results with a partner, zip up both files and post to a file sharing site like Dropbox or your own FTP site.

Digging deeper
Now that you have seen a small sample of GPA features, dig deeper.

In System Analyzer, study all the metrics in realtime that might interest you. Once you have a sense where your bottlenecks are, study the more-detailed metrics in that area. Collect CSV files and do offline analysis of them, looking for patterns. Use the Overrides to check the various things that might be bottlenecks in your game. Collect multiple frames to study in Frame Analyzer, and collect CPU traces to study in Platform Analyzer (something we didn't discuss here).

In Frame Analyzer, click different units of work ("ergs") at the top to see what's happening in them. Each corresponds to a DirectX call, like a Draw, Clear, etc. You can visualize that work differently, based on some selectors at the top, and you can look at the details of each call with the render target view and detail tabs at the bottom. Find the items you want to understand better, and start experimenting with them. You can do many experiments inside Frame Analyzer, and get realtime feedback on how that would change your frame.

To see the CPU activity, corrrelated to GPU activity, try capturing a CPU trace with the red "record" button in System Analyzer. Open the result in GPA Platform Analyzer to see a few seconds of CPU/task activity.

We hope these tools help you better understand how your game performs, and guide you toward making it run the best that it can!
For more complete information about compiler optimizations, see our Optimization Notice.