Using the differencing feature in Intel® Parallel Amplifier to find performance regressions

Introduction : 
Application development is an iterative process and it is easy to introduce new performance issues in code that is constantly evolving. If your typical use case is to check a version of the code out, modify existing code or add new code and check it back in, then Intel® Parallel Amplifier’s differencing feature can be used to track any performance regressions due to these changes. Intel® Parallel Amplifier shows the differences at the function and module level so you can determine which changes caused the regression

Version : 
Intel® Parallel Amplifier
Microsoft* Visual Studio 2005
Microsoft* Visual Studio 2008

Application Notes : 

Let's assume the following sequence of events:

  1. Check out source from source control
  2. Compile and run this version of the source against a baseline workload
  3. Modify existing code or add new code
  4. Compile and run the modified version on the baseline workload
  5. If results in step 2 and 4 match, check code back in

In the above work flow, checks are in place to ensure that the results generated by the original code and modified code are the same for check-in to occur. Many times, the performance of the two versions are not monitored closely to track regressions.  To track performance regressions in such a workflow, the above sequence of events will have to be slightly modified. Let's look at one such modified sequence below:

  1. Check out source from source control
  2. Compile and run this version of the source against a baseline workload
  3. Run Intel® Parallel Amplifier (Hotspot, Concurrency or Locks & waits analysis modes based on the application characteristics) - let's call this Result 1
  4. Modify existing code or add new code
  5. Compile and run the modified version on the baseline workload
  6. Run Intel® Parallel Amplifier on the modified code (same analysis type run as in step 3) - let's call this Result 2
  7. Compare the two runs from Step 3 and Step 6 using the differencing feature

When an application is run under Intel® Parallel Amplifier, the results are automatically opened in Visual Studio*. In order to invoke the differencing feature, you can use one of the two methods outlined below:


Usage model 1:

  1. Click on the "Compare Results" button in the toolbar circled in red shown in the picture below
    • difference-toolbar.GIF
  2. This will open a dialog requesting you to pick the two results you wish to compare. In this case, Result 1 will be pointing to the result file from the original source and Result 2 will be pointing to the result file from the modified source. 
    • compare-results-dialog.GIF
  3. Pressing the "Compare" button will calculate (Result 1 - Result 2). In our example scenario, a positive difference will indicate improved performance.  On the other hand, a negative difference shows a regression in performance. The picture below shows a performance improvement in the modified code.
    • difference-view.GIF

The order in which results files are loaded into the Compare Results Dialog shown in Step 2 determines whether a positive or a negative difference results in a performance regression.

Usage model 2:

  1. Right-click on the result you want to use as reference for comparison and select the "Compare Results" menu attribute in the context menu.
    • alternate-way.GIF
  2. The Compare Results dialog will appear with the selected results file selected for Result 1 and you can then select another result in Result 2 for the comparison. Press the  "Compare" button to see the difference view generated by differencing Result1 and Result2 fields in the selection dialog.

More Information: 
This article discusses the usage model for "Compare Results" feature in Intel® Parallel Amplifier. It highlights the different ways one can access this feature and describes a simple methodology to find performance regression. If you have questions regarding this feature please submit an issue to the Intel® Parallel Studio Forum.

For more complete information about compiler optimizations, see our Optimization Notice.