Simplify, Demystify and Speed Threading Design

Intel® Advisor XE is a threading prototyping tool for C, C++, C# and Fortran. Quickly model and compare the performance scaling of different threading designs without the cost and disruption of implementation. Find and fix data sharing issues during design when they are less expensive to fix.

Intel Advisor XE 2013’s suitability analysis gives you a performance estimate before you invest significant effort in implementation. Implement only the options that have a high return on investment.

• Profiling identifies where an application will benefit most from parallelism
• Compare the effort and benefit of alternatives before you invest in implementation
• Guides resolution of key data conflicts. Assists in adding synchronization.

Quotes:

“Intel® Advisor XE can be invaluable in developing the understanding required to parallelize existing code. It assists with identifying opportunities, designing tests, modeling scenarios and revealing flaws.”
Matt Osterberg,
Senior Software Engineer,
Vickery Research Alliance

“It's powerful to have the performance and correctness analysis combined together in Intel Advisor XE. Experimenting with different parallelization techniques is much faster and more convenient.”
Arthur Yuldashev,
Senior Lecturer,
Ufa State Aviation Technical University

• What is Intel® Advisor XE 2013?

• Intel Advisor XE 2013 is a set of tools that helps you add parallelism to your application in order to improve performance on multi-core systems. It guides you through a sequence of steps to analyze and transform your program for parallelism.

• Does Intel Advisor XE 2013 automatically parallelize my code?

• No. However, it does help you get to the point where you can parallelize code confidently and methodically.

• What are some key things I can learn about my program using Intel Advisor XE 2013 workflow?

• Intel Advisor XE 2013 helps you identify portions of your code which may be good candidates for parallelization. It can also help you determine when it is not a good idea to parallelize a piece of code.

• Why do I need Intel Advisor XE 2013? OpenMP, Intel® VTune™ Amplifier XE 2013 and Intel® Inspector XE 2013 seem good enough.

• Intel Advisor XE 2013 has several unique advantages because of its delayed implementation workflow:

  • Easier debugging. It works by annotating and modeling the "serial" code. Any change you make with Intel Advisor XE 2013 does not affect the correctness of your application. It will not crash due to "bugs" in your parallel implementation. If you use OpenMP directly, it's possible that "bugs" in your implementation may cause your program to behave incorrectly, generate the wrong answers, not converge and maybe even crash. Using a parallel break-point debugger on incorrect parallel programs is not an easy task. Many bugs are "random" in nature, and even when detected usually show you the symptom of the problem not the root cause.
  • Parallelize without adding schedule risk. Annotations describe your design to Intel Advisor XE 2013 letting it give you performance and correctness information, but make no changes to your compiled code. You can continue to release product updates during the design of your parallelism. All your test cases continue to work. Verify that your application is stable and correct before you go to the final step – implementing parallelism.
  • Performance estimates with less work. Intel Advisor XE 2013 minimizes the amount of work you need to do to get the first estimates of potential performance improvements. Due to Amdahl's law, we know that we must focus our parallelism on the parts of the application in which the most CPU time us used. Intel Advisor XE 2013 lets you quickly get this answer, whereas with OpenMP, your program may not run correctly, or may not even run at all, so you would need to spend significant time debugging just to get to the answer that you are starting at the wrong place in your application.
  • Compare optimizations. Intel Advisor XE 2013 can tell you how to improve your parallel model, based on sensitivity analysis of the model to 5 different kinds of parallel overhead. Intel Advisor XE 2013 by default gives you an evaluation of how your application may run under a default programming model. It also calculates how it would run under 5 different kinds of optimizations. If any of these optimizations give a substantially better answer, you know which aspect of your parallel model to tune, in order to try and achieve the benefits of the optimization.
  • Workflow guides people new to parallel design. You may be an expert, but others on your team may not. Intel Advisor XE 2013 gives you the opportunity to train your entire team on effective parallelization strategies so not only are you more productive (others can use Intel Advisor XE 2013 to answer their questions instead of asking you), but your entire team becomes more productive too.
  • Analyze scalability. Intel Advisor XE 2013 annotations allow you to see the performance of you proposed parallel solution for various core counts via the Suitability analysis. If you have the luxury of being in the design phase of adding parallelism, you can make sure your solution will scale to more cores when they are available. If you go straight to parallelism via OpenMP, TBB, etc. you will only see the performance on the hardware available but won’t have an indication of how your parallel implementation will scale.

• Can Intel Advisor XE 2013 be used on applications for Intel® Many Integrated Core Architecture (Intel® MIC Architecture)?

• Yes. Intel Advisor XE 2013 currently does not directly support applications compiled for Intel® MIC Architecture execution, but it can be a powerful tool in the process of creating these applications. The first step in creating an application for Intel® MIC Architecture is to identify the potential parallelism, and to validate that this parallelism is expressible on the host platform. This is exactly the problem for which Intel Advisor XE 2013 was created.

• Can you run Intel Advisor XE 2013 on an MPI application?

• Yes. Intel Advisor XE 2013 currently does not understand MPI parallelism, but can be used to add thread level parallelism to an MPI application.

  Intel Advisor XE 2013 is designed to analyze serial code to help you understand how to optimize it through parallelization. Each rank of an MPI program is really a serial application. Therefore by using the Intel Advisor XE 2013 methodology on this code you can determine if additional parallel opportunity is present which you might be able to exploit. Intel Advisor XE 2013 does not analyze your MPI API usage, it basically ignores it. To use Intel Advisor XE 2013 on an MPI program you annotate the sequential code between MPI API calls the same as you would any other serial code section. Then you launch the advixe-cl command-line tool via the mpirun command. Each MPI rank on which the advixe-cl command is applied will generate a separate output result. You can manually copy this output result back to your host machine and then use advixe-gui to open and view this result.

• Does Intel Advisor XE 2013 model memory bandwidth and cache behavior?

• No, Intel Advisor XE 2013 does not model memory bandwidth or cache behavior.

  Intel Advisor XE 2013 is meant as the first tool to help you understand the structure and behavior of your serial application as you work to refactor it into parallel form. This process is composed of multiple steps. Each of these steps is necessary to accomplish in order to have a successful result. Intel Advisor XE 2013 helps you with the first of these steps, and then you transition to using Intel VTune Amplifier XE 2013, and Intel Inspector XE 2013 for the final steps.

  In a practical parallel program you must identify sufficient computation to execute in parallel to achieve you performance goals. If you do not do this, then additional restrictions (such as problems caused by limited memory bandwidth, or limited cache size) are mostly irrelevant. Intel Advisor XE 2013 evaluates the theoretical upper bound of available parallelism taking only code partitioning into independent tasks into account. With this partitioning, Intel Advisor XE 2013 allows you to understand if this exposes sufficient parallelism to achieve reasonable performance, and helps you understand the sensitivity of your proposed parallel model to scheduling overhead found in parallel frameworks, and helps you understand if you need to fix any data declaration, or synchronization issues which would otherwise cause your application to generate incorrect results when run in parallel.

  Once you have a model of a parallel execution as expressed via annotations that is both suitable for possible performance, and for correctness, then you can express this via the parallel framework. With the code refactored to use a parallel framework, you no longer have a serial application, and you are done using Intel Advisor XE 2013. At this point you evaluate your newly written parallel application using Intel VTune Amplifier XE 2013 to see how it maps to actual parallel hardware (including memory bandwidth and cache size effects). And you can use Intel Inspector XE to evaluate correctness and synchronization issues that may not have been present in a model which was strictly defined by relaxed-sequential execution.

• How does Intel Advisor XE 2013 deal with already parallel applications? How do OpenMP/MPI co-exist with Intel Advisor XE 2013 annotations?

• Intel Advisor XE 2013 models serial code. In any application serial or parallel, almost all of the code executed is serial. For example, in an application that has multiple threads, each thread is actually equivalent to a serial program. Intel Advisor XE 2013 separates each of these "concurrent" threads of execution into a separate serial analysis.

  In practice, it is usually most effective and understandable to disable any explicit parallelization in your application before using Intel Advisor XE 2013. This is usually done by setting the number of threads to be 1, such that any existing parallelized code is mapped to a single thread of execution.

  If this can be done, then you really do have a serial program, and then your task in using Intel Advisor XE 2013 is in trying to identify a new opportunity which has not already been parallelized. This might be a serial code region between two already parallelized regions. Or it might be hierarchical parallelism, where you are adding another level of parallelism to an already parallelized region. For example, you might have parallelized an inner-loop in a function, but you find that it is also productive to parallelize an outer-loop to increase the resulting scalability of the function.

• How does Intel Advisor XE 2013 model an already parallel region?

• Intel Advisor XE 2013 does not model already parallel regions. It ignores the fact the program may already be parallel, and instead treats each sequential thread of execution separately.

• Does Intel Advisor XE 2013 model a Parallel Framework accurately?

• No, but it does use the information to estimate scheduling and synchronization overhead. Intel Advisor XE 2013's job is not to perfectly predict the performance of your final parallel program after you finish the refactoring. Intel Advisor XE 2013's purpose is to help you make decisions to guide you along the path to create an effective parallel program.

  In particular Intel Advisor XE 2013 is always using heuristics and simplifying assumptions to make what it does practical. For Suitability, it approximates the tasking structure to only focus on the critical (usually large tasks), and it uses statistics to model the smaller tasks. It uses a theoretical "ideal" model of a parallel machine in order to do a quick evaluation of your program to tell you if you are on the right track.

  Intel Advisor XE 2013 lets you choose a Parallel Framework because:

  • This lets you document your expectation of what Parallel Framework you will ultimately use.
  • We use the Parallel Framework to estimate scheduling and synchronization overhead.

  Different parallel frameworks typically have different costs due to the way they are implemented. The task dispatch cost in TBB is slightly higher than in Cilk because TBB cannot use special code-generate strategies that are possible with a language based approach such as Cilk. System API's (such as CreateThread, or POSIX routines like mutexes) may be much higher overhead than specially optimized routines in a parallel run-time.

  The main purpose of the Suitability view is to help you understand the sensitivity of your chosen parallelism model to various overhead. You see these results in two different ways:

  • When you use the check-marks (on/off), you either consider or do not consider 5 different types of overhead which apply to your parallel model.
  • When you switch to a different parallel framework, you choose a new set of overheads to apply to the models.

  Quite often, if your parallelism model is not sensitive to overhead, then you will see almost identical results from each of the framework choices.

  However, in the cases where it shows a big difference, this is either telling you that you should use a parallel framework with more efficient task-dispatch and synchronization, or you should continue to work to refine your parallelism model so that it is not as sensitive to the overhead of the parallel framework.

• Why does Intel Advisor XE 2013 use annotations as its modeling language instead of parallel framework syntax?

• An advantage of Intel Advisor XE 2013's modeling language is that it is a simplification over what would be required in a parallel framework to express a fully correct parallel application. You don't need to get it all right at the same time, you can use the annotations incrementally, starting with an incomplete, incorrect annotated parallel model, and over time refine it to be more accurate and complete. Intel Advisor XE 2013 also does not require extensive refactoring that might be necessary with Threading Building Blocks. Deferring this refactoring until you are confident you have a good solution proposed allows you to explore many different alternative solutions before picking the right one for your application.

  Annotations describe your design to Intel Advisor XE 2013, but make no changes to your compiled code. You can continue to release product updates during the design of your parallelism. All your test cases continue to work. Verify that your application is stable and correct before you go to the final step – implementing parallelism.

• Does Intel Advisor XE 2013 automatically convert modeling annotations into parallel framework code?

• No. This would be a good research topic in automatic code refactoring. Our focus with Intel Advisor XE 2013 is to automate the analysis that will help a developer productively add parallelism. We’d like to do it all “automatically”, but there are still some things people do better.

• Will Intel Advisor XE 2013 find all threading errors in my program when using correctness analysis?

• Though it is likely that Intel Advisor XE 2013 can find any memory or threading errors present in the code paths exercised in some applications at the highest levels of analysis, there is no guarantee that Intel Advisor XE 2013 will find all threading errors in your program.

• Do I have to recompile and rebuild my programs to use Intel Advisor XE 2013?

• As part of the Intel Advisor XE 2013 workflow, you will need to recompile and rebuild your programs after adding annotations and prior to running Suitability (a release build is recommended) and prior to running Correctness (a debug build is recommended). We recommend use of debug binaries or release binaries with debug information, so that error reports can be attributed to source code locations.

• Does Intel Advisor XE 2013 correctness analysis run in a separate process?

• No, the analysis related code/data is injected into your program’s address space at runtime.

• Can Intel Advisor XE 2013 attach to or detach from my running process?

• No, current Intel Advisor XE 2013 does not support process attach or detach. In order to find multithreaded bugs in your process, Intel Advisor XE 2013 has to monitor synchronization events and related annotations from the start of the program.

  Of course, you can always stop Intel Advisor XE 2013 manually at any time after analysis starts.