Analyzing MPI Workloads

Intel® Advisor allows you to analyze parallel tasks running on a cluster, so you can examine your MPI application for opportunities of adding vectorization or threading parallelism. Use the Intel® MPI gtool with mpiexec to invoke the advixe-cl command and spawn MPI processes across the cluster.

MPI analysis can be performed only through the command line interface, but the result can be viewed through the standalone GUI, as well as the command line.

Tips

Consider the following when running collections for an MPI application:

  • Only homogenous clusters are supported.

  • Analysis data can be saved to a shared partition, or to local directories on the cluster.

  • Only one processes' data can be viewed at a time.

  • Intel® Advisor saves collection results into a subdirectory under the Intel Advisor project directory. If you wish to collect and then view (in a separate session) data for more than one process, specify a new project directory when running new collection.

  • Specify one and the same project directory when running various Intel Advisor collections for the selected process.

MPI Implementations Support

You can use the Intel Advisor with the Intel® MPI Library and other MPI implementations, but be aware of the following details:

  • You may need to adjust the command examples in this section to work for non-Intel MPI implementations. For example, adjust commands provided for process ranks to limit the number of processes in the job.

  • An MPI implementation needs to operate in cases when there is the Intel Advisor process (advixe-cl) between the launcher process (mpiexec) and the application process. This means that the communication information should be passed using environment variables, as most MPI implementations do. Intel Advisor does not work on an MPI implementation that tries to pass communication information from its immediate parent process.

Intel® MPI Command Syntax

You can use Intel Advisor to generate the command line for collecting results on multiple MPI ranks. To do that,

  1. In Intel Advisor user interface go to Project Properties > Analysis Target tab > Survey Analysis Types > Survey Hotspots Analysis.
  2. Select the Use MPI Launcher checkbox.
  3. Specify the MPI run parameters, then copy the command line from Get command line text box to your clipboard.

Use the -gtool option of mpiexec with Intel® MPI Library 5.0.2 and higher:

$ mpiexec –gtool “advixe-cl -collect <analysis_type> -project-dir <project_PATH>:<ranks_set>” -n <N> <application_name> [myApplication_options]

where:

  • <analysis_type> is one of the Intel Advisor tools:

    • survey runs the target process and collects basic information about the hotspots.

    • tripcounts collects data on the loop trip counts.

    • dependencies collects information possible dependencies in your application, requires

      • Either loop ID as an additional parameter (-mark-up-list=<loop_ID). Find the loop ID in the Survey report (--report survey) or using the Command Line link in the Intel® Advisor GUI Workflow tab (right under the button).

      • Or annotations in the source code.

    • map collects information about memory access patterns for the selected loops. Also requires loop IDs for the analysis.

    • suitability checks suitability of the parallel site that you want to insert into your target application. Requires annotations to be added into the source code of your application, and also requires recompilation in Debug mode.

  • <ranks_set> is the set of MPI ranks to run the analysis for. Separate ranks with a comma, or use a dash "-" to set a range of ranks. Use all to analyze all the ranks.

For detailed syntax, refer to the Intel® MPI Library Reference Manual.

Generic MPI Command Syntax

Use mpiexec with the advixe-cl command to spawn processes across the cluster and collect data about the application.

Each process has a rank associated with it. This rank is used to identify the result data.

To collect performance or dependencies data for an MPI program with Intel Advisor, the general form of the mpiexec command is:

$ mpiexec -n <N> advixe-cl -project-dir <project_PATH> -collect <analysis_type> -search-dir src:r=<sources_PATH> -- myApplication [myApplication_options]

where:

  • <N> is the number of MPI processes to launch

  • <project_PATH> specifies the path/name of the project directory

  • <analysis_type> is survey, suitability or dependencies

  • <sources_PATH> is the path to the directory where annotated sources are stored

Viewing Results via Intel Advisor

As a result of collection, Intel Advisor creates a number of result directories in the directory specified with -project-dir. The nested result directories are named as rank.0, rank.1, ... rank.n, where the numeric suffix n corresponds to the MPI process rank.

To view the collected results for a specific rank, you can either open a result project file (*.advixeproj) that resides in the -project-dir via the Intel Advisor GUI, or run the Intel Advisor CLI report:

$ advixe-cl -report <analysis_type> -project-dir <project_PATH>:<ranks> 

You can view only one rank's results at a time.

Additional MPI Resources

For more details on analyzing MPI applications, see the Intel MPI Library and online MPI documentation on the Intel® Developer Zone at http://software.intel.com/en-US/articles/intel-mpi-library-documentation/

Other Intel® Developer Zone online resources that discuss usage of the Intel® Parallel Studio XE Cluster Edition with the Intel MPI Library:

Hybrid applications: Intel MPI Library and OpenMP* on the Intel Developer Zone at http://software.intel.com/en-US/articles/hybrid-applications-intelmpi-openmp/

For more complete information about compiler optimizations, see our Optimization Notice.