Installing Intel® VTune™ Amplifier in a Cluster Environment

There are two typical scenarios for installing Intel® VTune™ Amplifier in a cluster environment:

  1. Install and enable the kernel driver on specific nodes and for specific users. The users can launch VTune Amplifier on a node or machine from a shared partition mounted on their system and use this to analyze programs or system behavior on that node or machine. Even though the program's execution may be distributed among other nodes, a single instance of the tool is collecting performance data for the single node on which it is launched. This is the most common use case as it requires less administrative overhead.

  2. Install VTune Amplifier on a shared partition that is available to all users. You can either allow the VTune Amplifier installer to automatically install the SEP drivers or you can manually install the drivers after product installation succeeds. This use case requires additional administrative oversight.

There are a few changes to the typical installation steps when installing in a cluster environment. The steps that follow set up a cluster environment so that users that belong to a custom group (my_group) can run hardware event based analysis on specific nodes. Users can run VTune Amplifier using the GUI or the command line, depending on their display device.

Tip

VTune Amplifier users in a cluster environment are expected to set the results directory path within their home directory. By default, the tool uses ${HOME}/intel/amplxe/Projects/<project-name>, but users can set up a directory to save analysis results to a local path. Setting a local path is helpful when network connection speeds are slow as it can help speed data loading and processing while analyzing collected results. For example, a user could specify the /tmp directory, but should ensure that enough disk space is available where /tmp is mounted.

Advanced Installation

Note

Refer to the Prerequisites before you begin the installation.

Use the following steps to complete the installation with more control over driver installation:

  1. Run the install script with the following command to skip driver installation on the current machine. Running with the --SHARED_INSTALL option is required as users are expected to launch profiling on the compute nodes, in general, not necessarily on the main or head node or the node used by administrator for installation.

    ./install.sh --SHARED_INSTALL

    The product installation generates the amplxe-vars.{sh|csh} and amplxe-<version.build_number> module file for cluster environments that use environment modules. These files are added to the top-level product directory and are used to set up the user environment for launching VTune Amplifier. For more information about environment modules, see http://modules.sourceforge.net.

    Note

    Intel VTune Amplifier can still be used for profiling with the predefined analysis types based on software sampling even without the sampling driver installed. Users can launch the product from a shared file system to run the predefined analysis types (hotspots, threading, etc.).

  2. Build the SEP driver for the current OS. See the <install-dir>/sepdk/src/README.txt document for more details on building the driver.

    • Build in the src directory using the following commands:

      cd <install-dir>/sepdk/src ./build-driver -ni

    • Build in a custom directory using the --install-dir option to specify the installation directory. Make sure that the directories specified in the option already exist.

      1. Run the following command to build and install the driver:

        ./build-driver --install-dir=

      2. Use the following commands to copy scripts to the driver installation directory:

        cp insmod-sep <custom-install-dir> cp rmmod-sep <custom-install-dir> cp boot-script <custom-install-dir>

      3. Create a pax subdirectory in the new driver location and navigate to the new directory:

        cd pax

      4. Use the following commands to copy the pax driver scripts:

        cp insmod-pax <custom-install-dir>/pax cp rmmod-pax <custom-install-dir>/pax cp boot-script <custom-install-dir>/pax

  3. Install the event based sampling kernel driver on the selected nodes.

    Enter the node on which performance profiling will be completed and run the following commands from the shared directory where the driver is located:

    cd <install-dir>/sepdk/prebuilt

    ./insmod-sep --group my_group

    ./boot-script --install --group my_group

    The insmod-sep script loads the driver into the system on the current node. The boot-script script configures the driver boot script and installs it in the appropriate system directory. Use the --help option to view details on the available script options.

After installation is complete, you can verify that the kernel driver is installed and loaded on a node using the following command:

lsmod | grep sep

Example

A homogeneous cluster system is set up where the users only have direct access to one node, node1. The only disk space available for writing is the user's home directory. The users have all of their data and software on the file system mounted on node1. They start their tasks using job scripts, which involve MPI mechanisms for dispatching the tasks among the other nodes.

To allow users to run an analysis using VTune Amplifier, the administrator has to make sure that the product can be launched on each compute node. The kernel driver also must be installed and loaded on each compute node. After the product and drivers are installed, users can run performance collection on the nodes using the scheduling system scripts to launch an analysis. For example, for MPI applications the mpiexec script can be used on node1 to launch the profiling collector on the other nodes by specifying a user application to run as a parameter.

For more complete information about compiler optimizations, see our Optimization Notice.
Select sticky button color: 
Orange (only for download buttons)