Officially, the Intel Cluster Tools support RPM-based installs only. For a full list of supported OSes, check out our compatibility matrix.

If you’d still like to use the tools on Debian and Ubuntu systems, the following steps can help you in, at least, installing the tools. Note that the process below still requires that a valid license for the desired Cluster Tool is present on the machine.

1. Download the latest version of the Intel® MPI Library for Linux* from the Intel® Registration Center.


2. Untar the package and go down into the newly created directory

   $ tar –xzf l_mpi_p_3.2.1.009.tgz
   $ cd l_mpi_p_3.2.1.009/



3. Create a new directory and extract the RPM files using the provided install.sh script. Note that you’d need a valid license to do so.

   $ mkdir rpms
   
   $ ./install.sh --extract=./rpms
   Extracting components...
   Extracting files... ################################################ [100%]
   Product's rpm and EULA have been extracted in ./rpms
   Please carefully read EULA before using the product
   
   $ ls –l
   

   intel-mpi-em64t-3.2.1p-009.x86_64.rpm	# SDK for the Intel MPI Library for Linux*
   intel-mpi-rt-em64t-3.2.1p-009.x86_64.rpm	# Runtime component of the Intel MPI Library
   mpiEULA.txt	# End-User License Agreement for the Intel MPI Library. # Read before using.
   redist.txt	# List of Intel MPI Library files whose redistribution is allowed



4. Convert the RPM files into DEB files, to be installed on your Debian-based system using the alien utility

   $ which alien
   /usr/bin/alien
   
   $ alien intel-mpi-em64t-3.2.1p-009.x86_64.rpm
   intel-mpi-em64t_3.2.1p-10_amd64.deb generated

   Here note that converting the SDK package for the Intel MPI Library is enough, as the runtime libraries are already contained within.


5. Now that you have the .deb file generated, install using your preferred method. For example, this is how it’s done using the dpkg utility

   # This will use the default /opt/intel installation directory
   $ dpkg --install intel-mpi-em64t_3.2.1p-10_amd64.deb



6. Done! You are now free to enjoy the Intel MPI Library at your leisure.



Even though this example highlighted the Debian-based installation of the Intel MPI Library, the same technique can be applied to the Intel® Cluster Toolkit for Linux*, Intel® Cluster Toolkit Compiler Edition for Linux*, and the Intel® Trace Analyzer and Collector for Linux*.

Yes, Intel MPI does support mixed MPI/OpenMP applications. To make sure your hybrid code runs correctly, follow these steps:

1. Use the thread safe version of the Intel MPI Library by using the -mt_mpi compiler driver option
2. Set the I_MPI_PIN_DOMAIN environment variable to select the desired process pinning scheme. The recommended setting is omp:

   $ export I_MPI_PIN_DOMAIN=omp

   This will set the process pinning domain size to be equal to OMP_NUM_THREADS. Therefore, each MPI process can create $OMP_NUM_THREADS number of children threads for running within the corresponding domain. If OMP_NUM_THREADS is not set, each node is treated as a separate domain (which will allow as many threads per MPI processes as there are cores).

   NOTE: In order to pin OpenMP threads inside the domain, use the corresponding OpenMP feature by setting the KMP_AFFINITY environment variable.

3. See the “Interoperability with OpenMP*” section in the Intel® MPI Library for Linux* Reference Manual for more details

The below instructions describe how to run Intel MPI jobs using Sun Grid Engine. This document relates to Linux*. While there are some differences and additional steps when using Microsoft* Windows*, in general the procedure is the same.

All optional steps are recommended but not necessary for successful integration.

1. [Optional] Visit sun.com and get a brief overview of SGE
2. Installation

   See the Installation Guide from sun.com for details. Roughly, the steps are as follows:

   • Install Master Host (see ‘How to Install the MasterHost’ section);
   • Install Execution Host (see ‘How to Install ExecutionHosts’ section);
   • Register Administration Hosts (see the corresponding section in the Installation Guide);
   • Register Submit Hosts (see corresponding section);
   • Verify the installation (see corresponding section).

   IMPORTANT NOTES:

   • To finalize the installation process, you’ll have to configure the network services manually (by modifying /etc/services), which requires root privileges.
   • It’s possible to install/run SGE as a non-privileged user, but
     1. there are some limitations in that case;
     2. you need root privileges for the complete installation process (at least, for modifying /etc/services).
3. Create a new Parallel Environment (PE) for Intel MPI
   1. Create the appropriate configuration file for the new PE. It should contain the following lines:

      pe_name	impi
      slots	999
      user_lists	NONE
      xuser_lists	NONE
      start_proc_args	NONE
      stop_proc_args	NONE
      allocation_rule	$round_robin
      control_slaves	FALSE
      job_is_first_task	FALSE
      urgency_slots	min

   2. Add the new PE using the following command:

      ‘qconf –Ap <config_file>’

   USEFUL COMMANDS:
   * qconf –spl – view all PEs currently available;
   * qconf –sp <PE_name> - view settings for a particular PE;
   * qconf –dp <PE_name> - remove a PE;
   * qconf –mp <PE_name> - modify an existing PE.

   Also see the ‘Managing Special Environment’ section in the Administration Guide from sun.com if you need more details about PE configuration.

4. Associate a queue with the new PE

   Use the following commands for that:

   1. qconf –sql – to see all queues available;
   2. qconf –mq <queue_name> - to modify the queue’s settings. Find the ‘pe_list’ property in the open window and add the ‘impi’ string to that property.

   USEFUL COMMANDS:
   * qconf –sq <queue_name> - view the queue’s settings.

   See the Administration Guide if you need more details about the queue configuration process.

5. Add Intel MPI environment to your current environment by sourcing the appropriate mpivars.[c]sh script located in the <install_dir>/bin[64] directory
6. Build the MPI application to be run
7. [Optional] Make sure that Intel MPI works fine on the desired hosts. For this, manually run your application on the desired hosts individually
8. Submit your MPI job to SGE

   Use the following command for that:

   qsub -N <job_name> -pe impi <num_of_processes> \
-V <mpirun_absolute_name> -r ssh -np <num_of_processes> <app_absolute_name>

   
   where
   -V option is used so that all environment variables available in the current shell are exported to a job.

    

   USEFUL COMMANDS to monitor and control jobs:
   * qstat – show status of SGE jobs and queues;
   * qstat –j – show detailed information about jobs (can be useful for pending jobs);
   * qdel – remove existing job.
   After submitting the job you can monitor its status using the qstat command. When the job is finished, you can find the job’s output and error output in your HOME directory – just look for <job_name>.o<jobID> and <job_name>.e<jobID> files.

   See the User’s Guide, if you need more information about the job submission process.

Closer integration with SGE

Read the 'Tight Integration of Parallel Environments and Grid Engine Software' section in SGE's Administration Guide first.

To enable tight integration for Intel MPI, use the same procedure as the one mentioned above, but use a different configuration file for the PE at step #3.

The configuration file should contain the following lines:

pe_name	impi_tight
slots	999
user_lists	NONE
xuser_lists	NONE
start_proc_args	<SGE_install_dir>/mpi/startmpi.sh -catch_rsh $pe_hostfile
stop_proc_args	<SGE_install_dir>/mpi/stopmpi.sh
allocation_rule	$round_robin
control_slaves	TRUE
job_is_first_task	FALSE
urgency_slots	min

Does Intel® MPI Library limit the number of processes run on a single node?

Answer

No, it does not. Intel MPI has no internal restrictions on the number of MPI processes started on a node.

NOTE: Be careful when running a large number of processes on a single machine. At some point, you might start oversubscribing the processor (meaning starting more processes than there are cores). This could lead to performance degradation.

I have a cluster, which has for example 3 nodes, where the mpd.hosts file contains:

$ cat mpd.hosts
node2
node3

The master node, which is node1 is not listed in the mpd.hosts file. The shell commands:

$ mpdboot -r ssh -n 2 -f ~/mpd.hosts
$ mpiexec -n 4 /bin/hostname

node1
node1
node2
node2

Solution

By design, the master node (i.e., the node where the mpiexec command is launched from) needs to have a daemon running. If you want to run your application on node2 and node3, you can use the -host options with the mpiexec command as follows:

$ mpdboot -r ssh -n 3 -f ~/mpd.hosts
$ mpiexec -n 2 -host node2 /bin/hostname : -n 2 -host node3 /bin/hostname

$ mpdboot -r ssh -n 3 -f ~/mpd.hosts
$ mpiexec -nolocal -n 4 /bin/hostname

This will run the hostname command on nodes node2 and node3 only, even though there are three mpd daemons total.

• Release Notes for Intel® MPI Library
• Release Notes for Intel® Trace Analyzer and Collector
• Release Notes for Intel® Math Kernel Library
• Release Notes for Intel® Cluster Toolkit and Intel® Cluster Toolkit Compiler Edition

Operating System: