Intel Software Network articles Feed http://software.intel.com/en-us/articles/intel-mpi-library-for-windows-kb/type/tips-and-techniques/ en-us Distributed memory coarray programs with process pinning
Introduction : This article describes a method to compile and run a distributed memory coarray program using Intel® Fortran Compiler XE 12.0. An example using Linux* is presented.


Version : Intel® Fortran Compiler XE 12.0


Application Notes : To compile for distributed memory coarrays, use compiler option -coarrays=distributed (Linux* OS) or /Qcoarrays:distributed (Windows* OS).  This requires an Intel® Cluster Toolkit license.  To compile for shared memory coarrays, use compiler option -coarrays=shared (Linux* OS) or /Qcoarrays:shared (Windows* OS).  Compiling for shared memory coarrays does not require an Intel® Cluster Toolkit license.


Obtaining Source Code : The coarray example from the Composer XE 'coarray_samples' directory could be used.


Prerequisites : An Intel® Cluster Toolkit license is required for compilation, and the Intel® MPI Library must be installed on the cluster nodes.


Configuration Set Up : A key for running a distributed memory coarray program with process pinning on specific nodes is to use compiler option -coarray-config-file=filename (Linux* OS)or /Qcoarray-config-file:filename (Windows* OS).  This enables you to take full advantage of Intel® MPI Library features in the coarrays environment, in the same way that 'mpiexec -config filename' allows mpiexec to take its commands from 'filename'.

The contents of the configuration file for this example:

-host host1 -env I_MPI_PIN_PROCESSOR_LIST 0,2,4 -n 3 <path to executable>coarry_dist_host.x : -host host2 -env I_MPI_PIN_PROCESSOR_LIST 1,3,5 -n 3 <path to executable>coarry_dist_host.x

This says to execute six coarray images 'coarry_dist_host.x' on nodes host1 and host2, using processors 0,2,4 on host1, and processors 1,3,5 on host2.  The I_MPI_PIN_PROCESSOR_LIST environment variable is used to achieve the process pinning on the indicated nodes.


Source Code Changes : See Verifying Correctness


Building the Application : Compile for distributed coarrays, create one coarray image, and specify the coarray configuration file:

ifort -coarray=distributed -coarray-num-images=1 -coarray-config-file=coarray_config.txt coarry_dist_host.f90 -o coarry_dist_host.x

Running the Application : Simply specify the name of the executable:
> <path to executable>/coarry_dist_host.x
Hello from image 1 out of 6
total images, and running on host: host1

Hello from image 2 out of 6
total images, and running on host: host1

Hello from image 3 out of 6
total images, and running on host: host1

Hello from image 5 out of 6
total images, and running on host: host2

Hello from image 4 out of 6
total images, and running on host: host2

Hello from image 6 out of 6
total images, and running on host: host2
>

Verifying Correctness : Embed 'call hostnm(hostname)' in your coarray program, then print 'hostname' to verify the images are executed on the correct nodes/processors.


Benefits : This method enables coarray image pinning on specific nodes/node processors.  Better load balance across cluster nodes might be obtained, or a subset of nodes easily partitioned.


Known Issues or Limitations : 
-- Some users have reported MPI environment issues when trying to run the executable in a standalone fashion.  These issues are under investigation, but as a workaround try using mpiexec to launch the executable. 
--Distributed memory coarrays only work with Intel® MPI; other implementations of MPI are not supported.
]]> http://software.intel.com/en-us/articles/distributed-memory-coarray-programs-with-process-pinning/ Sun, 27 Feb 2011 00:00:00 -0800 http://software.intel.com/en-us/articles/distributed-memory-coarray-programs-with-process-pinning/#comments http://software.intel.com/en-us/articles/distributed-memory-coarray-programs-with-process-pinning/ Parallel Programming Intel® Cluster Toolkit for Linux* Knowledge Base Intel® Cluster Toolkit for Windows* Knowledge Base Intel® Fortran Compiler for Linux* Knowledge Base Intel® MPI Library for Linux* Knowledge Base Intel® MPI Library for Windows* Knowledge Base Intel® Visual Fortran Compiler for Windows* Knowledge Base How to create your own fabric provider What does an Independent Hardware Vendor (IHV) have to do to enable their interconnects to work with the Intel® MPI Library?
Use the following instructions to get the Intel® MPI Library running on your own adapters:

  1. Determine if a sockets interface is provided. If yes, the interconnect will work with TCP/IP performance out of the box.
  2. If no sockets interface is available, or if you’d like to enable maximum interconnect performance and functionality, a fabric provider layer must be developed. As an interconnect vendor, you have the following options: All three APIs are used to drive RDMA (remote direct memory access)-capable fabrics like InfiniBand*, Myrinet* MX*, and Qlogic* PSM*.

What steps should be performed to build your own Direct Access Programming Library (DAPL*) provider?

  1. Obtain the DAPL software by visiting the SourceForge repository.
  2. Visit the DAT Collaborative website for additional information on DAPL or to check out other InfiniBand* DAPL implementations. You can also join the OpenFabrics Alliance* for access to even more resources.
  3. Decide on a level of DAPL to support. The suggested level is 2.0.
  4. Develop your DAPL driver. You can use the Intel MPI Library Fabric Provider writer guide for hints and ideas. The document is available free of charge as long as you are willing to sign an Intel-standard Non-Disclosure Agreement (NDA).
  5. Test the DAPL layer with the DAPL test suite that comes with the DAPL source code.
  6. Obtain the Intel MPI Library. The software can be purchased from the Intel website by visiting www.intel.com/go/mpi.
  7. Link with the Intel MPI Library and test with the Intel MPI Library test suite. The Intel MPI Library test suite is provided upon request at the Intel® Premier Web Portal.

What steps should be performed to build your own OpenFabrics Alliance (OFA*) verbs provider?

  1. Obtain the OFA verbs software by visiting the OpenFabrics download page.
  2. Visit the OpenFabrics Alliance* website for additional information on the verbs or to join the alliance.
  3. Develop your OFA verbs driver. You can use the Intel MPI Library Fabric Provider writer guide for hints and ideas. The document is available free of charge as long as you are willing to sign an Intel-standard NDA.
  4. Obtain the Intel MPI Library. The software can be purchased from the Intel website by visiting www.intel.com/go/mpi.
  5. Link with the Intel MPI Library and test with the Intel MPI Library test suite. The Intel MPI Library test suite is provided upon request at the Intel® Premier Web Portal.

What steps should be performed to build your own Tag Matching Interface (TMI) provider?

  1. Obtain the TMI API by submitting a request through the Intel® Premier Web Portal. You need to sign an NDA with Intel before we can provide you the document.
  2. Develop your TMI driver. You can use the Intel MPI Library Fabric Provider writer guide for hints and ideas. The document is available free of charge upon signing the Intel-standard NDA.
  3. Obtain the Intel MPI Library. The software can be purchased from the Intel website by visiting www.intel.com/go/mpi.
  4. Link with the Intel MPI Library and test with the Intel MPI Library test suite. The Intel MPI Library test suite is provided upon request at the Intel® Premier Web Portal.
]]> http://software.intel.com/en-us/articles/create-your-own-fabric-provider/ Sat, 24 Apr 2010 19:00:00 -0700 http://software.intel.com/en-us/articles/create-your-own-fabric-provider/#comments http://software.intel.com/en-us/articles/create-your-own-fabric-provider/ Intel® MPI Library for Linux* Knowledge Base Intel® MPI Library for Windows* Knowledge Base Release notes Note: This web page refers to the entire set of Intel® Cluster Tools.

Operating System:

SUSE* Linux Enterprise Server 9.0, SUSE* Linux Enterprise Server 10, Red Hat* Enterprise Linux 4.0, Red Hat* Enterprise Linux 5.0, Windows* Compute Cluster Server 2003, Windows* HPC Server 2008
]]> http://software.intel.com/en-us/articles/intel-cluster-tools-release-notes/ Mon, 22 Jun 2009 22:00:00 -0700 http://software.intel.com/en-us/articles/intel-cluster-tools-release-notes/#comments http://software.intel.com/en-us/articles/intel-cluster-tools-release-notes/ Intel® Cluster Toolkit for Linux* Knowledge Base Intel® Cluster Toolkit for Windows* Knowledge Base Intel® MPI Library for Linux* Knowledge Base Intel® MPI Library for Windows* Knowledge Base Intel® Trace Analyzer and Collector for Linux* Knowledge Base Intel® Trace Analyzer and Collector for Windows* Knowledge Base Tips and tricks - FAQ
  • Are there any DAPL* implementations available for Microsoft* Windows*?
  • How do you run an Intel® MPI Library application located on a network resource?
  • How do you pin processes to prevent undesired process migration?

    • Are there any DAPL* implementations available for Microsoft* Windows*?

    Yes. Find Windows OpenFabrics* (WinOF*) at OpenFabrics* Alliance website http://www.openfabrics.org/downloads/WinOF/ †

     

    How do you run an Intel® MPI Library application located on a network resource?

    Use the mpiexec -gwdir or -wdir options to specify a working directory for the application.

    For instance: mpiexec.exe -machinefile machine_file -n 4 -wdir \\host\share_folder IMB-MPI1.exe

     

    How do you pin processes to prevent undesired process migration?

    The Intel MPI Library automatically pins processes to CPUs to prevent undesired process migration. Use the I_MPI_PIN, I_MPI_PIN_MODE and I_MPI_PIN_PROCESSOR_LIST environment variables to control process pinning.

    For example: mpiexec -env I_MPI_DEVICE shm -env I_MPI_PIN enable -np 4 ./prog

    See the Intel® MPI Library for Windows Reference Manual for more details.

     

    † This link will take you off of the Intel Web site. Intel does not control the content of the destination Web Site.

    ]]>     http://software.intel.com/en-us/articles/tips-and-tricks-faq/ Wed, 17 Jun 2009 22:00:00 -0700 http://software.intel.com/en-us/articles/tips-and-tricks-faq/#comments http://software.intel.com/en-us/articles/tips-and-tricks-faq/ Intel® MPI Library for Windows* Knowledge Base     Hybrid applications: Intel MPI Library and OpenMP* I have a mixed application which uses both MPI and OpenMP* calls. Does the Intel® MPI Library support this type of hybrid functionality?

    Yes, Intel MPI does support mixed MPI/OpenMP applications. To make sure your hybrid code runs correctly, follow these steps:

    1. Use the thread safe version of the Intel MPI Library by using the -mt_mpi compiler driver option
    2. Set the I_MPI_PIN_DOMAIN environment variable to select the desired process pinning scheme. The recommended setting is omp:
      $ export I_MPI_PIN_DOMAIN=omp

      This will set the process pinning domain size to be equal to OMP_NUM_THREADS. Therefore, each MPI process can create $OMP_NUM_THREADS number of children threads for running within the corresponding domain. If OMP_NUM_THREADS is not set, each node is treated as a separate domain (which will allow as many threads per MPI processes as there are cores).

      NOTE: In order to pin OpenMP threads inside the domain, use the corresponding OpenMP feature by setting the KMP_AFFINITY environment variable.

    3. See the “Interoperability with OpenMP*” section in the Intel® MPI Library for Linux* Reference Manual for more details
    ]]>     http://software.intel.com/en-us/articles/hybrid-applications-intelmpi-openmp/ Wed, 20 May 2009 22:00:00 -0700 http://software.intel.com/en-us/articles/hybrid-applications-intelmpi-openmp/#comments http://software.intel.com/en-us/articles/hybrid-applications-intelmpi-openmp/ Intel® MPI Library for Linux* Knowledge Base Intel® MPI Library for Windows* Knowledge Base     Does Intel® MPI Library limit the number of procs on a node Question

    Does Intel® MPI Library limit the number of processes run on a single node?

    Answer

    No, it does not. Intel MPI has no internal restrictions on the number of MPI processes started on a node.

    NOTE: Be careful when running a large number of processes on a single machine. At some point, you might start oversubscribing the processor (meaning starting more processes than there are cores). This could lead to performance degradation.

    ]]>     http://software.intel.com/en-us/articles/does-intel-mpi-library-limit-the-number-of-procs-on-a-node/ Wed, 25 Feb 2009 22:00:00 -0800 http://software.intel.com/en-us/articles/does-intel-mpi-library-limit-the-number-of-procs-on-a-node/#comments http://software.intel.com/en-us/articles/does-intel-mpi-library-limit-the-number-of-procs-on-a-node/ Intel® MPI Library for Linux* Knowledge Base Intel® MPI Library for Windows* Knowledge Base