Intel MKL 11.3 introduced a wrapper code for MPI interface to Intel MKL, which can help users to use Intel® MKL cluster functions with the customized MPI libraries.
While different MPI libraries are compatible on the application programming interface (API) level, they are often incompatible at the application binary interface (ABI) level. So Intel MKL provides some different libraries to support the different MPIs. For example, one should link with libmkl_blacs_lp64.a to use application with MPICH*, libmkl_blacs_openmpi_lp64.a to use Open MPI*. If users link Intel MKL cluster functions with the customized MPI libraries, which is not supported by Intel MKL, and may create some unexpected result.
Note, only the Intel MKL cluster functions (Cluster Sparse Solver, Cluster FFT, Scala PACK, BLACS) depend on the different MPI implementation. For other MKL functions, they do not have such dependency.
Intel MKL 11.3 added a MPI wrapper code, so user can extend Intel MKL cluster functions usage to some customized MPI libraries. Users can build custom BLACS library using wrapper code to extend MKL Cluster functions’ support on their MPI libraries.
2. Using Intel MKL MPI wrapper code
2.1 Build the Intel MKL wrapper code.
The MKLMPI adaptor is provided with the source code, which can be found in MKLROOT/interfaces/mklmpi directory. To build custom BLACS one should run make (nmake) from this directory with corresponding parameters. For example:
>make sointel64 interface=ilp64 MPICC=’mpicc’
This command will build custom dynamic MKL BLACS library with default library name libmkl_blacs_custom_ilp64.so and put it in default directory: ../../lib/intel64. For more information, please use “make help“ (or “nmake help” on Windows).
2.2 Linking with custom MKL BLACS library
To link with custom MKL BLACS library, you only need to replace the default Intel MKL BLACS with the custom one (for example, replace mkl_blacs_intelmpi with mkl_blacs_custom).
Note that, when you link with the dynamic library at Windows* system, you should use mkl_blacs_lp64_dll.lib, but before run the application, it needs to set MKL_BLACS_MPI environment variable. For example, it needs to set to CUSTOM for the default name of the custom MKL BLACS library (mkl_blacs_custom_lp64.dll for LP64 interface and mkl_blacs_custom_ilp64.dll for ILP64 case), or set to any other filename that you use for the custom MKL BLACS library.
3. An example: Using Intel MKL MPI wrapper with Open MPI* at Windows*
Suppose the Open MPI is installed at C:\Program Files (x86)\OpenMPI_v1.6.2-x64. The %MPIROOT% environment variable is set to the installed path
3.1 Build custom MKL BLACS library:
\>nmake libintel64 MPICC="mpicc -DOMPI_IMPORTS -DOPAL_IMPORTS -DORTE_IMPORTS" INSTALL_DIR=%MKLROOT%\lib\intel64 (**)
3.2 Using the library with the Intel MKL example:
*) we can also set MKL_BLACS_MPI=mkl_blacs_custom_lp64.dll
**) Using compiler flags -DOMPI_IMPORTS -DOPAL_IMPORTS -DORTE_IMPORTS was needed because of peculiarity of our Open MPI* installation, and may be or may be not needed in each specific case.
Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.
Notice revision #20110804