Forthcoming Intel® MKL 11.0 (Expected date of release: Autumn 2012)

•  Intel® MKL 11.0 will have backward incompatibility with Intel® MKL 10.2 update 3

•  Open MP static library on Microsoft* Windows

•  GMP* Arithmetic functions

•  Intel® MKL Reference Manual will be removed from product package

•  Intel® Pentium III will no longer be supported

•  Red Hat* EL4 and PGI* Fortran 10.x support will be dropped

•  PGI* Fortran 77 support will be removed

• a powerful N-dimensional array object
• sophisticated (broadcasting) functions
• tools for integrating C/C++ and Fortran code
• useful linear algebra, Fourier transform, and random number capabilities.

Besides its obvious scientific uses, NumPy can also be used as an efficient multi-dimensional container of generic data.

For more information on NumPy, please visit http://NumPy.scipy.org/

Version Information

This application note was created to help NumPy users to make use of the latest versions of Intel MKL on Linux platforms.

The instructions given in this articles apply to Intel MKL 10.3 and above and Intel Compiler 11.0 and above.

 
Step 2 - Downloading NumPy Source Code

The NumPy source code can be downloaded from:

http://NumPy.scipy.org/

Prerequisites

Intel MKL can be obtained from the following options:

Download a FREE evaluation version of the Intel MKL product.
Download the FREE non-commercial* version of the Intel MKL product.

All of these can be obtained at: Intel® Math Kernel Library product web page.

Intel® MKL is also bundled with the following products

Intel® Parallel Studio XE 2011
Intel Parallel Composer XE 2011
Intel Cluster Studio 2011

Step 3 - Configuration

Use the following commands to extract the NumPy tar files from the downloaded NumPy-x.x.x.tar.gz.

1.	$gunzip numpy-x.x.x.tar.gz 
2.	$tar -xvf numpy-x.x.x.tar 

 
The above will create a directory named numpy-x.x.x

Make sure that C++ and FORTRAN compilers are installed and they are in PATH. Also set LD_LIBRARY_PATH to your compiler (C++ and FORTRAN), and MKL libraries.

Step 4 - Building NumPy

Change directory to numpy-x.x.x
Create a site.cfg from the existing one
 
Edit site.cfg as follows:

Add the following lines to site.cfg in your top level NumPy directory to use Intel® MKL, if you are building on Intel 64 platform:

[mkl]
library_dirs = /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64
include_dirs = /opt/intel/composer_xe_2011.sp1.6.233/mkl/include
mkl_libs = mkl_rt
lapack_libs =

If you are building NumPy for 32 bit, please add as the following

[mkl]
library_dirs = /opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/ia32
include_dirs = /opt/intel/composer_xe_2011_sp1.6.233/mkl/include
mkl_libs = mkl_rt
lapack_libs = 

Modify cc_exe in numpy/distutils/intelccompiler.py to be something like:

self.cc_exe = 'icc -O3 -g -fPIC -fp-model strict -fomit-frame-pointer -openmp -xhost'

Here we use, -O3, optimizations for speed and enables more aggressive loop transformations such as Fusion, Block-Unroll-and-Jam, and collapsing IF statements, -openmp for OpenMP threading and -xhost option tells the compiler to generate instructions for the highest instruction set available on the compilation host processor. If you are using the ILP64 interface, please add -DMKL_ILP64 compiler flag.

Run icc --help for more information on processor-specific options, and refer Intel Compiler documentation for more details on the various compiler flags.

Compile and install NumPy with the Intel compiler: (on 64-bit platforms replace "intel" with "intelem")

python setup.py config --compiler=intel build_clib --compiler=intel build_ext --compiler=intel install

If you build NumPY for Intel64 bit platforms:

$export LD_LIBRARY_PATH=/opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/compiler/lib/intel64:$LD_LIBRARY_PATH

If you build NumPY for ia32 bit platforms:

$export LD_LIBRARY_PATH=/opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/ia32:/opt/intel/composer_xe_2011_sp1.6.233/compiler/lib/ia32:$LD_LIBRARY_PATH

It is possible that LD_LIBRARY_PATH causes a problem, if you have installed MKL and Composer XE in other directories than the standard ones. The only solution I've found that always works is to build Python, NumPy and SciPy inside an environment where you've set the LD_RUN_PATH variable, e.g:

$export LD_RUN_PATH=~/opt/lib:~/intel/composer_xe_2011_sp1.6.233/compiler/lib:~/intel/composer_xe_2011_sp1.6.233/mkl/lib/ia32

Note: We recommend users to use arrays with 'C' ordering style which is row-major, which is default than Fortran Style which is column-major, and this is because NumPy uses CBLAS and also to get better performance.

Appendex A: Example:

Please see below an example Python script for matrix multiplication that you can use Numply installed with Intel MKL which has been provided for illustration purpose.

import numpy as np
import time

N = 6000
M = 10000

k_list = [64, 80, 96, 104, 112, 120, 128, 144, 160, 176, 192, 200, 208, 224, 240, 256, 384]

def get_gflops(M, N, K):
	return M*N*(2.0*K-1.0) / 1000**3

np.show_config()

for K in k_list:
	a = np.array(np.random.random((M, N)), dtype=np.double, order='C', copy=False)
	b = np.array(np.random.random((N, K)), dtype=np.double, order='C', copy=False)
	A = np.matrix(a, dtype=np.double, copy=False)
	B = np.matrix(b, dtype=np.double, copy=False)

	C = A*B

	start = time.time()

	C = A*B
	C = A*B
	C = A*B
	C = A*B
	C = A*B

	end = time.time()

	tm = (end-start) / 5.0

	print "{0:4}, {1:9.7}, {2:9.7}".format(K, tm, get_gflops(M, N, K) / tm)

Appendix B: Performance Comparison

Please click Examples.py to download the examples for LU, Cholesky and SVD.



 

Introduction : 

By default the Intel(R) Debugger requires the ability to open either a GUI or at least a dedictaed separate xterm window. This article describes how to launch a command line verion of MPI enabled IDB in a command line only terminal window.


Version :
This article applies to the Intel(R) Debugger for Linux* v11.0 and higher
The proposed command ine and testing of this approach was based on Intel(R) MPI Library 3.x, but newer versions should work as well.
The screen command needs to be available

Configuration Set Up :
The usage model this article applies to is as follows:

- network based Linux MPI cluster with application possibly running distributed over several nodes

- only available access to the cluster is via remote terminal login (puTTY, ssh, ...)

- no X window output redirection to the login machine is possible 

- Please ensure to IDB_PARALLEL_SHELL set to enable multi-node debugging

"export IDB_PARALLEL_SHELL=/usr/bin/ssh"

Solution:

- Edit mpiexec.py launch script in the bin directory of your Intel(R) MPI Library installation
- Launch debugger with "screen mpiexec -idb -n <node number> <debuggee executable>"

Python Script Changes :

1. In the line that defines dbg_cmd replace "xterm" with "screen" and remove "-e" in mpiexec.py. In addition remove the "&" at the end of the line.

E.g. replace the line

dbg_cmd = 'xterm -e ' + tv_exe + ' -pid ' + `os.getpid()` + ' -mpi2 -parallel /usr/bin/python &'

with

dbg_cmd = 'screen' + tv_exe + ' -pid ' + `os.getpid()` + ' -mpi2 -parallel /usr/bin/python'

2. Replace the line

os.system(dbg_cmd)

with the following lines:

cpid = os.fork()

if (cpid == 0):
os.system(dbg_cmd)
sys.exit(0)

#include <stdio.h>
#include “mkl_vsl.h”
 
int main()
{
   double r[1000]; /* buffer for random numbers */
   double s; /* average */
   VSLStreamStatePtr stream;
   int i, j;
    
   /* Initializing */        
   s = 0.0;
   vslNewStream( &stream, VSL_BRNG_SFMT19937, 777 );
    
   /* Generating */        
   for ( i=0; i<10; i++ );
   {
      vdRngGaussian( VSL_RNG_METHOD_GAUSSIAN_ICDF, stream, 1000, r, 5.0, 2.0 );
      for ( j=0; j<1000; j++ );
      {
         s += r[j];
      }
   }
   s /= 10000.0;
    
   /* Deleting the stream */        
   vslDeleteStream( &stream );
    
   /* Printing results */        
   printf( “Sample mean of normal distribution = %f\n”, s );
    
   return 0;
}

This guide is intended to help the current GROMACS users to potentialy improve performance by utilizing Intel® Math Kernel Library (Intel® MKL). This article explains how to build 64-bit GROMACS with Intel MKL for Intel64 based applications.

GROMACS is a versatile package to perform molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles.

[Reference: http://www.gromacs.org/About_Gromacs *]

Version Information

This application note was created to help users who use GROMACS to also incorporate the latest versions of Intel MKL 10.3.x or higher on Linux* platforms on Intel® Xeon® * processor-based systems. Specifically, we'll address Intel MKL version 10.3.1.

This application note applies to Intel® Composer XE 2011 and GROMACS 4.0.7.

Step 2- Downloading GROMACS Source Code

Prerequisites:

1. Intel® MKL contains highly optimized BLAS, LAPACK, and FFT and also the wrappers for FFTW, To get evaluation/ non -commercial/ licensed copy of Intel MKL, refer to  Intel® Math Kernel Library product Page.
2. To get evaluation/non-commercial/licensed copy of Intel compilers, refer to  Intel C++ and Fortran compilers product page (Intel MKL and Intel Compilers are now available in a single package - Intel Composer XE 2011).
3. To get GROMACS - http://www.gromacs.org/

Note: This application note is written specifically for use with the Intel Compilers. Though using Intel MKL with GROMACS built with other compilers is also possible.

Unpack:

 Use the following commands to extract the GROMACS files:

 tar -xf gromacs-x.x.x.tar.gz

This will create a directory called gromacs-x.x.x. 

Step 3 - Building GROMACS

Update configure:

Modify configure.ac  from

mkl*)

###########

# Intel Math Kernel Library version 6 and later.

##########

AC_MSG_CHECKING([for mkl_dfti.h])

AC_TRY_COMPILE([#include<mkl_dfti.h>],,AC_MSG_RESULT(yes),AC_MSG_ERROR([Cannot find mkl_dfti.h header from Intel Math Kernel Library>=6.0.]\

))

## Check for library

# AC_CHECK_LIB([guide],main,,AC_MSG_ERROR([Cannot find libguide (Intel MKL)]))

AC_CHECK_LIB([mkl],DftiComputeForward,,

AC_MSG_ERROR([Cannot find Intel Math Kernel Library >= 6.0]))

MDLIB_LIBOBJS="$MDLIB_LIBOBJS gmx_fft_mkl.lo"

;;

to

mklfftw3*)

###########

# FFTW3 wrappers in Intel Math Kernel Library version 6 and later.

##########

AC_MSG_CHECKING([for fftw3.h])

AC_TRY_COMPILE([#include<fftw3.h>],,AC_MSG_RESULT(yes),AC_MSG_ERROR([Cannot find fftw3.h header from Intel Math Kernel Library>=6.0.]\

))

## Check for library

AC_CHECK_LIB([mkl_intel_ilp64],fftw_execute,LIBS="-lmkl_intel_thread -lmkl_core -liomp5 -lpthread $LIBS",

AC_MSG_ERROR([Cannot find Intel Math Kernel Library >= 6.0]),[-lmkl_intel_ilp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread])

MDLIB_LIBOBJS="$MDLIB_LIBOBJS gmx_fft_fftw3.lo"

;;

mkl*)

###########

# Intel Math Kernel Library version 6 and later.

##########

AC_MSG_CHECKING([for mkl_dfti.h])

AC_TRY_COMPILE([#include<mkl_dfti.h>],,AC_MSG_RESULT(yes),AC_MSG_ERROR([Cannot find mkl_dfti.h header from Intel Math Kernel Library>=6.0.]\

))

## Check for library

AC_CHECK_LIB([mkl_intel_ilp64],DftiComputeForward,LIBS="-lmkl_intel_thread -lmkl_core -liomp5 -lpthread $LIBS",

AC_MSG_ERROR([Cannot find Intel Math Kernel Library >= 6.0]),[-lmkl_intel_ilp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread])

MDLIB_LIBOBJS="$MDLIB_LIBOBJS gmx_fft_mkl.lo"

;;

Run autoreconf without any parameters. Autoconf (that includes autoreconf) could be found here http://www.gnu.org/software/autoconf/

Set environment:

• Make sure that C/C++ and Fortran compilers are installed properly and <PATH> environment variable is set to compiler installation directory.
• Set <LD_LIBRARY_PATH> to compiler(C/C++ or Fortran) and Intel MKL library installation directory.
• Make sure that  <MKLROOT>  environment variable to point to the MKL installation directory on your system.
• Another easy option to configure Intel Compilers and Intel MKL environment to build 64-bit application is to run mklvars.sh script file, which is part of <MKLROOT>/bin directory
• Also make sure that other environment variables like CC, CXX, F77, CFLAGS, CPPFLAGS, LDFLAGS are set as below.

export CC=icc

export CXX=icc

export F77=ifort

export CFLAGS="-O3 -ipo- -static -std=c99 -fPIC -DMKL_LP64 -DM_PI=3.1415926535897932384"

export CPPFLAGS="-I$MKLROOT/include -I$MKLROOT/include/fftw"

export LDFLAGS="-L$MKLROOT/lib/intel64 -L$MKLROOT/../compiler/lib/intel64"

export LD_LIBRARY_PATH="$MKLROOT/lib/intel64:$MKLROOT/../compiler/lib/intel64:$LD_LIBRARY_PATH"

Configure and make:

Build GROMACS and install it in the standard place

configure --with-fft=mklfftw3

make

make install 

Step 4 - Running GROMACS

Add standard GROMACS path to environment variable <PATH>

You should be able to run GROMACS. For more information about GROMACS, refer to http://www.gromacs.org/About_Gromacs*

The release of PARDISO 4.0.0 from the University of Basel is not backward compatible and thus introduces some incompatibilities with the implementation of PARDISO that is available with the Intel® Math Kernel Library. This article outlines those places where the two interfaces have diverged.