Using Intel® MKL with R

Overview

R is a programming language for statistical computing. The open source package also provides an environment for creating and running R programs. This guide will show how to use the BLAS and LAPACK libraries within Intel® Math Kernel Library (Intel® MKL) to improve the performance of R. To use other Intel MKL functions, you read this article on Extending R with Intel MKL.

Reference: http://www.r-project.org/

Prerequisites:

Build R use with Intel MKL

R can be configured to use the Intel® MKL BLAS and LAPACK by using --with-blas.

If you are building R with GNU gcc and GNU gfortran, to use the GNU OpenMP* threaded version of Intel® MKL in R on a Linux operating system add the following:

source /opt/intel/mkl/bin/mklvars.sh intel64

MKL="-Wl,--no-as-needed -lmkl_gf_lp64 -Wl,--start-group -lmkl_gnu_thread  -lmkl_core  -Wl,--end-group -fopenmp  -ldl -lpthread -lm"

./configure --with-blas="$MKL" --with-lapack

The order of the libraries is important. --with-lapack will cause the LAPACK from Intel MKL to be used as well.

The intel® MKL libraries link line actually is varied based on different compiler and threading model.  Developer can select the right libraries by the tool named Intel® MKL Link Advisor to suit their own needs

( MKL Link Advisor webpage version: https://software.intel.com/sites/products/mkl/mkl_link_line_advisor.htm)

Here are some other variations:

A: If you are building R with Intel C/C++ Compiler icc and Intel Fortran Compiler ifort, then please use 

source /opt/intel/compilers_and_libraries_2018.x.xxx/linux/bin/compilervars.sh intel64/mklvars.sh intel64

MKL="   -Wl,--no-as-needed   -Wl,--start-group \

          -lmkl_intel_lp64       \

          -lmkl_intel_thread    \

          -lmkl_core          \

      -Wl,--end-group                           \

      -liomp5 -lpthread  -lm"

	./configure --with-blas="$MKL" --with-lapack 

or

source /opt/intel/compilers_and_libraries_2018.x.xxx/linux/bin/compilervars.sh intel64/mklvars.sh intel64

MKL="-Wl,--no-as-needed   -lmkl_rt  -liomp5 -lpthread  -lm"

	./configure --with-blas="$MKL" --with-lapack 

The default number of threads will equal the number of physical cores on the system, but can be controlled by setting OMP_NUM_THREADS or MKL_NUM_THREADS.

B. You can use the following if you want to use a sequential version of Intel® MKL and GNU GFORTRAN and GCC:


source /opt/intel/compilers_and_libraries_2018.x.xxx/linux/bin/compilervars.sh intel64

MKL=" -L$(MKLROOT)/lib/intel64                       \

      -Wl,--start-group                         \

          -lmkl_gf_lp64       \

          -lmkl_sequential    \

          -lmkl_core          \

      -Wl,--end-group"

	

C. The following will link Intel® MKL statically the GNU threading runtime:


source /opt/intel/compilers_and_libraries_2018.x.xxx/linux/bin/compilervars.sh intel64

MKL=" -L$(MKLROOT)/lib/intel64                       \

      -Wl,--start-group                         \

         ${MKL_LIB_PATH}/libmkl_gf_lp64.a       \

         ${MKL_LIB_PATH}/libmkl_gnu_thread.a    \

         ${MKL_LIB_PATH}/libmkl_core.a          \

      -Wl,--end-group                           \

      -lgomp -lpthread"

Check if Intel® MKL is working

Check the config.log to see if Intel® MKL was working, 

configure:29075: checking for dgemm_ in -L/opt/intel/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64/ -lmkl_gf_lp64 -lmkl_sequential -lmkl_core
configure:29096: gcc -std=gnu99 -o conftest -g -O2  -I/usr/local/include  -L/usr/local/lib64 conftest.c -L/opt/intel/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64/ -lmkl_gf_lp64 -lmkl_sequential -lmkl_core  -lgfortran -lm -lrt -ldl -lm  >&5
conftest.c: In function 'main':
conftest.c:210: warning: implicit declaration of function 'dgemm_'
configure:29096: $? = 0
configure:29103: result: yes
configure:29620: checking whether double complex BLAS can be used
configure:29691: result: yes
configure:29711: checking whether the BLAS is complete

Or after build, If MKL library was dynamically linked in the R, type the simple command to verify this

  

$ ldd  ./R-3.4.2/lib/libR.so

	        linux-vdso.so.1 =>  (0x00007fffa45ff000)

	        libmkl_gf_lp64.so => /opt/intel/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64/libmkl_gf_lp64.so (0x00007fe963097000)

	        ......

Performance Results

To provide some indication of the performance improvements that this can provide, I ran R-benchmark-25.R found on the R benchmarks site on a system with a 32-core Intel® Core™ i7-975 processor Extreme Edition processor (8MB LLC, 3.33GHz) with 6GB of RAM and running RHEL 6 x86_64. I first installed R-2.15.3 and ran the benchmark without Intel MKL. to get:

175.825 sec

After installing Intel MKL 11.0 update 2, and following the methods above I was able to get:

17.289 sec

Known Issues

1.  Intel MKL blocks the use of FFTW in R 

Problem: When Intel MKL library was linked in R as the BLAS,  The MKL libraries like libmkl_rt.so also integrated FFTW3  interface by default.  As not all of FFTW3 are implemented in library, it may cause some R extension based on FFTW* failed. Please see the threads.  https://software.intel.com/en-us/forums/intel-math-kernel-library/topic/781812

Solution:  Build libRblas.so Using the Custom Shared Object Builder.  Intel MKL provide the tools under install folder : 

> cp -r /opt/intel/mkl/tools/builder .

>cd builder; source /opt/intel/mkl/bin/mklvars.sh intel64; make libintel64 export=blas_example_list parallel=gnu name=libRblas

2.  Intel OpenMP library hang R when upgrade to 2018 update 1 &2018 update 2.  

Problem: Please see the threads https://software.intel.com/en-us/forums/intel-math-kernel-library/topic/760830,  It is libiomp5.so bug, so affect R with Intel MKL, which based on the libiomp5 libraries. 

Solution: The issue is fixed in 2018 update 3. 

For more complete information about compiler optimizations, see our Optimization Notice.

6 comments

Top
kiran s.'s picture

I have compiled with the same mkl library as shown below

its giving me the error while make install

./configure --with-blas="$MKL" --with-lapack="$MKL" --prefix=/home1/cdac/R-lang/mkl_thread CC=icc CXX=icpc FC=ifort F77=ifort

ERROR BELOW

 

make[4]: Leaving directory `/home1/cdac/R-lang/R-3.1.2/src/library/tools'
make[4]: Entering directory `/home1/cdac/R-lang/R-3.1.2/src/library/tools'
make[5]: Entering directory `/home1/cdac/R-lang/R-3.1.2/src/library/tools/src'
../../../../etc/Makeconf:8: *** missing separator (did you mean TAB instead of 8 spaces?).  Stop.
make[5]: Leaving directory `/home1/cdac/R-lang/R-3.1.2/src/library/tools/src'
make[4]: *** [mksrc] Error 1
make[4]: Leaving directory `/home1/cdac/R-lang/R-3.1.2/src/library/tools'
make[3]: *** [all] Error 2
make[3]: Leaving directory `/home1/cdac/R-lang/R-3.1.2/src/library/tools'
make[2]: *** [R] Error 1
make[2]: Leaving directory `/home1/cdac/R-lang/R-3.1.2/src/library'
make[1]: *** [R] Error 1
make[1]: Leaving directory `/home1/cdac/R-lang/R-3.1.2/src'
make: *** [R] Error 1

 

 

Prad B.'s picture

Todd,

Great article. A lot of people would like to use Intel MKL, however they find it difficult to integrate it with R or Python themselves, particularly with Windows.

Not sure if it is feasible for you, but it would be great if

1. You can write a similar article to show how to build R or Python with Intel MKL in Windows

2. Or, alternatively, instead of selling Intel MKL licenses for Windows, you can sell a license for R or Python binaries built with Intel MKL. That way Intel can sell much more of  Intel MKL licenses.

Thanks

 

steven v.'s picture

Hi,  Thank you for the hint;

yes it is gcc with gfortran; calling composer made no difference

excerpt from shell env:

CPATH=/opt/intel/composer_xe_2013_sp1.0.080/mkl/include
PATH=/opt/intel/composer_xe_2013_sp1.0.080/bin/intel64:/opt/intel/composer_xe_2013_sp1.0.080/mpirt/bin/intel64:/home/steven/bin/ec2/bin:/home/steven/bin:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games

MKL=-Wl,--start-group -lmkl_gf_lp64 -lmkl_gnu_thread -lmkl_core -Wl,--end-group -lgomp -lpthread

INCLUDE=/opt/intel/composer_xe_2013_sp1.0.080/mkl/include

./configure --with-blas="$MKL" --with-lapack

checking for cblas_cdotu_sub in vecLib framework... no
checking for dgemm_ in -Wl,--start-group -lmkl_gf_lp64 -lmkl_gnu_thread -lmkl_core -Wl,--end-group -lgomp -lpthread... no
checking for dgemm_... no
checking for ATL_xerbla in -latlas... no
checking for dgemm_ in -lblas... yes
checking for dgemm_ in -ldgemm... no
checking for dgemm_ in -lblas... (cached) yes

direct:

checking for cblas_cdotu_sub in vecLib framework... no
checking for dgemm_ in  -DMKL_ILP64 -m64 -I/opt/intel/composer_xe_2013_sp1.0.080/mkl/include -L/opt/intel/composer_xe_2013_sp1.0.080/mkl/lib/intel64 -lmkl_gf_ilp64 -lmkl_gnu_thread -lmkl_core -lgomp -ldl -lpthread -lm... no
checking for dgemm_... no

linked against libblas :(

    linux-vdso.so.1 =>  (0x00007fff2f2ea000)
    libblas.so.3 => /usr/lib/libblas.so.3 (0x00007f8daec19000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f8dae915000)
    libreadline.so.6 => /lib/x86_64-linux-gnu/libreadline.so.6 (0x00007f8dae6d2000)
    liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f8dae4b0000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f8dae2a8000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f8dae0a3000)
    libgomp.so.1 => /usr/lib/x86_64-linux-gnu/libgomp.so.1 (0x00007f8dade94000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f8dadc77000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8dad8ae000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f8daf206000)
    libgfortran.so.3 => /usr/lib/x86_64-linux-gnu/libgfortran.so.3 (0x00007f8dad596000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f8dad380000)
    libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007f8dad156000)
    libquadmath.so.0 => /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007f8dacf1a000)

thanks, steve

 

 

 

 

 

Ying H. (Intel)'s picture

Hi Steven, 

I saw you used compilervars.sh intel64, do you have intel compiler installed? is it intel fortran compiler or C compiler?  on the other hand, from message, you mentioned, gcc , so  I guess you use gcc + gfortran, right? you may try call composer_xe_2013.x.xx/mkl/bin//mklvars.sh intel64  or link all library explicitly. 

Best Regards,

Ying 

steven v.'s picture

 

hello, for some reasons I can't get R linked with MKL;  can someone tell me where I am wrong?

>source /opt/intel/bin/compilervars.sh intel64 

>export MKL="-Wl,--start-group -lmkl_gf_lp64 -lmkl_gnu_thread -lmkl_core -Wl,--end-group -lgomp -lpthread"

>./configure --with-blas="$MKL" --with-lapack

gcc (Ubuntu 4.8.1-2ubuntu1~12.04) 4.8.1

composer_xe_2013_sp1.0.080

 

checking whether ctanh exists and is declared... yes
checking for cblas_cdotu_sub in vecLib framework... no
checking for dgemm_ in -Wl,--start-group -lmkl_gf_lp64 -lmkl_intel_thread -lmkl_core -Wl,--end-group -liomp5 -lpthread... no
checking for dgemm_... no
checking for ATL_xerbla in -latlas... no
checking for dgemm_ in -lblas... no
checking for dgemm_ in -lblas... (cached) no
checking for dgemm_ in -lblas... (cached) no

 

Ying H. (Intel)'s picture

For troubleshooting, please read the two forum threads:

http://software.intel.com/en-us/forums/topic/397399

http://software.intel.com/en-us/forums/topic/326016

WBR
Ying

Add a Comment

Have a technical question? Visit our forums. Have site or software product issues? Contact support.