Building VASP* with Intel® MKL and Intel® Compilers
Step 1 – Overview
This guide is intended to help users on how to build VASP (Vienna Ab-Initio Package Simulation) using Intel® Math Kernel Library (Intel® MKL) and Intel® Compilers and Intel® MPI.
VASP is a package for performing ab-initio quantum-mechanical molecular dynamics (MD) using pseudo potentials and a plane wave basis set. The approach implemented in VAMP/VASP is based on a finite-temperature local-density approximation (with the free energy as variational quantity) and an exact evaluation of the instantaneous electronic ground state at each MD-step using efficient matrix diagonalization schemes and an efficient Pulay mixing. These techniques avoid all problems occurring in the original Car-Parrinello method which is based on the simultaneous integration of electronic and ionic equations of motion. The interaction between ions and electrons is described using ultrasoft Vanderbilt pseudopotentials (US-PP) or the projector augmented wave method (PAW). Both techniques allow a considerable reduction of the necessary number of plane-waves per atom for transition metals and first row elements. Forces and stress can be easily calculated with VAMP/VASP and used to relax atoms into their instantaneous groundstate. [Ref: VASP]
This application note was created to help users who benchmark clusters using VASP to also incorporate the latest version of Intel® MKL on Linux platforms on Xeon systems. Specifically, we'll address Intel® MKL version 11.1
More information on VASP can be found from http://cms.mpi.univie.ac.at/vasp/
Step 2 – Downloading VASP Source Code
VASP is not public-domain or share-ware, and will be distributed only after a license contract has been signed. Please visit VASP homepage to know more details on obtaining the license.
- Intel® MKL contains highly optimized FFT and also the wrappers for FFTW, which can be obtained from the following options:
- Download a FREE evaluation version of the Intel® MKL product.
- Download the FREE non-commercial* version of the Intel® MKL product.
- Purchase a copy of Intel® MKL
All of these can be obtained at: http://www.intel.com/software/products/mkl.
- Intel® C++ and FORTRAN compilers can be obtained from http://www.intel.com/softwre/products/compilers.
- Intel MPI can be obtained from http://www.intel.com/software/products/cluster.
Open source MPI (MPICH2) can be obtained from http://www-unix.mcs.anl.gov/mpi/mpich/ .
Step 3 - Configuration
Use the following commands to extract the VASP files:
$tar –xvzf vasp.tgz vasp.lib.tgz
This will create vasp and vasp.lib directories
Make sure that MPI, C++ and FORTRAN compilers are installed and in the PATH. Also set LD_LIBRARY_PATH to your compiler (C++ and FORTRAN), MPI, and Intel® MKL libraries.
Note: This application note is written specifically for use with the Intel compilers and MPI.
Step 4 – Building VASP
- Build FFTW wrapper library for Intel® MKL
Change directory to <your MKL installation>/interfaces/fftw3xf
Assuming default installation
Build the fftw 3.x FORTRAN wrapper library for Intel® 64 (previously known as em64t) platform using the Intel compilers.
$make libintel64 compiler=intel
This will create the fftw wrapper library “libfftw3xf_intel.a” in lib/em64t directory of your Intel® MKL installation.
- Build libdmy.a
Change directory to vasp.x.x.lib
Modify the makefile.linux_ifc_P4 Makefile to point to the correct Intel FORTRAN compiler.
Run the following command from vasp.x.x.lib, using the Makefile for linux using the Intel compiler.
$make –f makefile.linux_ifc_P4
After a successful compilation, libdmy.a will be built in the same directory.
- Build VASP
Change directory to vasp.x.x
Export the environment variable MPIFC to point to the Intel MPI Fortran compiler mpiifort.
Edit the makefile.linux_ifc_P4 to link with Intel® MKL libraries and change the Fotran compiler
Under FORTRAN compiler and linker part of the makefile
CPP = $(CPP_) -DMPI -DHOST=\"LinuxIFC\" -DIFC \ -DCACHE_SIZE=32000 -DPGF90 -Davoidalloc -DNGZhalf \ -DMPI_BLOCK=64000 -DscaLAPACK -Duse_collective -Ddo_loops –DMKL_ILP64
Change the FORTRAN flags section as shown here.
FFFLAGS = -FR -names lowercase -assume byterecl -I$(MKLROOT)/include/fftw –xAVX
MKLROOT=/opt/intel/composer_xe_2013_sp1/mkl MKL_PATH=$(MKLROOT)/lib/intel64 MKL_FFTW_PATH=$(MKL_PATH)/interfaces/fftw3xf BLAS= -mkl=cluster LAPACK= -mkl=cluster … .…… …………
Comment out the existing FFT3D line
#FFT3D = fftdfurth.o fftdlib.o
FFT3D= fftmpiw.o fftmpi_map.o fftw3d.o fft3dlib.o $(MKL_FFTW_PATH)/libfftw3xf_intel.a INCS = -I$(MKLROOT)/include/fftw
To use Intel® MKL ScaLAPACK, modify additional options for CPP for parallel version adding
-DscaLAPACK to the CPP line as mentioned above and specify Intel® MKL ScaLAPACK as below.
Run the following command to build vasp
$make –f makefile.linux_ifc_P4
This will create the VASP executable in the current directory.
Step 5 - Running VASP
Run vasp by executing mpiexec command with your required parameters. Below, for e.g., to run 32 processes use as shown with your workloads.
$mpiexec –n 32 ./vasp
Appendix B - Known Issues and Limitations
There could be a compilation error on nonlr.F file as reported below.
mpif90 -fc=ifort -FR -names lowercase -assume byterecl -O2 -ip -I/opt/intel/compiler/2013_sp1.1.106/composer_xe_2013_sp1.1.106/mkl/include/fftw -c nonlr.f90 nonlr.F(3069): error #6404: This name does not have a type, and must have an explicit type. [LM] DO LM=1,LMMAXC nonlr.F(3069): error #6404: This name does not have a type, and must have an explicit type. [LM] DO LM=1,LMMAXC ----------------^ nonlr.F(3069): error #6063: An INTEGER or REAL data type is required in this context. [LM] DO LM=1,LMMAXC ----------------^ nonlr.F(3206): error #6404: This name does not have a type, and must have an explicit type. [LM] DO LM=1,LMMAXC ----------------^ nonlr.F(3206): error #6063: An INTEGER or REAL data type is required in this context. [LM] DO LM=1,LMMAXC ----------------^
By applying the patch mentioned below to the nonlr.F, this compilation error can be fixed.
3002c3002 < INTEGER IP, LMBASE, ISPIRAL, ISPINOR, NLIIND, NIS, NT, LMMAXC, NI, INDMAX, L, LM, IND --- > INTEGER IP, LMBASE, ISPIRAL, ISPINOR, NLIIND, NIS, NT, LMMAXC, NI, INDMAX, L, IND 3144c3144 < INTEGER IP, LMBASE, ISPIRAL, ISPINOR, NLIIND, NIS, NT, LMMAXC, NI, INDMAX, L, LM, IND --- > INTEGER IP, LMBASE, ISPIRAL, ISPINOR, NLIIND, NIS, NT, LMMAXC, NI, INDMAX, L, IND
Appendix C – References