| May 3, 2009 9:00 AM PDT | |
Step 1 – Overview
Summary
This document describes how to build the Portable Extensible Toolkit for Scientific Computation (PETSc) with Intel® Math Kernel Library (Intel® MKL) BLAS and LAPACK.
PETSc is a set of libraries that provides functions for building high-performance large-scale applications. PETSc library includes routines for vector manipulation, sparse matrix computations, distributed arrays, linear and non-linear solvers, and extensible PDE solvers.
This application note focuses on building PETSc for Intel® Architecture Processors including IA-32 and Intel® 64 architecture, running Linux*.
Version Information
This document applies to Intel MKL 10.3 and Intel Composer XE for Linux* and PETSc 3.1-p8.
Step 2 – Download
PETSc releases are available for download from the PETSc web site at http://www.mcs.anl.gov/petsc/petsc-as/
To purchase or download a 30-day evaluation version of Intel MKL go to http://software.intel.com/en-us/intel-mkl/.
Step 3 – Configuration
- Use the following command to extract the PETSc files. A new folder petsc-3.1-p8 will be created:
$ tar –xvzf petsc-3.1-p8.tar.gz
- Change to the PETSc folder:
$ cd petsc-3.1-p8
- Set the PETSC_DIR environment variable to point to the location of the PETSc installation folder. For bash shell use:
$ export PETSC_DIR=$PWD
Step 3 – Build PETSc
PETSc includes a set of python configuration files which support the use of various compilers, MPI implementations and math libraries. The examples below show options for configuring PETSc to link to Intel MKL BLAS and LAPACK functions. Developers need to ensure that other options are configured appropriately for their system. See the PETSc installation documentation for details.
- Invoke the configuration script with the following options to build PETSc with Intel MKL 10.3 or Intel Composer XE (installed to the default location /opt/intel/mkl).
- For Intel processors with Intel 64 use the following option:
$ ./config/configure.py
...
--with-blas-lapack-dir=/opt/intel/composerxe-2011/mkl/lib/intel64 - For Intel 32-bit processors use the following options:
$ ./config/configure.py
...
--with-blas-lapack-dir==/opt/intel/composerxe-2011/mkl/lib/ia32
- For Intel processors with Intel 64 use the following option:
- Use the make file to build PETSc:
$ make all
Step 4 - Run PETSc
Run the PETSc tests to verify the build worked correctly:
$ make test
Appendix A – System Configuration
PETSc build and testing was completed on a system with an Intel® Xeon® CPU X5570 running RedHat Enterprise Linux* 5.3
Appendix B - References
This article applies to: Intel® Math Kernel Library Knowledge Base
For more complete information about compiler optimizations, see our Optimization Notice.
Comments (4) 
| July 19, 2009 6:14 PM PDT
Anthony Skjellum | [Second submit is just to correct e-mail address; sorry for mistyped e-mail address in first submission.] |
| July 24, 2009 3:04 PM PDT
AmandaS (Intel)
|
The behavior you've described, listed below, is suspicious and will be treated as a performance defect. Please submit your test case(s) to either Premier Support (https://premier.intel.com) or the Intel MKL User Forum (http://software.intel.com/en-us/forums/intel-math-kernel-library/) so we can reproduce and invsestigate the problem: - not seeing performance benefits from PETSC with MKL on large applications - not seeing peformance benefits when using MKL versus FORTRAN BLAS compiled with Intel Fortran - PETSC 2.3.3-p15 appears to be faster without MKL on the large-scale code Thanks, --Amanda S. Intel Corp. |
| July 27, 2011 10:23 PM PDT
cwenlong
|
when i did like the article , I got error: ld: cannot find -lPEPCF90 but lPEPCF90 is ivf 7.0 and eariler ps: Install complete. It is useable with PETSC_DIR=/home/chengwl/opt/petsc-3.1-p8 [and no more PETSC_ARCH]. Now to check if the libraries are working do (in current directory): make PETSC_DIR=/home/chengwl/opt/petsc-3.1-p8 test ==================================== [chengwl@hksbs-s13 petsc-3.1-p8]$ make PETSC_DIR=/home/chengwl/opt/petsc-3.1-p8 test Running test examples to verify correct installation C/C++ example src/snes/examples/tutorials/ex19 run successfully with 1 MPI process --------------Error detected during compile or link!----------------------- see mcs.anl.gov/petsc/petsc-2/documentation/troubleshooting.html ifort -c -g -I/home/chengwl/opt/petsc-3.1-p8/include -I/home/chengwl/opt/petsc-3.1-p8/include -I/home/chengwl/opt/petsc-3.1-p8/include/mpiuni -I/home/chengwl/opt/petsc-3.1-p8/include -I/home/chengwl/opt/petsc-3.1-p8/include/mpiuni -o ex5f.o ex5f.F ifort -g -o ex5f ex5f.o -Wl,-rpath,/home/chengwl/opt/petsc-3.1-p8/lib -L/home/chengwl/opt/petsc-3.1-p8/lib -lpetsc -lX11 -Wl,-rpath,/opt/intel/composerxe-2011.4.191/mkl/lib/intel64 -L/opt/intel/composerxe-2011.4.191/mkl/lib/intel64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread -lPEPCF90 -ldl -L/opt/intel/Compiler/11.1/064/lib/intel64 -L/opt/intel/composerxe-2011.4.191/compiler/lib/intel64 -L/opt/intel/composerxe-2011.4.191/mkl/lib/intel64 -L/opt/intel/cce/10.1.018/lib -L/usr/lib/gcc/x86_64-redhat-linux/4.1.2 -limf -lsvml -lipgo -lirc -lgcc_s -lirc_s -lifport -lifcore -lm -lpthread -lm -ldl -limf -lsvml -lipgo -lirc -lgcc_s -lirc_s -ldl ld: cannot find -lPEPCF90 make[3]: [ex5f] 错误 1 (忽略) /bin/rm -f ex5f.o Completed test examples |
Trackbacks (0)
Leave a comment 
Chao Y (Intel)
| ||
AmandaS (Intel)
|



Anthony Skjellum
However, the instructions refer to an old version of PETSC. Generically, those instructions can be easily adapted.
What is troubling, however, is that in the experiments that I have done, I have seen counter-intuitive performance behaviors. I have not seen performance benefits from PETSC on large applications
a) when moving to newer versions, b) when using MKL versus FORTRAN BLAS just compiled with Intel Fortran Compiler.
PETSC 2.3.3-p15 appears to be faster without MKL on the large-scale code I've been experimenting with than with it. PETSC 3.0.0 without MKL appears to be much slower than PETSC 2.3.3 without MKL, and PETSC 3.0.0 with MKL is a bit faster than PETSC 3.0.0 without MKL...
One thought is that for situations with small problem sizes, MKL is less efficient than Fortran blas, but we're still trying to figure out how you can lower performance, when MKL works very when benchmarked.
That is, FORTRAN BLAS have higher small problem performance!?!