Building the HPCC* benchmark with Intel® C++ and Fortran Compilers

Introduction

This guide is intended to help Intel® compiler customers build the HPCC* benchmark. The HPC Challenge (HPCC) benchmark is used to evaluate and test a wide variety of performance parameters for high-performance computing system.

Version information

The version of HPC Challenge (HPCC): 1.0.0
The version of Intel® C++ and Fortran Compilers for Linux* or Mac OS* X: 10.0
The version of Intel® Math Kernel Library 9.1 (MKL): 9.1

Application notes

The HPC Challenge (HPCC) benchmark is comprised of 7 individual benchmarks or tests designed to stress various components of a high performance computing system and a test harness or wrapper program to execute these tests. The target environment for HPCC can vary dramatically based on interconnect, adapters, node types, batch subsystem, etc. Please read all relevant documentation for your target system.

This application note demonstrates the general framework for building HPCC with the Intel compilers but does NOT claim to represent all possible configurations and variations of the build for all possible target environments and optimizations.

Obtaining the source code

HPCC is obtained from the HPC Challenge website, http://icl.cs.utk.edu/hpcc/index.html. Please review their licensing and download instructions for access to this code.

Obtaining the latest version of Intel® C++ Compiler and Intel® Fortran Compiler

Licensed users of the Intel compilers may download the most recent versions of the compiler from the Intel Download Center: Intel® Registration Center.
Others can download the evaulation copy from /en-us/.

Prerequisites

Hardware: This note applies to users with stand-alone computers with two or more cores or distributed-memory clusters.

Software: The HPCC benchm ark requires a existing MPI library to be installed. MPI is the Message Passing Interface. There are many MPI library implementations, including and industry-leading implementation from Intel. For information on obtaining an evaluation copy of Intel MPI, please visit http://www.intel.com/cd/software/products/asmo-na/eng/308295.htm and select the "Get Evaluation" link.

The HPCC benchmark also requires an existing linear algebra library to be installed on your system. For maximum benchmark performance IT IS ABSOLUTELY CRITICAL to have a highly-tuned and optimized linear algebra library on your system. The Intel® Math Kernel Library (MKL) is one such library. It has a highly tuned and optimized linear algebra library tuned for all existing Intel processor families. For more information on Intel® Math Kernel Library and to obtain a free evaluation copy, please visit /en-us/.

Configuration and set-up information

The first step to build HPCC is to uncompress and extract the source directory from the source tarball. The source tarball will extract to a directory named " hpcc-1.0.0". Move the hpcc tarball to a directory under which you wish to extract the hpcc source files and execute the following:

tar -zxvf hpcc-1.0.0.tar.gz
cd hpcc-1.0.0

HPCC uses hand-edited 'make' configuration files to determine the build environment, location of compilers and requisite libraries. These make configuration files must reside in the hpcc-1.0.0/hpl subdirectory. The naming of the make configuration files are important. The name of the make configuration file should follow the form Make.<type> where " <type>" is some user-defined architecture or compiler name. For example, for a build using the Intel compilers, it may be reasonable to name this file " Make.Intel". There are numerous examples of these Make.<type> files in the subdirectory hpcc-1.0.0/hpl/setup.

Assuming your system has MPI wrappers for the Intel compilers, and the Intel Math Kernel Library located in /opt/intel/mkl/9.1, a reasonable Make.Intel file might look like:

# ---- Example hpcc-1.0.0/hpl/Make.Intel make configuration file for HPCC ----
# ----------------------------------------------------------------------
# - shell --------------------------------------------------------------
# ----------------------------------------------------------------------
#
SHELL = /bin/sh
#
CD = cd
CP = cp
LN_S = ln -s
MKDIR = mkdir
RM = /bin/rm -f
TOUCH = touch
#
# ----------------------------------------------------------------------
# - Platform identifier ------------------------------------------------
# ----------------------------------------------------------------------
#
ARCH = $(arch)
#
# ----------------------------------------------------------------------
# - HPL Directory Structure / HPL library ------------------------------
# -- --------------------------------------------------------------------
#
TOPdir = ../../..
INCdir = $(TOPdir)/include
BINdir = $(TOPdir)/bin/$(ARCH)
LIBdir = $(TOPdir)/lib/$(ARCH)
#
HPLlib = $(LIBdir)/libhpl.a
#
# ----------------------------------------------------------------------
# - Message Passing library (MPI) --------------------------------------
# ----------------------------------------------------------------------
#
MPinc tells the C compiler where to find the Message Passing library header files, MPlib is defined to be the name of the library to be used. The variable MPdir is only used for defining MPinc and MPlib.
#
MPdir = /usr/local/openmpi-1.2.2
MPinc = -I$(MPIdir)/include
MPlib = -L$(MPIdir)/lib -lmpi
#
# ----------------------------------------------------------------------
# - Linear Algebra library (BLAS or VSIPL) -----------------------------
# ----------------------------------------------------------------------
# LAinc tells the C compiler where to find the Linear Algebra library
# header files, LAlib is defined to be the name of the library to be
# used. The variable LAdir is only used for defining LAinc and LAlib.
#
LAdir = /opt/intel/mkl/9.1
LAinc = -I$(LAdir)/include
LAlib = -L$(LAdir)/lib/em64t -lmkl
#
# ----------------------------------------------------------------------
# - F77 / C interface --------------------------------------------------
# ----------------------------------------------------------------------
#
You can skip this section if and only if you are not planning to use a BLAS library featuring a Fortran 77 interface. Otherwise, it is necessary to fill out the F2CDEFS variable with the appropriate options. One and only one option should be chosen in each of the 3 following categories:
  1. name space (How C calls a Fortran 77 routine)
    #
    # -DAdd_ : all lower case and a suffixed underscore (Suns,
    # Intel, ...), [default]
    # -DNoChange : all lower case (IBM RS6000),
    # -DUpCase : all upper case (Cray),
    # -DAdd__ : the FORTRAN compiler in use is f2c.
    #
  2. C and Fortran 77 integer mapping
    #
    # -DF77_INTEGER=int : Fortran 77 INTEGER is a C int, [default]
    # -DF77_INTEGER=long : Fortran 77 INTEGER is a C long,
    # -DF77_INTEGER=short : Fortran 77 INTEGER is a C short.
  3. Fortran 77 string handling
    #
    # -DStringSunStyle : The string address is passed at the string location on the stack, and the string length is then passed as an F77_INTEGER after all explicit stack arguments, [default]
    # -DStringStructPtr : The address of a structure is passed by a
    # Fortran 77 string, and the structure is of the
    # form: struct {char *cp; F77_INTEGER len;},
    # -DStringStructVal : A structure is passed by value for each Fortran
    # 77 string, and the structure is of the form:
    # struct {char *cp; F77_INTEGER len;},
    # -DStringCrayStyle : Special option for Cray machines, which uses
    # Cray fcd (fortran character descriptor) for
    # interoperation.
    #
    F2CDEFS = -DAdd_ -DF77_INTEGER=int -DStringSunStyle
    #
    # ----------------------------------------- -----------------------------
    # - HPL includes / libraries / specifics -------------------------------
    # ----------------------------------------------------------------------
    #
    HPL_INCLUDES = -I$(INCdir) -I$(INCdir)/$(ARCH) $(LAinc) $(MPinc)
    HPL_LIBS = $(HPLlib) $(LAlib) $(MPlib)
    #
    # - Compile time options -----------------------------------------------
    #
    # -DHPL_COPY_L force the copy of the panel L before bcast;
    # -DHPL_CALL_CBLAS call the cblas interface;
    # -DHPL_CALL_VSIPL call the vsip library;
    # -DHPL_DETAILED_TIMING enable detailed timers;
    #
    # By default HPL will:
    # *) not copy L before broadcast,
    # *) call the BLAS Fortran 77 interface,
    # *) not display detailed timing information.
    #
    HPL_OPTS =
    #
    # ----------------------------------------------------------------------
    # ----------------------------------------------------------------------
    #
    HPL_DEFS = $(F2CDEFS) $(HPL_OPTS) $(HPL_INCLUDES)
    #
    # ----------------------------------------------------------------------
    # - Compilers / linkers - Optimization flags ---------------------------
    # ----------------------------------------------------------------------
    #
    CC = mpicc
    CCNOOPT = $(HPL_DEFS) -O0 -g
    CCFLAGS = $(HPL_DEFS) -O3 -xP -ip -fno-alias
    #
    # On some platforms, it is necessary to use the Fortran linker to find
    # the Fortran internals used in the BLAS library.
    #
    LINKER = mpicc
    LINKFLAGS = $(CCFLAGS)
    #
    ARCHIVER = ar
    ARFLAGS = rc
    RANLIB = echo
    #
    # ----------------------------------------------------------------------
Some important configuration variables to consider:

MPdir should point to the root directory of your MPI installation.

MPinc should point to the MPI include directory.

MPlib should point to the MPI library directory and should have -lmpi or -l<name of MPI library> where <name of MPI library> is the filename of your MPI libraray, minus the leading "lib" prefix.

LAdir should point to the root directory of the linear algebra package. In this example, we are showing the default location for Intel® Math Kernel Library version 9.1.

LAinc should point to the location of the include files for your linear algebra package.

LAlib should point to the location of library directory for your linear algebra package. In our example, we are targeting the Intel® 64 version of the MKL package ( subdirectory named "em64t")

.

F2CDEFS - please note the setting for this variable to work with the Intel Fortran compiler.

CC - this variable defines the C compiler. In this example, we use the mpicc wrapper and assume that this wrapper invokes the Intel C compiler icc.

If you wish to invoke icc directly, please change the CC definition to icc.

CCNOOPT - this variable defines compiler options for the NO-OPTimized build. ICC defaults to optimization -O2, this we need to explicitly call out -O0 -g to get no optimization and full symbolic information.

CCFLAGS - this variable defines the compiler options for the optimized build. The options shown, -O3 -xP -ip -fno-alias, should give good performance for Intel® Xeon family processors. Intel® 64 and IA-32 customers Please read the documentation on the -x option and select the processor type for your target processor. For IA-64 customers, the following options are recommended: -O3 -fno-alias

LINKER - this variable defines the linker command. In this example the mpicc wrapper is used for the link phase. You may also wish to use icc as your command for the link phase, or some other compiler driver or the ' ld' command.

The make configure file has a lot of possible options. To learn all of the possible configuration options, see the help provided in the documentation found at http://icl.cs.utk.edu/hpcc/index.html.

Building HPCC

After your hpcc-1.0.0/hpl/Make.Intel file is created, you can build the HPCC benchmark. Change directory to the main hpcc-1.0.0/ directory. From here, the HPCC benchmark can be built with the command:

make arch=Intel

The "Intel" keyword for "arch" tells the make system to look for hpl/Make.Intel for build configuration settings.

To 'clean' the results of a previous build, use the command:
make arch=Intel clean

A successful build will leave an executable file ' hpcc' in the main hpcc-1.0.0/ directory.

Running the HPCC benchmark

Once hpcc binary is built, it can be run as you would any MPI application on your system. The hpcc binary will look for the input file hpcc-1.0.0/hpccinf.txt. This file controls the size of the problem(s) to run. Please see the HPCC website for configuraton information for this file. This file must be hand-edited and created prior to running hpcc.

Typical MPI implementations provide commands mpirun or mpiexec to launch parallel applications. Below is a simple example to run a 2 process HPCC run:

mpirun -np 2 ./hpcc

There are many possible options to mpirun and mpiexec, as well as interactions with batch scheduling systems such as PBS. Please see your specific MPI documentation for all the specifics for your system or cluster.

Benefits

This article provides the steps on how to build the HPC Challenge (HPCC)* benchmark using the Intel compilers.

Known issues and limitations

At this time, there are no known issues with using the Intel compilers to build the HPC Challenge benchmark. Please see the HPCC website http://icl.cs.utk.edu/hpcc/index.html for all known issues and limitations.

Optimization Notice

Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.

Notice revision #20110804

For more complete information about compiler optimizations, see our Optimization Notice.