Building HDF5* with Intel® compilers


Introduction

This guide is intended to help Intel® compiler customers build and use the HDF5 library. HDF5 is the latest generation of the HDF libraries, a general purpose library and associated file formats for storing and sharing scientific data. HDF5 is maintained, promoted, and co-developed along with active community support by The HDF Group (THG). THG is a not-for-profit corporation with the mission to sustain HDF technologies and to provide support to HDF user communities. The homepage for THG and HDF5 can be found at http://www.hdfgroup.org/HDF5/.


Version information

HDF5 1.8.8 and later.
Intel® C++ Compiler for Linux* or Mac OS* X
Intel® Fortran Compilers for Linux* or Mac OS* X


Application Notes

HDF5 is a data format and an associated software library designed to store, access, manage, exchange, and archive diverse, complex data in continuously evolving heterogeneous computing and storage environments. HDF5 is extensively used with scientific research, engineering development, and other data.

This application note demonstrates the framework for building HDF5 with the Intel compilers but does NOT claim to represent all possible configurations and variations of the build for all possible target environments.


Obtaining the Source Code

The HDF5 source files should be obtained from the HDF5 Software downloads page at http://www.hdfgroup.org/HDF5/release/obtain5.htmlPlease note the External Library requirements for SZIP and ZLIB and download those if you do not already have those libraries.


Obtaining the latest version of Intel C++ Compiler and Intel Fortran Compiler

Licensed users of the Intel compilers may download the most recent versions of the compiler from the Intel® Download Center: Intel® Registration Center. Other users can download the evaluation copy from https://software.intel.com/en-us/articles/try-buy-tools.


Prerequisites

Software: As mentioned on the HDF5 software downloads page, either SZIP-2.1 or ZLIB libraries can be used for file compression/decompression. Precompiled binaries or sources are available from the HDF5 Software Downloads page for these two libraries.

SZIP: Determine an appropriate location to install SZIP.  Directory /usr/local/szip-2.1 may be a reasonable choice. If you wish to build and install szip from the source files, use the procedure shown below:

export CC=icc
export CXX=icpc
export FC=ifort
export CFLAGS='-O3 -xHost -ip'
export CXXFLAGS='-O3 -xHost -ip'
export FCFLAGS='-O3 -xHost -ip'
tar -zxvf szip-2.1.tar.gz
cd szip-2.1
./configure --prefix=/usr/local/szip-2.1
make
make check
make install

The above example uses the BASH shell syntax for setting environment variables. For other shells, use the appropriate commands to set environment variables CC, CXX, etc. before the make command. Confirm that after the 'make check' command, the result of the tests return the result "All test passed."

Check the directory specified by your --prefix= setting. This directory should contain lib/ and include/directories. For more information on building szip, consult the file named INSTALL in the source directory.

ZLIB:The information shown is for zlib version 1.2.7. zlib is a general purpose data compression library and is a prerequisite for building HDF5. Determine an appropriate location to install zlib. /usr/local/zlib-1.2.7 may be a reasonable choice. If you wish to build and install zlib from the source files, use the procedure shown below:

export CC=icc
export CFLAGS='-O3 -xHost -ip'
tar -zxvf zlib-1.2.7.tar.gz
cd zlib-1.2.7 
./configure --prefix=/usr/local/zlib-1.2.7 
make
make check
make install

The above example uses the BASH shell syntax for setting environment variables. For other shells, use the appropriate commands to set environment variables CC, CXX, etc. before the make command. Confirm that after the 'make check' command, the result of the tests return the result "test OK".

Check the directory specified by your --prefix= setting. This directory should contain lib/, include/, and share/directories. For more information on building zlib, consult the file named "README" in the source directory.


Configuration andSetup Information for HDF5

HDF5 uses an Autoconf "configure" script to determine the build environment and tools and create the necessary build configuration. The first step is to set environment variables to control which compilers are used for the build. These environment variables select the Intel C++ Compiler and the Intel Fortran Compiler.

export CC=icc
export F9X=ifort
export CXX=icpc

As shown above, the environment CC, CXX and F9X are used to specify which compilers are used to build HDF5. The example shown above uses both the Intel C++ Compiler ( CC=icc CXX=icpc ) and the Intel Fortran Compiler ( F9X=ifort ). Note that the Intel C++ compiler driver is named 'icpc'. Do NOT use 'icc' as the C++ compiler. The Intel compilers are GNU compatible, thus you may mix and match the Intel compilers with GNU compilers for C++ and Fortran. However, the mixing of GNU compilers with Intel compilers has not been tested with this application.

There are environment variables such as CFLAGS to pass compiler options to the C compiler. However, the configuration script will automatically detect the Intel compilers and use the appropriate optimization options. Thus, the user need not specify optimization settings unless one wants to override the default settings set by configure.


To Extract the Source files and to Configure and Build HDF5

tar -zxvf hdf5-1.8.15.tar.gz
cd hdf5-1.8.15
./configure --prefix=/usr/local/hdf5-1.8.15 --enable-fortran --enable-cxx
... output of configure ...
make
... watch for fatal errors ...
make check
... verify that all tests return "PASS"
make install

The configure script has many options. Refer to the help provided in the output of "./configure --help" or read the contents of the file README.


Using HDF5

There is user documentation in the doc/html subdirectory of the source file directory. Look for the file "index.html" and view this file in a browser. This doc/html directory can be copied over to the installation directory.

In general, user program include <hdf5.h> and link with -lhdf5. Additional libraries may be necessary. Please see the user documentation for all the details on the use of HDF5. Users are encouraged to use the compiler helper scripts h5cc, h5fc and h5c++ to build their applications. These helper scripts are installed in the bin/ subdirectory of the installation directory.


Known Issues and Limitations

For more complete information about compiler optimizations, see our Optimization Notice.

7 comments

Top
Flavio D.'s picture

I followed all the steps:

./configure --prefix=/usr/local/hdf5-1.8.15 --enable-fortran --enable-cxx

but when I did: make
Showed up this:

 

make[2]: Entering directory '/home/fuser/hdf5-1.8.18/fortran/src'
make  all-am
make[3]: Entering directory '/home/fuser/hdf5-1.8.18/fortran/src'
  FCLD     libhdf5_fortran.la
ar: `u' modifier ignored since `D' is the default (see `U')
make[3]: Leaving directory '/home/fuser/hdf5-1.8.18/fortran/src'
make[2]: Leaving directory '/home/fuser/hdf5-1.8.18/fortran/src'
Making all in test
make[2]: Entering directory '/home/fuser/hdf5-1.8.18/fortran/test'
  FCLD     fortranlib_test
fortranlib_test-tH5T.o: In function `th5t_mp_enumtest_':
tH5T.f90:(.text+0x3683): undefined reference to `h5t_provisional_mp_h5tenum_insert_f90_'
tH5T.f90:(.text+0x36ba): undefined reference to `h5t_provisional_mp_h5tenum_insert_f90_'
Makefile:880: recipe for target 'fortranlib_test' failed
make[2]: *** [fortranlib_test] Error 1
make[2]: Leaving directory '/home/fuser/hdf5-1.8.18/fortran/test'
Makefile:747: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/home/fuser/hdf5-1.8.18/fortran'
Makefile:586: recipe for target 'all-recursive' failed
make: *** [all-recursive] Error 1

 

someone knows how can I fix that?

I'm using parallel_studio_xe_2017.3.053

 

Piotr G.'s picture

Hi, 

If want to enable Fortran2003 interface for HDF5 library, you must use additional flag:

tar -zxvf hdf5-1.8.15.tar.gz
cd hdf5-1.8.15
./configure --prefix=/usr/local/hdf5-1.8.15 --enable-fortran --enable-fortran2003 --enable-cxx
... output of configure ...
make
... watch for fatal errors ...
make check
... verify that all tests return "PASS"
make install

Just learnt about that..

Piotr

Dong L.'s picture

Hello, I'm intalling the parallel HDF on my account on Tianhe Supercomtuer.

I have do the following configuration:

export CC=mpicc

export CXX=mpicxx

export FC=mpif90

export RUNSERIAL="yhrun -n 1 -t 30 -p debug"

export RUNPARALLEL="yhrun -n 1 -t 30 -p debug"

[taojj@ln3%tianhe hdf5-1.8.15-patch1]$ ./configure --with-default-api-version=v16 --enable-production  --enable-parallel  --prefix=/vol-th/home/taojj/luodong/sfwchombo/hdf5parallel 

there is no error in make step, but when I use make check, there is error:

Testing  testhdf5 ============================ testhdf5  Test Log============================yhrun: job 2573362 queued and waiting for resourcesyhrun: job 2573362 has been allocated resourcesyhrun: error: cn1390: task 0: Exited with exit code 1/vol-th/home/taojj/luodong/HDF5/hdf5-1.8.15-patch1/test/./testhdf5: line 202: mpicc: command not foundn0.04user 0.01system 0:24.75elapsed 0%CPU (0avgtext+0avgdata 22080maxresident)k0inputs+0outputs (0major+2034minor)pagefaults 0swapsmake[3]: *** [testhdf5.chkexe_] Error 1make[3]: Leaving directory `/vol-th/home/taojj/luodong/HDF5/hdf5-1.8.15-patch1/test'make[2]: *** [build-check-s] Error 2make[2]: Leaving directory `/vol-th/home/taojj/luodong/HDF5/hdf5-1.8.15-patch1/test'make[1]: *** [test] Error 2make[1]: Leaving directory `/vol-th/home/taojj/luodong/HDF5/hdf5-1.8.15-patch1/test'make: *** [check-am] Error 2 It seems that mpicc commond not found. Can anybody give some idea to solve the problem? 

dogarcar's picture

Thanks for this detailed information!

Thomas D.'s picture

I was very happy to find these directions.

make check failed with 8 errors however when trying to build HDF5 during h5import tests. It complained about '../h5dump/h5dump no such file or directory' and 'Configuration parameters are invalid in dbinuin32.h5.dmp'

Any ideas?

anonymous's picture

That depends on whether you'd like to build 32 or 64 bit executables (the 64 bit platform can build either if you installed the IA-32/Intel 64 compilers).

For 32 bit, the above configuration should be sufficient.

For 64 bit, replace any -xP with -xT and add -O3 for additional optimization.

nobleeu's picture

Do any of these directions change if you just want 64-bit on a APPLE MACOSX Intel?

Add a Comment

Have a technical question? Visit our forums. Have site or software product issues? Contact support.