Compile and run GROMACS 4.5.3 in the Intel® Cluster Ready Reference Recipe S5520UR-ICR1.1-ROCKS5.3-CENTOS5.4-C2 v1.0

Perquisites

You need to have deployed the latest Intel® Cluster Ready Reference Design S5520UR-ICR1.1-ROCKS5.3-CENTOS5.4-C2 v1.0.

This reference design targets the next components:

Set up the environment

MPICH2 will be used to build and run GROMACS. To configure the environment to use MPICH2 run the next command:

[shell]mpi-selector-menu[/shell]


A selection menu will be displayed. Select “mvapich2_gcc-1.4.1” for the current user (1u) and exit.

[shell]Current system default: <none> Current user default: mvapich2_gcc-1.4.1 "u" and "s" modifiers can be added to numeric and "U" commands to specify "user" or "system-wide". 1. mvapich2_gcc-1.4.1 2. mvapich_gcc-1.2.0 3. openmpi_gcc-1.4.1 U. Unset default Q. Quit Selection (1-3[us], U[us], Q):1u … WARNING: Changes made to mpi-selector defaults will not be visible until you start a new shell! [/shell]

  • Exit (log out) from the terminal and log back in for the changes to take place

Compilation

FFTW
GROMACS uses FFTW ( "Fastest Fourier Transform in the West" ) as its Fast Fourier library. It also supports Intel's MKL;  you can find more information in this article. Here we will focus on pure open source solutions.

  • Log in to the cluster with your normal user
  • Create a staging directory to build the tools and an "opt" directory to install them

 

[shell]mkdir opt
mkdir src
cd src[/shell]

 

  • Make sure that your proxy setting are correct
[shell]export http_proxy=proxy_url:port
export ftp_proxy=proxy_url:port [/shell]
  • Download FFTW package and uncompress it
[shell]wget http://www.fftw.org/fftw-3.2.2.tar.gz
tar xf fftw-3.2.2.tar.gz
cd fftw-3.2.2[/shell]
  • Configure the build to use threads (no need to compile it with MPI support). Here I use "/home/icr/" as a general user, you should replace every appearance of "/home/icr" by "/home/<your user>".

[shell]./configure --enable-threads --prefix=/home/icr/opt/fftw[/shell]

  • Build and install it
[shell]make
make install[/shell]
After installation you'll end up with a directory structure similar to this:

[shell]~/opt/fftw/
|-- bin
| |-- fftw-wisdom
| `-- fftw-wisdom-to-conf
|-- include
| |-- fftw3.f
| `-- fftw3.h
|-- lib
| |-- libfftw3.a
| |-- libfftw3.la
| |-- libfftw3_threads.a
| |-- libfftw3_threads.la
| `-- pkgconfig
| `-- fftw3.pc
`-- share
|-- info
| `-- fftw3.info
`-- man
`-- man1
|-- fftw-wisdom-to-conf.1
`-- fftw-wisdom.1

8 directories, 12 files
[/shell]
For further information on how to configure and build FFTW please refer to http://www.fftw.org/.

GROMACS

  • Download GROMACS package and uncompress it
[shell]cd /home/icr/src wget ftp://ftp.gromacs.org/pub/gromacs/gromacs-4.5.3.tar.gz tar xf gromacs-4.5.3.tar.gz cd gromacs-4.5.3[/shell]

  • This particular version of GROMACS has a bug which needs to be fixed before compilation. The next two patches need to be applied:

[shell]wget http://redmine.gromacs.org/projects/gromacs/repository/revisions/47eef81f8b01a30b9e096c15cb86a77c1b4c0258/diff/src/mdlib/sim_util.c?format=diff -O sim_util.patch wget http://redmine.gromacs.org/projects/gromacs/repository/revisions/47eef81f8b01a30b9e096c15cb86a77c1b4c0258/diff/src/tools/gmx_tune_pme.c?format=diff -O gmx_tune_pme.diff # Apply the patches patch -p1 < sim_util.patch patch -p1 < gmx_tune_pme.diff[/shell]

  • Configure the building environment to use the FFTW that we compiled
[shell]export CPPFLAGS="-I/home/icr/opt/fftw/include"
export LDFLAGS="-L/home/icr/opt/fftw/lib"

./configure --enable-mpi --disable-float --prefix=/home/icr/opt/gromacs
[/shell]
  • Build and install it
[shell]make
make install[/shell]

For further information on how to configure and build GROMACS please refer to http://www.gromacs.org/.


Test

These steps show how to test the installation of GROMACS 4.5.3.

  • Create a test directory and download GROMACS’ official benchmark
[shell]mkdir /home/icr/gromacs_tests
cd /home/icr/gromacs_tests
wget ftp://ftp.gromacs.org/pub/benchmarks/gmxbench-3.0.tar.gz
tar xf gmxbench-3.0.tar.gz
cd d.dppc
[/shell]

  • Add GROMACS binaries to user’s path and run grommp
[shell]export PATH=$PATH:/home/icr/opt/gromacs/bin
grompp_d
[/shell]
  • Copy the system’s ICR node list file and modify it by removing any comment such as "# type:head"
[shell]cp /etc/intel/clck/nodelist .
vi nodelist # Modify to obtain similar results
cat nodelist
head-node
compute-1
compute-2…
[/shell]

  • Run mdrun (use “-np” to define the number of processes to start, a simple formula is number_nodes x number_physical_cores_per_node)
[shell]mpirun -r ssh -f nodelist -np 396 /home/icr/opt/gromacs/bin/mdrun_d[/shell]

If the execution ended successfully a message like this should be displayed


[shell]starting mdrun 'DPPC in Water'
5000 steps, 10.0 ps.

NOTE: Turning on dynamic load balancing


Writing final coordinates.

Back Off! I just backed up confout.gro to ./#confout.gro.9#

Average load imbalance: 14.8 %
Part of the total run time spent waiting due to load imbalance: 4.2 %
Steps where the load balancing was limited by -rdd, -rcon and/or -dds: X 0 % Y 0 % Z 6 %


Parallel run - timing based on wallclock.

NODE (s) Real (s) (%)
Time: 10.170 10.170 100.0
(Mnbf/s) (GFlops) (ns/day) (hour/ns)
Performance: 11463.476 401.899 84.972 0.282
[/shell]


Appendix

This are both patches in case GROMACS' tracker is down:

[shell]commit 47eef81f8b01a30b9e096c15cb86a77c1b4c0258 Author: Carsten Kutzner <ckutzne@gwdg.de> Date: Tue Mar 1 13:06:31 2011 +0100 Fixed bug #714: deprecated structure in gettimeofday diff --git a/src/mdlib/sim_util.c b/src/mdlib/sim_util.c index d8d94ff..60ef80e 100644 --- a/src/mdlib/sim_util.c +++ b/src/mdlib/sim_util.c @@ -112,10 +112,9 @@ gmx_gettime() { #ifdef HAVE_GETTIMEOFDAY struct timeval t; - struct timezone tz = { 0,0 }; double seconds; - gettimeofday(&t,&tz); + gettimeofday(&t,NULL); seconds = (double) t.tv_sec + 1e-6*(double)t.tv_usec;[/shell]

[shell]commit 47eef81f8b01a30b9e096c15cb86a77c1b4c0258 Author: Carsten Kutzner <ckutzne@gwdg.de> Date: Tue Mar 1 13:06:31 2011 +0100 Fixed bug #714: deprecated structure in gettimeofday diff --git a/src/tools/gmx_tune_pme.c b/src/tools/gmx_tune_pme.c index b775d17..e70f5ea 100644 --- a/src/tools/gmx_tune_pme.c +++ b/src/tools/gmx_tune_pme.c @@ -1964,10 +1964,9 @@ static double gettime() { #ifdef HAVE_GETTIMEOFDAY struct timeval t; - struct timezone tz = { 0,0 }; double seconds; - - gettimeofday(&t,&tz); + + gettimeofday(&t,NULL); seconds = (double) t.tv_sec + 1e-6*(double)t.tv_usec;[/shell]
Para obtener información más completa sobre las optimizaciones del compilador, consulte nuestro Aviso de optimización.