Compile and run GROMACS 4.5.3 in the Intel® Cluster Ready Reference Recipe S5520UR-ICR1.1-ROCKS5.3-CENTOS5.4-C2 v1.0

Perquisites

You need to have deployed the latest Intel® Cluster Ready Reference Design S5520UR-ICR1.1-ROCKS5.3-CENTOS5.4-C2 v1.0.

This reference design targets the next components:

Set up the environment

MPICH2 will be used to build and run GROMACS. To configure the environment to use MPICH2 run the next command:

[shell]mpi-selector-menu[/shell]


A selection menu will be displayed. Select “mvapich2_gcc-1.4.1” for the current user (1u) and exit.

[shell]Current system default: <none> Current user default: mvapich2_gcc-1.4.1 "u" and "s" modifiers can be added to numeric and "U" commands to specify "user" or "system-wide". 1. mvapich2_gcc-1.4.1 2. mvapich_gcc-1.2.0 3. openmpi_gcc-1.4.1 U. Unset default Q. Quit Selection (1-3[us], U[us], Q):1u … WARNING: Changes made to mpi-selector defaults will not be visible until you start a new shell! [/shell]

  • Exit (log out) from the terminal and log back in for the changes to take place

Compilation

FFTW
GROMACS uses FFTW ( "Fastest Fourier Transform in the West" ) as its Fast Fourier library. It also supports Intel's MKL;  you can find more information in this article. Here we will focus on pure open source solutions.

  • Log in to the cluster with your normal user
  • Create a staging directory to build the tools and an "opt" directory to install them

 

[shell]mkdir opt
mkdir src
cd src[/shell]

 

  • Make sure that your proxy setting are correct
[shell]export http_proxy=proxy_url:port
export ftp_proxy=proxy_url:port [/shell]
  • Download FFTW package and uncompress it
[shell]wget http://www.fftw.org/fftw-3.2.2.tar.gz
tar xf fftw-3.2.2.tar.gz
cd fftw-3.2.2[/shell]
  • Configure the build to use threads (no need to compile it with MPI support). Here I use "/home/icr/" as a general user, you should replace every appearance of "/home/icr" by "/home/<your user>".

[shell]./configure --enable-threads --prefix=/home/icr/opt/fftw[/shell]

  • Build and install it
[shell]make
make install[/shell]
After installation you'll end up with a directory structure similar to this:

[shell]~/opt/fftw/
|-- bin
| |-- fftw-wisdom
| `-- fftw-wisdom-to-conf
|-- include
| |-- fftw3.f
| `-- fftw3.h
|-- lib
| |-- libfftw3.a
| |-- libfftw3.la
| |-- libfftw3_threads.a
| |-- libfftw3_threads.la
| `-- pkgconfig
| `-- fftw3.pc
`-- share
|-- info
| `-- fftw3.info
`-- man
`-- man1
|-- fftw-wisdom-to-conf.1
`-- fftw-wisdom.1

8 directories, 12 files
[/shell]
For further information on how to configure and build FFTW please refer to http://www.fftw.org/.

GROMACS

  • Download GROMACS package and uncompress it
[shell]cd /home/icr/src wget ftp://ftp.gromacs.org/pub/gromacs/gromacs-4.5.3.tar.gz tar xf gromacs-4.5.3.tar.gz cd gromacs-4.5.3[/shell]

  • This particular version of GROMACS has a bug which needs to be fixed before compilation. The next two patches need to be applied:

[shell]wget http://redmine.gromacs.org/projects/gromacs/repository/revisions/47eef81f8b01a30b9e096c15cb86a77c1b4c0258/diff/src/mdlib/sim_util.c?format=diff -O sim_util.patch wget http://redmine.gromacs.org/projects/gromacs/repository/revisions/47eef81f8b01a30b9e096c15cb86a77c1b4c0258/diff/src/tools/gmx_tune_pme.c?format=diff -O gmx_tune_pme.diff # Apply the patches patch -p1 < sim_util.patch patch -p1 < gmx_tune_pme.diff[/shell]

  • Configure the building environment to use the FFTW that we compiled
[shell]export CPPFLAGS="-I/home/icr/opt/fftw/include"
export LDFLAGS="-L/home/icr/opt/fftw/lib"

./configure --enable-mpi --disable-float --prefix=/home/icr/opt/gromacs
[/shell]
  • Build and install it
[shell]make
make install[/shell]

For further information on how to configure and build GROMACS please refer to http://www.gromacs.org/.


Test

These steps show how to test the installation of GROMACS 4.5.3.

  • Create a test directory and download GROMACS’ official benchmark
[shell]mkdir /home/icr/gromacs_tests
cd /home/icr/gromacs_tests
wget ftp://ftp.gromacs.org/pub/benchmarks/gmxbench-3.0.tar.gz
tar xf gmxbench-3.0.tar.gz
cd d.dppc
[/shell]

  • Add GROMACS binaries to user’s path and run grommp
[shell]export PATH=$PATH:/home/icr/opt/gromacs/bin
grompp_d
[/shell]
  • Copy the system’s ICR node list file and modify it by removing any comment such as "# type:head"
[shell]cp /etc/intel/clck/nodelist .
vi nodelist # Modify to obtain similar results
cat nodelist
head-node
compute-1
compute-2…
[/shell]

  • Run mdrun (use “-np” to define the number of processes to start, a simple formula is number_nodes x number_physical_cores_per_node)
[shell]mpirun -r ssh -f nodelist -np 396 /home/icr/opt/gromacs/bin/mdrun_d[/shell]

If the execution ended successfully a message like this should be displayed


[shell]starting mdrun 'DPPC in Water'
5000 steps, 10.0 ps.

NOTE: Turning on dynamic load balancing


Writing final coordinates.

Back Off! I just backed up confout.gro to ./#confout.gro.9#

Average load imbalance: 14.8 %
Part of the total run time spent waiting due to load imbalance: 4.2 %
Steps where the load balancing was limited by -rdd, -rcon and/or -dds: X 0 % Y 0 % Z 6 %


Parallel run - timing based on wallclock.

NODE (s) Real (s) (%)
Time: 10.170 10.170 100.0
(Mnbf/s) (GFlops) (ns/day) (hour/ns)
Performance: 11463.476 401.899 84.972 0.282
[/shell]


Appendix

This are both patches in case GROMACS' tracker is down:

[shell]commit 47eef81f8b01a30b9e096c15cb86a77c1b4c0258 Author: Carsten Kutzner <ckutzne@gwdg.de> Date: Tue Mar 1 13:06:31 2011 +0100 Fixed bug #714: deprecated structure in gettimeofday diff --git a/src/mdlib/sim_util.c b/src/mdlib/sim_util.c index d8d94ff..60ef80e 100644 --- a/src/mdlib/sim_util.c +++ b/src/mdlib/sim_util.c @@ -112,10 +112,9 @@ gmx_gettime() { #ifdef HAVE_GETTIMEOFDAY struct timeval t; - struct timezone tz = { 0,0 }; double seconds; - gettimeofday(&t,&tz); + gettimeofday(&t,NULL); seconds = (double) t.tv_sec + 1e-6*(double)t.tv_usec;[/shell]

[shell]commit 47eef81f8b01a30b9e096c15cb86a77c1b4c0258 Author: Carsten Kutzner <ckutzne@gwdg.de> Date: Tue Mar 1 13:06:31 2011 +0100 Fixed bug #714: deprecated structure in gettimeofday diff --git a/src/tools/gmx_tune_pme.c b/src/tools/gmx_tune_pme.c index b775d17..e70f5ea 100644 --- a/src/tools/gmx_tune_pme.c +++ b/src/tools/gmx_tune_pme.c @@ -1964,10 +1964,9 @@ static double gettime() { #ifdef HAVE_GETTIMEOFDAY struct timeval t; - struct timezone tz = { 0,0 }; double seconds; - - gettimeofday(&t,&tz); + + gettimeofday(&t,NULL); seconds = (double) t.tv_sec + 1e-6*(double)t.tv_usec;[/shell]
For more complete information about compiler optimizations, see our Optimization Notice.

2 comments

Top
markabraham's picture

Also, the above will not work for GROMACS 4.5.4 without --enable-shared on the FFTW configuration. See GROMACS installation instructions for more detail.

markabraham's picture

The above instructions will build a double-precision version of GROMACS, which is unnecessarily slow for most molecular dynamics applications. The accuracy of single precision is normally quite adequate.

The reader should also consult the GROMACS webpage (http://www.gromacs.org/) for non-ROCKS-specific installation instructions, and advice from the GROMACS team.

Add a Comment

Have a technical question? Visit our forums. Have site or software product issues? Contact support.