Installing the Intel Cluster Toolkit Compiler Edition on a GE + IB cluster

Installing the Intel Cluster Toolkit Compiler Edition on a GE + IB cluster

Ritratto di hpcsoftwaremail.csi.cuny.edu

All,
I have successfully installed the ICTCE on a simple GE cluster with one switch and set of interfaces. What, if any, are the tricks and complications when installing on a system with two networks (GE and IB). I obviously wish the be able to run MPI over the IB network, but would like to use and test either as I wish.
The installation guide offers no information on this question as far as I can tell. The instructions ask you with construct the machines.LINUX file using the hostname generated names for the compute nodes. These associate with the GE interface. This file may only be used for the push out of the installation, but I would like to know for sure.
What might be different in the installation procedure on a dual network cluster (GE and IB).
Thanks,
Richard Walsh Parallel Applications and Systems Manager, CUNY HPC Center

10 post / 0 new
Ultimo contenuto
Per informazioni complete sulle ottimizzazioni del compilatore, consultare l'Avviso sull'ottimizzazione
Ritratto di Gergana Slavova (Intel)

Hi Richard,

The Intel Cluster Toolkit or any of its components doesn't really care what networks are available on your system during installation. Because of our architecture, what network fabric you use to run your MPI programs is all tunable at runtime and not something you have to worry about at install-time.

In fact, we have a few customers who start using the tools on a GigE cluster. After some time (and the availability of money), they buy a few IB network switches, and get those plugged in as well. They don't have to re-install the Cluster Tools yet again, but simply have to change an environment option before running.

After you've completed your installation, you can select between different networks as follows (this procedure is also described in the Getting Started PDF document):

When running over GigE, use the ssm (shared memory + sockets) device:

# assuming your cluster is setup to use ssh (-r ssh)
# and you have a mpd.hosts file available,
# which lists all hosts on your cluster, one hostname per line

$ mpirun -r ssh -f mpd.hosts -genv I_MPI_DEVICE ssm ./a.out

When running over IB, to use the rdssm (RDMA + shared memory) device:

# assuming your cluster is setup to use ssh (-r ssh)
# and you have a mpd.hosts file available,
# which lists all hosts on your cluster, one hostname per line

$ mpirun -r ssh -f mpd.hosts -genv I_MPI_DEVICE rdssm ./a.out

In fact, the Intel MPI Library would pick the fastest available fabric on your cluster at runtime. In your case, it'll use the IB network as default. If you'd like to instead run over GigE, you can go head and select the ssm device as I describe above.

I hope this helps. Let us know if you hit any problems during installation, or at runtime.

Regards,
~Gergana

Gergana Slavova
Technical Consulting Engineer
Intel® Cluster Tools
E-mail: gergana.s.slavova_at_intel.com
Ritratto di hpcsoftwaremail.csi.cuny.edu
Quoting - Gergana Slavova (Intel)

Hi Richard,

The Intel Cluster Toolkit or any of its components doesn't really care what networks are available on your system during installation. Because of our architecture, what network fabric you use to run your MPI programs is all tunable at runtime and not something you have to worry about at install-time.

In fact, we have a few customers who start using the tools on a GigE cluster. After some time (and the availability of money), they buy a few IB network switches, and get those plugged in as well. They don't have to re-install the Cluster Tools yet again, but simply have to change an environment option before running.

After you've completed your installation, you can select between different networks as follows (this procedure is also described in the Getting Started PDF document):

When running over GigE, use the ssm (shared memory + sockets) device:

# assuming your cluster is setup to use ssh (-r ssh)
# and you have a mpd.hosts file available,
# which lists all hosts on your cluster, one hostname per line

$ mpirun -r ssh -f mpd.hosts -genv I_MPI_DEVICE ssm ./a.out

When running over IB, to use the rdssm (RDMA + shared memory) device:

# assuming your cluster is setup to use ssh (-r ssh)
# and you have a mpd.hosts file available,
# which lists all hosts on your cluster, one hostname per line

$ mpirun -r ssh -f mpd.hosts -genv I_MPI_DEVICE rdssm ./a.out

In fact, the Intel MPI Library would pick the fastest available fabric on your cluster at runtime. In your case, it'll use the IB network as default. If you'd like to instead run over GigE, you can go head and select the ssm device as I describe above.

I hope this helps. Let us know if you hit any problems during installation, or at runtime.

Regards,
~Gergana

Gergana, Spaceeba bolshoi ... !! Richard
Ritratto di Gergana Slavova (Intel)
Quoting - hpcsoftwaremail.csi.cuny.edu Gergana, Spaceeba bolshoi ... !! Richard

Any time, Richard :)

Gergana Slavova
Technical Consulting Engineer
Intel® Cluster Tools
E-mail: gergana.s.slavova_at_intel.com
Ritratto di hpcsoftwaremail.csi.cuny.edu
Quoting - Gergana Slavova (Intel)

Any time, Richard :)

Gergana, Who is the lady in your window, if I might ask? Richard
Ritratto di Gergana Slavova (Intel)

Hi Richard,

It's Lady Ada Lovelace, she of the awesome computing skills, also recognized as the "first programmer" ... such are my idols :)

Regards,
~Gergana

Gergana Slavova
Technical Consulting Engineer
Intel® Cluster Tools
E-mail: gergana.s.slavova_at_intel.com
Ritratto di Dmitry Kuzmin (Intel)
Quoting - hpcsoftwaremail.csi.cuny.edu

Gergana, Spaceeba bolshoi ... !! Richard

Richard, I'm afraid Gergana doesn't understand russian... do you, Gergana?

For those who don't understand: "thanks a lot"

Dmitry

Ritratto di Tim Prince

I had the impression Gergana was a native speaker of Bulgarian.

Ritratto di Gergana Slavova (Intel)
Quoting - tim18 I had the impression Gergana was a native speaker of Bulgarian.

Yup, you're entirely correct, Tim. I was born in Bulgaria so that's my native tongue.

Thanks for the translation, Dmitry. I've actually seen a few Russian movies (my parents like them a lot) so "spasiba" was pretty easy :)

~Gergana

Gergana Slavova
Technical Consulting Engineer
Intel® Cluster Tools
E-mail: gergana.s.slavova_at_intel.com
Ritratto di hpcsoftwaremail.csi.cuny.edu
Quoting - Gergana Slavova (Intel)

Yup, you're entirely correct, Tim. I was born in Bulgaria so that's my native tongue.

Thanks for the translation, Dmitry. I've actually seen a few Russian movies (my parents like them a lot) so "spasiba" was pretty easy :)

~Gergana

All, Ada Lovelace ... of course ... and sorry about assuming that you were a Russian speaker. Thanks to all ... the install went smoothly, building OpenMPI now, which is also humming along ... Take care, Richard Walsh

Accedere per lasciare un commento.