If you're planning on running with the Intel® MPI Library in an environment that includes Intel® Xeon Phi™ coprocessor cards, you need to make sure all cards contain the appropriate libraries and scripts for the MPI job. This process is probably easier than you think, as shown below. Note that all steps shown here need to be repeated for all Xeon Phi coprocessor cards in your system.
The Intel MPI Library installation already comes with precompiled libraries which have been ported to the Intel Xeon Phi coprocessor architecture. All of these are available in the mic/ directory of your Intel MPI installation. We recommend you simply copy them over to the appropriate bin/ and lib/ directories on the Xeon Phi card:
[user@host] $ scp /opt/intel/impi/<version>/mic/bin/* node0-mic0:/bin
[user@host] $ scp /opt/intel/impi/<version/mic/lib/* node0-mic0:/lib64
In this example, you need to execute either as root or a user with sudo permissions.
If don't have root access on the Intel Xeon Phi card, you can just as easily copy them to your home directory:
[user@host] $ scp /opt/intel/impi/<version>/mic/bin/* node0-mic0:~/bin
[user@host] $ scp /opt/intel/impi/<version>/mic/lib/* node0-mic0:~/lib64
Now, when executing your MPI job, make sure to add those directories to your PATH and LD_LIBRARY_PATH settings:
# Let us know you're planning on using Intel Xeon Phi coprocessor cards
[user@host] $ export I_MPI_MIC=enable
# Create your hosts file which includes the hostnames of the cards
[user@host] $ cat mpi_hosts
# Set your environment
[user@host] $ export PATH=/home/user/bin:$PATH
[user@host] $ export LD_LIBRARY_PATH=/home/user/lib64:$LD_LIBRARY_PATH
# Run the MPI job
[user@host] $ mpirun –f mpi_hosts –perhost 1 –n 2 ~/test_hello
Hello world: rank 0 of 2 running on node0-mic0
Hello world: rank 1 of 2 running on node0-mic1
Finally, you can avoid unnecessary and repeated copying by installing all files in an NFS-shared location between the Xeon host and the Xeon Phi coprocessor.
For more details on how to run Intel MPI Library jobs on the Intel Xeon Phi coprocessors, check out this article.