Shared libraries are not found when running a rank on a Xeon Phi™ coprocessor.
error while loading shared libraries: lib*.so: cannot open shared object file: No such file or directory
When launching a job running in either native or symmetric mode, ranks launched on Xeon Phi" coprocessors under the LSF* job scheduler will not correctly set LD_LIBRARY_PATH. Instead, the default LD_LIBRARY_PATH for the coprocessor will be used. If shared libraries are located in other locations, they will not be found.
You can explicitly set LD_LIBRARY_PATH for the ranks running on a coprocessor using -env LD_LIBRARY_PATH $MIC_LD_LIBRARY_PATH.
mpiexec.hydra -n 1 -host node ./a.out : -n 1 -host node-mic0 -env LD_LIBRARY_PATH \
This will set LD_LIBRARY_PATH on the coprocessor to the value of MIC_LD_LIBRARY_PATH on the host, which includes paths to the Intel® Many Integrated Core (MIC) architecture versions of shared libraries for Intel® Software Development products. If you are using additional libraries, you should add their paths to MIC_LD_LIBRARY_PATH and follow the same procedure.
This behavior is corrected in Version 4.1 Update 1 of the Intel® MPI Library. Please visit the Intel® Registration Center to download this version or a later version if you are experiencing this problem.