Using Multiple DAPL* Providers with the Intel® MPI Library

签署人: James T. 已发布: 09/19/2013 最后更新时间: 09/19/2013

Introduction

If your MPI program sends messages of drastically different sizes (for example, some 16 byte messages, and some 4 megabyte messages), you want optimum performance at all message sizes.  This cannot easily be obtained with a single DAPL* provider.  This is due to latency being a major factor for smaller messages, but bandwidth being more important for larger messages, and providers often making a tradeoff to improve one or the other.  The Intel® MPI Library, as of Version 4.1 Update 1, now supports up to three providers for a single rank of your job.

Details

The ideal scenario is to allow the automatic provider detection to choose the providers for your job.  The automatic detection has additional features for finding the appropriate adapter to use in systems with multiple adapters, and can easily set each rank to use different adapters.  If you want to control the provider selection directly, you can do so with the environment variable I_MPI_DAPL_PROVIDER_LIST.  This takes a comma separated list of up to three providers with the syntax I_MPI_DAPL_PROVIDER_LIST=<Provider1,Provider2,Provider3>, where:

  • Provider1 is used for small messages.  This provider should be a low latency provider.
  • Provider2 is used for large messages inside of a single node.  This is most applicable in jobs involving Intel® Xeon Phi™ Coprocessors, or if you are using DAPL* for intranode communications.
  • Provider3 is used for large messages between multiple nodes.

 The following environment variables can also take a comma separated list of options.  The first option will be applied to the first provider, the second to the second, and the third to the third.  If only one value is set, it is applied to all providers, and if an invalid value is given, all providers will fall back to default values.

  • I_MPI_DAPL_DIRECT_COPY_THRESHOLD (I_MPI_RDMA_EAGER_THRESHOLD, RDMA_IBA_EAGER_THRESHOLD)
  • I_MPI_DAPL_TRANSLATION_CACHE (I_MPI_RDMA_TRANSLATION_CACHE)
  • I_MPI_DAPL_TRANSLATION_CACHE_AVL_TREE
  • I_MPI_DAPL_CONN_EVD_SIZE (I_MPI_RDMA_CONN_EVD_SIZE, I_MPI_CONN_EVD_QLEN)
  • I_MPI_DAPL_RDMA_RNDV_WRITE (I_MPI_RDMA_RNDV_WRITE, I_MPI_USE_RENDEZVOUS_RDMA_WRITE)

 

产品和性能信息

1

英特尔的编译器针对非英特尔微处理器的优化程度可能与英特尔微处理器相同(或不同)。这些优化包括 SSE2、SSE3 和 SSSE3 指令集和其他优化。对于在非英特尔制造的微处理器上进行的优化,英特尔不对相应的可用性、功能或有效性提供担保。该产品中依赖于微处理器的优化仅适用于英特尔微处理器。某些非特定于英特尔微架构的优化保留用于英特尔微处理器。关于此通知涵盖的特定指令集的更多信息,请参阅适用产品的用户指南和参考指南。

通知版本 #20110804