2. Tools
  3. Intel® oneAPI(Beta)
  4. / Tools / ...

Intel® oneAPI Collective Communications Library(Beta)

Scalable & Efficient Distributed Training for Deep Neural Networks

Implement Multi-Node Communication Patterns

The Intel® oneAPI Collective Communications Library enables developers and researchers to more quickly train newer and deeper models. This is done by using optimized communication patterns to distribute model training across multiple nodes.

The library is designed for easy integration into deep learning (DL) frameworks, whether you are implementing them from scratch or customizing existing ones.

  • Built on top of lower-level communication middleware. MPI and libfabrics transparently support many interconnects, such as Intel® Omni-Path Architecture, InfiniBand*, and Ethernet.
  • Optimized for high performance on Intel® CPUs and GPUs.
  • Allows the tradeoff of compute for communication performance to drive scalability of communication patterns.
  • Enables efficient implementations of collectives that are heavily used for neural network training, including all-gather, all-reduce, and reduce-scatter.

Develop in the Cloud

Get what you need to build, test, and optimize your oneAPI projects for free. With an Intel® DevCloud account, you get 120 days of access to the latest Intel® hardware—CPUs, GPUs, FPGAs—and Intel oneAPI tools and frameworks. No software downloads. No configuration steps. No installations.

Get Access

 

Download the Toolkit

The Intel oneAPI Collective Communications Library is included as part of the Intel® oneAPI Base Toolkit.

Get It Now

Features

Common APIs to Support DL Frameworks

This library exposes a collective API that supports:

  • Commonly used collective operations found in deep learning and machine learning workloads
  • Interoperability with OpenCL™ APIs and SYCL* from The Khronos Group*

 

Unique DL Optimizations

The runtime implementation enables several optimizations that are unavailable in MPI and other communication libraries, including:

  • Asynchronous progress for compute communication overlap
  • Dedication of one or more cores to ensure optimal network use
  • Message prioritization, persistence, and out-of-order execution
  • Collectives in low-precision data types

Documentation & Code Samples

Documentation

View All Documentation

Code Samples

Reference Implementations

Key Specifications

Processors:

  • Intel® Xeon® processors

GPUs:

  • Intel® Processor Graphics Gen9

Memory:

  • Dynamic RAM
  • Intel® Optane® DC persistent memory

Host operating systems:

  • Linux*

Target operating systems:

  • Linux

Languages:

  • Data Parallel C++ (DPC++)
  • C and C++

Compilers:

  • GNU Compiler Collection (GCC)*

Distributed environments:

  • MPI (MPICH-based, Open MPI)
  • Libfabrics

For more information, see the system requirements.

Get Help

Get Help Your success is our success. Access these forum and GitHub* resources when you need assistance.

 

Product and Performance Information

1

Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.

Notice revision #20110804

Give Feedback