Installing the Intel® Distribution for Python* and Intel® Performance Libraries with pip and PyPI

By David A Liu,

Published:12/07/2017   Last Updated:03/07/2018

The Intel® Distribution for Python* provides accelerated performance to some of the most popular packages in the Python ecosystem, and now select packages have the added the option of installing from the Python Package Index (PyPI) using pip. The packages require the use of pip version 9.0.1, and are available utilizing the following instructions:

Performance Packages

The two most popular packages in numerical and scientific work (numpy and scipy) are available with the following commands below.  For more information on the nature of their accelerations and performance benchmarks, please visit the link here.

Also, Intel-optimized-scikit-learn, daal4py (Intel® DAAL in Python) and tbb4py (Intel® TBB for Python) are also available on PyPI now.

Package Name pip command Platform Availability
numpy pip install intel-numpy Linux, Win, macOS(10.12)
scipy pip install intel-scipy
scikit-learn pip install intel-scikit-learn
daal4y pip install daal4py
tbb4py pip install tbb4py

Optimized Python packages such as intel-scikit-learn, intel-scipy and pydaal utilize intel-numpy.

Based on PyPI's dependency resolution on Intel variants, If one installs intel-numpy, one would also get mkl_fft and mkl_random (with NumPy). Similarly, if one installs intel-scipy, one would also get intel-numpy along with SciPy. And,  if one installs intel-scikit-learn, one would also get intel-numpy,intel-scipy along with Scikit-Learn.

Note: If standard NumPy, SciPy and Scikit-Learn packages are already installed, the packages must be uninstalled  before installing the Intel® variants of these packages(intel-numpy etc) to avoid any conflicts. As mentioned earlier, pydaal uses intel-numpy, hence it is important to first remove the standard Numpy library(if installed) and then install pydaal.

To uninstall existing packages, run the command:

pip uninstall numpy scipy scikit-learn -y  

Specialized NumPy packages

Several specialized Intel packages act as a complement to numpy and scipy, which provide accelerated Fast Fourier Transforms and improved Random functionality through the MKL when paired with numpy and scipy.

Package Name pip command Platform Availability
mkl_fft pip install mkl_fft Linux, Win, macOS(10.12)
mkl_random pip install mkl_random

Note: In order to utilize these packages, the standard NumPy installation must be removed first using the command: pip uninstall numpy -y

 

Intel® Runtime Packages

The runtime packages are built runtime distributable libraries that allow for dispatch of vectorization on Intel hardware.  For Python packages that depend on these runtimes, they can be individually downloaded as well.  For more information, please visit the link here.

Package Name pip command Platform Availability
mkl pip install mkl Linux, Win, macOS(10.12)
ipp pip install ipp
daal pip install daal
intel-openmp pip install intel-openmp
tbb pip install tbb
impi pip install impi Linux, Win

Development only packages

For those building their own Python packages with Intel® Parallel Studio XE or building and linking with the Intel® Performance Libraries, the devel packages assist in providing the development runtimes pre-built for testing, and are available with the following commands:

Package Name pip command Platform Availability
mkl-devel pip install mkl-devel Linux, Win, macOS(10.12)
ipp-devel pip install ipp-devel
daal-devel pip install daal-devel

Troubleshooting

While `pip install`-ing any package, if installation fails with the following error message :

zlib.error: Error -5 while decompressing data: incomplete or truncated stream

retry after running the following command: rm -rf ~/.cache/pip

 

 

Product and Performance Information

1

Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.

Notice revision #20110804