Intel® Optimization for TensorFlow* Installation Guide

TensorFlow* is a widely-used machine learning framework in the deep learning arena, demanding efficient utilization of computational resources. In order to take full advantage of Intel® architecture and to extract maximum performance, the TensorFlow framework has been optimized using Intel® Math Kernel Library for Deep Neural Networks (Intel® MKL-DNN) primitives, a popular performance library for deep learning applications. For more information on the optimizations as well as performance data, see this blog post TensorFlow* Optimizations on Modern Intel® Architecture .

Anaconda* has now made it convenient for the AI community to enable high-performance-computing in TensorFlow. Starting from TensorFlow v1.9, Anaconda has and will continue to build TensorFlow using Intel® Math Kernel Library for Deep Neural Networks (Intel® MKL-DNN) primitives to deliver maximum performance in your CPU.

This install guide features several methods to obtain Intel Optimized TensorFlow including off-the-shelf packages or building one from source that are conveniently categorized into Binaries, Docker Images, Build from Source.  

Quick Links

Anaconda

*Supports Py27 and Py36

PIP Wheels

Docker Containers

Build from source

1. Binaries

Install the latest Intel® Optimization for TensorFlow* from Anaconda* Cloud

Available for Linux*, Windows*, MacOS*

TensorFlow* version: 1.14.0 (Linux)

Installation instructions:

If you don't have conda package manager, download and install Anaconda

Linux and MacOS

Open Anaconda prompt and use the following instruction

conda install tensorflow

In case your anaconda channel is not the highest priority channel by default(or you are not sure), use the following command to make sure you get the right TensorFlow with Intel optimizations

conda install tensorflow -c anaconda

Windows

Open Anaconda prompt and use the following instruction

conda install tensorflow-mkl

(or)

conda install tensorflow-mkl -c anaconda

Besides the install method described above, Intel Optimization for TensorFlow is distributed as wheels, docker images and conda package on Intel channel. Follow one of the installation procedures to get Intel-optimized TensorFlow.

 

Note: All binaries distributed by Intel were built against the TensorFlow v1.13.1 tag in a centOS container with gcc 4.8.5 and glibc 2.17 with the following compiler flags (shown below as passed to bazel*)

--cxxopt=-D_GLIBCXX_USE_CXX11_ABI=0 --copt=-march=corei7-avx --copt=-mtune=core-avx-i --copt=-O3 

Install the latest Intel® Optimization for TensorFlow* from Intel Channel

Available for Linux*

TensorFlow* version: 1.13.1

Installation instructions:

Open Anaconda prompt and use the following instruction. Available for Python 2.7 and 3.6.

conda install tensorflow -c intel

Get Intel® Optimization for TensorFlow* from Intel® Distribution for Python

Available for Linux*

TensorFlow* version: 1.13.1

Installation instructions:

Open Anaconda prompt and use the following instruction. Available for Python 2.7 and 3.6.

conda create -n IDP intelpython3_full -c intel

(or)

conda create -n IDP intelpython2_full -c intel

Install the Intel® Optimization for TensorFlow* Wheel Into an Existing Python* Installation Through PIP

Available for Linux*

TensorFlow version: 1.13.1

Installation instructions:

Run the below instruction to install the wheel into an existing Python* installation, preferably Intel® Distribution for Python*. Python versions supported are 2.7, 3.4, 3.5, 3.6

pip install intel-tensorflow

Warning on ISA above AVX2:

"The TensorFlow library was not compiled to use <Instruction set> instructions, but these are available on your machine and could speed up CPU computations."

This warning can be ignored since Intel MKL-DNN library with which the TensorFlow is compiled utilizes the latest Instruction sets available in your processor to perform compute-intensive operations.

Older CPUs may not support this version of TensorFlow and may result in "Illegal instruction (core dumped)" error.

2. Docker Images

Get Intel® Optimization for TensorFlow*  Docker Images 

Google DL Containers

Starting version 1.14, Google released DL containers for TensorFlow on CPU optimized with Intel MKL DNN by default. The TensorFlow CPU container names are in the format "tf-cpu.<framework version>" and support Python3.  Below is a sample command to download the docker image locally and launch the container for TensorFlow 1.14

docker run -d -p 8080:8080 -v /home:/home  gcr.io/deeplearning-platform-release/tf-cpu.1-14

This command will start the TensorFlow 1.14 with MKL DNN enabled in detached mode, bind the running Jupyter server to port 8080 on the local machine, and mount local /home directory to /home in the container. The running JupyterLab instance can be accessed at localhost:8080.

To launch an interactive bash instance of the docker container, run the below command

docker run -v /home:/home -it gcr.io/deeplearning-platform-release/tf-cpu.1-14 bash

Obtaining and Running Intel Containers

These docker images are all published at http://hub.docker.com in intelaipg/intel-optimized-tensorflow namespace and can be pulled with the following command:

 

docker pull docker.io/intelaipg/intel-optimized-tensorflow:latest

 

For example, to run the python 2.7 data science container directly, simply

docker run -it -p 8888:8888 intelaipg/intel-optimized-tensorflow

And then go to your browser on http://localhost:8888/


For those who want to navigate through the browser, follow the link

https://hub.docker.com/r/intelaipg/intel-optimized-tensorflow/

Available Container Configurations 

The following tags/configurations are available:

Docker Container Configurations
ConfigurationTags
Python 2.7 with jupyter and sample notebooks

latest

latest-mkl

1.13.1-mkl

Python 2.7 with TensorFlow development toolchain

latest-devel

latest-devel-mkl

1.13.1-devel-mkl

1.13.1-devel

Python 3.5 with jupyter and sample notebooks

latest-py3

latest-mkl-py3

1.13.1-mkl-py3

Python 3.5 with TensorFlow development toolchain

latest-devel-py3

latest-devel-mkl-py3

1.13.1-devel-mkl-py3

1.13.1-devel-py3

To get the latest Release Notes on Intel-optimized TensorFlow, please refer this article

3. Build from Source

Build TensorFlow from Source with Intel® MKL

Linux build

Building TensorFlow from source is not recommended. However, if instructions provided above do not work due to unsupported ISA, you can always build from source.

Building TensorFlow from source code requires Bazel installation, refer to the instructions here, Installing Bazel.

Installation instructions:

  1. Ensure numpy, keras-applications, keras-preprocessing, pip, six, wheel, mock packages are installed in the Python environment where TensorFlow is being built and installed.
  2. Clone the TensorFlow source code and checkout a branch of your preference
    git clone https://github.com/tensorflow/tensorflow
    git checkout r1.14
    
  3. Run "./configure" from the TensorFlow source directory
  4. Execute the following commands to create a pip package that can be used to install the optimized TensorFlow build.
    • PATH can be changed to point to a specific version of GCC compiler:
      export PATH=/PATH//bin:$PATH
      
    • LD_LIBRARY_PATH can also be to new:
      export LD_LIBRARY_PATH=/PATH//lib64:$LD_LIBRARY_PATH
      
    • Set the compiler flags support by the GCC on your machine  to build TensorFlow with Intel® Math Kernel Library (Intel® MKL). Ensure appropriate "march" and "mtune" flags are set. Refer the gcc online docs to know the flags supported by your GCC version.
    • #e.g
      bazel build --config=mkl --cxxopt=-D_GLIBCXX_USE_CXX11_ABI=0 --copt=-march=sandybridge --copt=-mtune=ivybridge --copt=-O3 //tensorflow/tools/pip_package:build_pip_package
      

      (or)

    • Alternatively, set appropriate "Instruction sets" flags you want to compile the library with: 

Flags set in the command below will add AVX, AVX2 and AVX512 instructions which will result in "illegal instruction" errors when you use older CPUs. If you want to build on older CPUs, set the instruction flags accordingly

bazel build --config=mkl -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-mavx512f --copt=-mavx512pf --copt=-mavx512cd --copt=-mavx512er //tensorflow/tools/pip_package:build_pip_package

        3. Install the optimized TensorFlow wheel

     bazel-bin/tensorflow/tools/pip_package/build_pip_package ~/path_to_save_wheel
     pip install --upgrade --user ~/path_to_save_wheel/<wheel_name.whl>

Windows Build

Prerequisites

Install the below Visual C++ 2015 build tools from https://visualstudio.microsoft.com/vs/older-downloads/

  • Microsoft Visual C++ 2015 Redistributable Update 3
  • Microsoft Build Tools 2015 Update 3

Installation

  1. Refer to Linux Section and follow Steps 1 through 3
  2. To build TensorFlow with MKL-DNN support, we need two additional steps.
      set PATH=%PATH%;output_dir\external\mkl_windows\lib   

         3. Bazel build with the with "mkl" flag and the "output_dir" to use the right mkl libs

       bazel --output_base=output_dir build --config=mkl --config=opt //tensorflow/tools/pip_package:build_pip_package

          4. Install the optimized TensorFlow wheel

       bazel-bin\tensorflow\tools\pip_package\build_pip_package C:\temp\path_to_save_wheel
       pip install C:\temp\path_to_save_wheel\<wheel_name.whl>

            

Sanity Check

Once Intel-optimized TensorFlow is installed, running the below command must print "True" if Intel MKLDNN optimizations are present.

python -c "import tensorflow; print(tensorflow.pywrap_tensorflow.IsMklEnabled())"

Additional Capabilities and Known Issues

  1. Intel-optimized TensorFlow enables Intel MKL calls by default, If at any point you wish to disable Intel MKL primitive calls, this can be disabled by setting TF_DISABLE_MKL flag to 1 before running your TensorFlow script.
    export TF_DISABLE_MKL=1

    However, note that this flag will only disable Intel MKL-DNN calls, but not MKL-ML calls. 

    Although Intel MKL DNN is responsible for most optimizations, certain ops are optimized by MKL-ML library, including matmul, transpose, etc. Disabling MKL-ML calls are not supported by TF_DISABLE_MKL flag at present and Intel is working with Google to add this functionality.  

  2. CPU affinity settings in Anaconda's TensorFlow:  If MKL enabled TensorFlow is installed from the anaconda channel (not Intel channel), the "import tensorflow" command sets the KMP_BLOCKTIME and OMP_PROC_BIND environment variables if not already set. However, these variables may have effects on other libraries such as Numpy/Scipy which use OpenMP or MKL. Alternatively, you can either set preferred values or unset them after importing TensorFlow. More details available in the TensorFlow GitHub issue
    import tensorflow # this sets KMP_BLOCKTIME and OMP_PROC_BIND 
    import os 
    # delete the existing values 
    del os.environ['OMP_PROC_BIND'] 
    del os.environ['KMP_BLOCKTIME']

 

Useful Resources

Support

Please submit your queries at the Intel AI Frameworks forum for additional questions or support.

Archived Wheels

Version

Wheels(2.7, 3.5, 3.6)

1.6

https://anaconda.org/intel/tensorflow/1.6.0/download/tensorflow-1.6.0-cp27-cp27mu-linux_x86_64.whl

(or)

*/tensorflow-1.6.0-cp35-cp35mu-linux_x86_64.whl

(or)

*/tensorflow-1.6.0-cp36-cp36mu-linux_x86_64.whl

1.9

https://storage.googleapis.com/intel-optimized-tensorflow/tensorflow-1.9.0-cp27-cp27mu-linux_x86_64.whl

https://storage.googleapis.com/intel-optimized-tensorflow/tensorflow-1.9.0-cp35-cp35m-linux_x86_64.whl

https://storage.googleapis.com/intel-optimized-tensorflow/tensorflow-1.9.0-cp36-cp36m-linux_x86_64.whl

1.10

https://storage.googleapis.com/intel-optimized-tensorflow/tensorflow-1.10.0-cp27-cp27mu-linux_x86_64.whl

https://storage.googleapis.com/intel-optimized-tensorflow/tensorflow-1.10.0-cp35-cp35m-linux_x86_64.whl

 https://storage.googleapis.com/intel-optimized-tensorflow/tensorflow-1.10.0-cp36-cp36m-linux_x86_64.whl

1.11

https://storage.googleapis.com/intel-optimized-tensorflow/tensorflow-1.11.0-cp27-cp27mu-linux_x86_64.whl

https://storage.googleapis.com/intel-optimized-tensorflow/tensorflow-1.11.0-cp34-cp34m-linux_x86_64.whl

https://storage.googleapis.com/intel-optimized-tensorflow/tensorflow-1.11.0-cp35-cp35m-linux_x86_64.whl

https://storage.googleapis.com/intel-optimized-tensorflow/tensorflow-1.11.0-cp36-cp36m-linux_x86_64.whl

 

 

For more complete information about compiler optimizations, see our Optimization Notice.

23 comments

Top

Installing tensorflow-mkl did not work with my Anaconda installation on Windows. Got error

AttributeError: module 'keras_applications' has no attribute 'set_keras_submodules'

I fared better with Docker ... BUT

1. the performance of the script does not scale with enabled CPUs in the docker!

2. the sample Keras script for MNIST dataset yield much worse performance compared to NVIDIA trained one. What on earth could be the reason for that???

Thanks for providing this useful Guide.

I find the code  "python -c "import tensorflow; print(tensorflow.pywrap_tensorflow.IsMklEnabled()"" in Sanity Check missing close parentheses for  the "print" functioin.

Thanks

WangLi

Thanks for providing this useful Guide.

I find the code  "python -c "import tensorflow; print(tensorflow.pywrap_tensorflow.IsMklEnabled()"" in Sanity Check missing close parentheses for  the "print" functioin.

Thanks

WangLi

 

Would it be possible to get a wheel for a Windows Machine?

Hi There,

I tried installing the conda install tensorflow the  everything have to seemed to work fine. I tested using the python example. But I'm after the c++ api of the Intel optimised tensorlfow, with the above installation how do i go about running an example like this:

#include <stdio.h>
#include <tensorflow/c/c_api.h>

int main() {
  printf("Hello from TensorFlow C library version %s\n", TF_Version());
  return 0;
}

I cant seem to find the include files with the above installation?

Will I need to build tensorflow from source?

Am I missing something here completely? Any help would be much appericated, thanks.

 

 

 

 

Hi,

Intel Optimiziations for TensorFlow sounds super useful.

I haven't tried it yet as I see there's Linux support only (unless I build from source).
I'm using OSX and I was wondering:

  1. Will there be an Intel TensoFlow whl for OSX at any point ? (If so, when ?)
  2. As far as I understand this runs on the CPU. Would it be possible at any point to use TensorFlow with GPU acceleration on Intel GPUs (e.g. Intel Iris Pro ) ? (If so, when ?)

Thank you,
George

I am unable to complete Bazel with the following error:

ERROR: /root/.cache/bazel/_bazel_potero/bb908902e7a4c1214e822194ebe85aca/external/double_conversion/BUILD.bazel:12:1: C++ compilation of rule '@double_conversion//:double-conversion' failed (Exit 1)
gcc: error: unrecognized command line option '-mavx512f'
gcc: error: unrecognized command line option '-mavx512pf'
gcc: error: unrecognized command line option '-mavx512cd'
gcc: error: unrecognized command line option '-mavx512er'
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 6.517s, Critical Path: 1.78s
INFO: 3 processes: 3 local.
FAILED: Build did NOT complete successfully

Any help would be aoppreciated.

Preethi: You could probably be using an older gcc(<4.9) version and trying to pass avx512 flags. Please check the GCC documentation for you version to see if these flags are available.

Thank you, Preethi, but the original question still stands: will this speed up Tensorflow on my Core  i7-7500U CPU, or are those optimizations only effective for high-end Xeon processors? Thanks!

Preethi: Intel MKL-DNN, which is used to build the optimized TensorFlow supports Core processors. Refer the System Requirements section in the GitHub page here: https://github.com/intel/mkl-dnn

 

Hello,

I have a humble laptop with an Intel(R) Core(TM) i7-7500U CPU @ 2.70GHz, running Windows 10 (oh, what a shame!), Anaconda and Jupyter Lab. Would I benefit at all of installing tensorflow-mkl? If there are any benefits, are there some pre-requisites/pre-installs that I would need to do?

In more general terms, what are the processors that would benefit from this install and from the MKL-DNN library as such?

Preethi Venkatesh: Hi Benjamin, The new version of  default TensorFlow (1.9) from the Anaconda Channel is build using MKL-DNN optimizations and is available for download now. just do a "conda install tensorflow". No other prerequisites needed.

Thanks!

Ben

MAD\rcooksey's picture

Is this method still relevant? https://software.intel.com/en-us/articles/using-intel-distribution-for-python-with-anaconda

It avoids the pip <-> conda conflict.

 

 

Pages

Add a Comment

Have a technical question? Visit our forums. Have site or software product issues? Contact support.