Release Notes for Intel® Distribution of OpenVINO™ toolkit 2019

NOTE: For the Release Notes for the previous version, refer to Release Notes for Intel® Distribution of OpenVINO™ toolkit 2018

Introduction

The Intel® Distribution of OpenVINO™ toolkit is a comprehensive toolkit for quickly developing applications and solutions that emulate human vision. Based on Convolutional Neural Networks (CNNs), the toolkit extends CV workloads across Intel® hardware, maximizing performance.

The Intel® Distribution of OpenVINO™ toolkit:

  • Enables CNN-based deep learning inference on the edge.
  • Supports heterogeneous execution across Intel CV accelerators, using a common API for the CPU, Intel® Integrated Graphics, Intel® Movidius™ Neural Compute Stick (NCS), Intel® Neural Compute Stick 2, Intel® Vision Accelerator Design with Intel® Movidius™ VPUs and Intel® FPGAs.
  • Speeds time-to-market through an easy-to-use library of CV functions and pre-optimized kernels.
  • Includes optimized calls for CV standards, including OpenCV*, OpenCL™, and OpenVX*.

New and Changed in the Release 1

Executive summary

  • Intel® Distribution of OpenVINO™ Toolkit 2019 R1 includes functional and security updates. Users should update to the latest version.

  • This release extends the list of supported OSes to include Apple macOS* for inference on CPU in preview mode. Install all the key components of the toolkit such as Model Optimizer, Inference Engine, OpenCV and others for accelerated inference on systems with macOS.
  • Parallelism schemes are switched from OpenMP* to Threading Building Blocks (TBB) to increase performance in a multi-network scenario. Most customers' deployment pipeline executes multiple network combinations and TBB gives the most optimal performance in such use cases.
  • Added support for many new operations in ONNX*, TensorFlow* and MXNet* frameworks. Topologies like Tiny YOLO v3, full DeepLab v3, bi-directional LSTMs now can be run using Deep Learning Deployment toolkit for optimized inference.
  • More than 10 new pre-trained models are added including gaze estimation, action recognition encoder/decoder,  text recognition, instance segmentation networks to expand to newer use cases. Few models with binary weights are introduced to further boost the performance for these networks.
  • FPGA plugin migrated to DLA 2019 R1 with new bitstreams for Intel® Arria® 10 FPGA GX Development Kit, Intel® Programmable Acceleration Card with Intel® Arria® 10 FPGA GX and Intel® Vision Accelerator Design with an Intel® Arria® 10 FPGA boards. The toolkit now supports automatic scheduling between multiple FPGA devices.

Backward incompatible changes in compare with v.2018

  • Changed default installation locations to /opt/intel/openvino_<version> for Linux* OS and C:\Program Files (x86)\IntelSWTools\openvino_<version> for Windows* OS.  For simplicity, symbolic links to the latest installation are also created: /opt/intel/openvino/ for Linux and C:\Program Files (x86)\IntelSWTools\openvino for Windows.
  • Changed default environment variable from the INTEL_CVSDK_DIR to INTEL_OPENVINO_DIR.
  • Deprecated and removed Computer Vision Algorithms (CVA) component from the toolkit.
  • Removed product documentation from the package. For documentation for the latest releases, see the new documentation site: https://docs.openvinotoolkit.org/
  • Removed Open Model Zoo models binaries from the package. To get the latest models, use the Model Downloader script from <INSTALL_DIR>/openvino_<version>/deployment_tools/model_downloader/.

Model Optimizer

Common changes

  • Updated the IR version from 4 to 5. The IR of version 2 can be generated using the `--generate_deprecated_IR_V2` command line parameter.
  • Implemented experimental feature to generate the IR with "Shape" layers. The feature makes possible to change model input shapes in the Inference Engine (using dedicated reshape API) instead of re-generating models in the Model Optimizer. The feature is enabled by providing command line parameter "--keep_shape_ops" to the Model Optimizer.
  • Introduced new graph transformation API to perform graph modifications in the Model Optimizer.
  • Added ability to enable/disable the Model Optimizer extension using the environment variables MO_ENABLED_TRANSFORMS and MO_DISABLED_TRANSFORMS respectively.
  • Fixed issue with Deconvolution shape inference for case with stride not equal to 1.
  • Added Concat optimization pass that removes excess edges between Concat operations.

ONNX*

  • Added support of the following ONNX* operations: ArgMax, Clip, Exp, DetectionOutput, PriorBox, RNN, GRU with parameters "direction", "activations", "clip", "hidden_size", "linear_before_reset". LSTM parameters "activations", "clip", "direction" are now supported.
  • Extended support of the ConvTranspose operation to support ND.
  • Resolved issue with the Gemm operation with biases.

TensorFlow*

  • Added support of the following TensorFlow* operations: ReverseSequence, ReverseV2, ZerosLike, Exp, Sum.
  • Added support of the following TensorFlow* topologies: quantized image classification topologies, TensorFlow Object Detection API RFCN version 1.10+, Tiny YOLO v3, full DeepLab v3 without need to remove pre-processing part.
  • Added support of batch size more than 1 for TensorFlow Object Detection API Faster/Mask RCNNs and RFCNs.

Caffe*

  • Added support of the following Caffe* operations: StridedSlice, Bias.
  • Caffe fallback for shape inference is deprecated.

MXNet*

  • Added support of the following MXNet* operations: Embedding, Zero with "shape" equal to 0, RNN with mode="gru", "rnn_relu", "rnn_tanh" and parameters "num_layer", "bidirectional", "clip". 
  • Added support of bidirectional LSTMs.
  • Added support of LSTMs with batch size more than 1 and multiple layers.
  • Fixed loading Gluon models with attributes equal to "None".

Inference Engine

Common changes

  • Automatic preprocessing (resize + layout conversion) now fully supports batches.
  • Added support for LSTM, GRU and RNN sequence layers to the NN Builder API.

CPU plugin

  • Parallelism schemes are switched from OpenMP* to Threading Building Blocks (TBB) to increase performance in a multi-network scenario. TBB gives the most optimal performance when running multiple networks at once (testing showed up to 3.5X improvement). However, for some particular networks, this change may lead to either performance degradation or improvement. Please see documentation for more details.
  • Improved support for Low-Precision 8-bit Integer inference:
    • Introduced a new Python* calibration tool. It expands support for 8-bit Integer inference to new domains of neural networks. The support of dataset formats and network accuracy metrics are highly customized and allow you to calibrate, for example, semantic segmentation models (Unet2d).
    • Introduced support for ReLU6, Fully Connected, resample layers in INT8.
    • Significantly decreased size of memory used for storing intermediate tensors data in INT8 data type. This decreased virtual memory footprint for 2x-4x depending on topology.
  • Updated Intel® MKL-DNN version to v0.18.
  • Added support for layers:
    • DepthToSpace
    • Expand
    • Fill
    • Range
    • ReverseSequence
    • ShuffleChannels
    • SpaceToDepth
    • Squeeze
    • StridedSlice
    • Unsqueeze
    • GRUCell
    • RNNCell
    • LSTMSequence
    • GRUSequence
    • RNNSequence
  • Added new Activation types:
    • Exp
    • Not
  • Added new Eltwise operations:
    • Min
    • Max
    • Sub
    • Div
    • Squared_diff
    • Floor_mod
    • Pow
    • Logical_AND
    • Logical_OR
    • Logical_XOR
    • Less
    • Less_equal
    • Greater
    • Greater_equal
    • Equal
    • Not_equal
  • Eltwise supports broadcasting.
  • Fixed memory allocation for Int8 topologies.
  • Improved logic which detects necessary reorders
  • Improved support of Split layer with multiple connections to outputs
  • Added fuse of FullyConnected and ReLU
  • Fixed SEGFAULT for machines with the big number of possible CPUs

GPU plugin

  • Added support for recurrent networks, including RNN, GRU and LSTM layers. The functionality now is fully aligned with CPU plugin functionality.
  • Added support for layers: DepthToSpace, Gather, ShuffleChannels, StridedSlice, ReverseSequence
  • Added support for new parameters in existing layers: 
    • Activation with type "not".
    • Eltwise new operations: "squared_diff, "greater", "greater_equal", "less", "less_equal", "or", "xor", "and", "sub", "div", "min", "max".
    • New clip options for DetectionOutput and Proposal layers.
    • "bilinear" mode for PSROIPooling.
  • Updated Compute Library for Deep Neural Networks (clDNN) version to 13.1.
  • Fixed infinite performance regression on subsequent runs which occurred on several topologies.
  • Fixed minor memory leak on every inference iteration.


FPGA Plugin

  • Migrated to DLA 2019R1:
    • New bitstreams for Intel® Arria® 10 FPGA GX Development Kit, Intel® Programmable Acceleration Card (PAC) with Intel® Arria® 10 FPGA GX and Intel® Vision Accelerator Design with an Intel® Arria® 10 FPGA boards.
    • Added dilated convolution primitive support which enables dilation, DSSD, semantic-segmentation-adas-0001, road-segmentation-adas-0001 topologies.
    • Numerous bug fixes.
  • Migrated to OpenCL™ 18.1.1 for Intel® Vision Accelerator Design with an Intel® Arria® 10 FPGA, Intel® Arria® 10 FPGA GX Development Kit boards.
  • Updated BSP for Intel® PAC with Intel® Arria® 10 FPGA GX board from DCP 1.1 to DCP 1.2 version, OpenCL™ RTE 17.1.1.
  • Implemented multi FPGA device automatic scheduling feature in the Inference Engine FPGA plugin. The feature is demonstrated on security_camera_barrier_demo app and shows 1.7x scalability on two Intel® Arria® 10 FPGA GX Development Kit boards.

MYRIAD Plugin

  • Improved performance for a number of networks to close performance gap with 2018R5 release.
  • Reduced time required for networks compilation.
  • Support for IR-integrated statistics (if available) to improve inference accuracy. This feature is described in the MYRIAD plugin documentation.
  • Improved network allocation process and memory management on VPU.
  • Stability and accuracy fixes and improvements.
  • Hardware accelerated Deconvolution layer.
  • Hardware accelerated Clamp layer.
  • Support for Eltwise Sub layer.
  • Support for Argmax layer.
  • Support for LSTMCell layer for LSTM:CTPM model.

HDDL Plugin

  • Supported accelerator card with F75114 USB IO expander reset device, previously supported TCA9535 I2C IO expander only.
  • Added bypass scheduler.
  • Updated OpenSSL* with CVE fixing.
  • Optimized HDDL Service output.
  • Added KEY_VPU_HDDL_BIND_DEVICE to configure whether the network should bind to a VPU.
  • Auto detect VPU number (total_device_num=0 in hddl_autoboot.config).
  • Run without reset device (abort_if_hw_reset_failed=false in hddl_autoboot.config) .

GNA plugin

  • The GNA plugin status switched from preview to gold.
  • Introduced support for heterogeneous mode, so the layers that are not supported on GNA (such as normalize) can be run on CPU.

OpenCV*

  • Version updated to 4.1.0
  • opencv_dnn module now integrates Inference Engine via the NN Builder API introduced in the OpenVINO 2018R5 release.
  • opencv_videoio module introduces plugin-based backends on Linux platforms.

OpenVX*

  • Optical Flow Pyramid (LK) accuracy issues on GPU are fixed.
  • Bits pack/unpack nodes functionality is corrected.
  • Major Color Copy Pipeline (CCP) sample update and refactoring.

Samples and Demos

Python*

  • Introduced the following demos: 3d_segmentation_demo, action_recognition, instance_segmentation_demo.
  • greengrass_samples were removed from the distribution. Actual version can be found on GitHub: https://github.com/intel/Edge-Analytics-FaaS/releases
  • Improved samples/demos documentation.

C++

  • Removed warnings on Windows, switched to a more strict warnings level (-Wall) on Linux. Fixed klockwork issues.
  • benchmark_app was updated with support of statistics reports collecting (including per-layer performance management counters). Latency calculation was added for asynchronous mode. New serialization logic for executable graph info was added (currently for CPU only).
  • Added support for multiple FPGA and THROUGHPUT mode for CPU in the  security_barrier_camera_demo.
  • interactive_face_detection_demo was updated with smoothing of personal attributes, improvement of emotions visualization, bug fixes.
  • HDDL plugin was mentioned in the documentation for all supported samples/demos.
  • multichannel_demo was updated with corrected handling of number of connected web cameras.
  • mask_rcnn_demo was updated to support a new version of the TensorFlow Object Detection API Mask-RCNNs topologies.
  • Improved build scripts for samples and demos. For Windows OS, the scripts now not only generate Microsoft Visual Studio* solutions but also build them. Scripts output was unified for all supported operating systems. Build script is improved to support Raspbian* OS.

Common Changes

  • Python and C++ Classification samples/demos are updated with unified Classification Output.
  • Samples and Demos documentation was updated.
  • Bug fixes.

Tools

  • Python* Calibration Tool is implemented. Sample configuration files for the Calibration Tool are available for download from the Intel® Developer Zone.

Open Model Zoo

Extended the Open Model Zoo, which includes additional CNN pre-trained models and pre-generated Intermediate Representations (.xml + .bin):

  • [NEW] action-recognition-0001-decoder. This is an general-purpose action recognition model for Kinetics-400 dataset. The model uses Video Transformer approach with ResNet34 encoder.Please refer to the kinetics dataset specification to see list of action that are recognised by this model.This model is only decoder part of the whole pipeline. It accepts stack of frame embeddings, computed by action-recognition-0001-encoder, and produces prediction on input video. Video frames should be sampled to cover ~1 second fragment (i.e. skip every second frame in 30 fps video).
  • [NEW] action-recognition-0001-encoder. This is an general-purpose action recognition model for Kinetics-400 dataset. The model uses Video Transformer approach with ResNet34 encoder.Please refer to the kinetics dataset specification to see list of action that are recognised by this model.This model is only encoder part of the whole pipeline. It accepts video frame and produces embedding.Use action-recognition-0001-decoder to produce prediction from embeddings of 16 frames.Video frames should be sampled to cover ~1 second fragment (i.e. skip every second frame in 30 fps video).
  • [NEW] driver-action-recognition-adas-0002-encoder. This is an action recognition model for the driver monitoring use case. The model uses Video Transformer approach with MobileNetv2 encoder. It is able to recognize the following actions: drinking, doing hair or making up, operating the radio, reaching behind, safe driving, talking on the phone, texting.This model is only encoder part of the whole pipeline. It accepts video frame and produces embedding. Use driver-action-recognition-adas-0002-decoder to produce prediction from embeddings of 16 frames. Video frames should be sampled to cover ~1 second fragment (i.e. skip every second frame in 30 fps video).
  • [NEW] driver-action-recognition-adas-0002-decoder. This is an action recognition model for the driver monitoring use case. The model uses Video Transformer approach with MobileNetv2 encoder. It is able to recognize the following actions: drinking, doing hair or making up, operating the radio, reaching behind, safe driving, talking on the phone, texting.This model is only decoder part of the whole pipeline. It accepts stack of frame embeddings, computed by driver-action-recognition-adas-0002-encoder, and produces prediction on input video. Video frames should be sampled to cover ~1 second fragment (i.e. skip every second frame in 30 fps video).
  • [NEW] gaze-estimation-adas-0002. This is a custom VGG-like convolutional neural network for gaze direction estimation.
  • [NEW] instance-segmentation-security-0033. This model is an instance segmentation network for 80 classes of objects.It is a Mask-RCNN-like model with ResNeXt152 backbone and Feature Pyramid Networks block for feature maps refinement.
  • [NEW] instance-segmentation-security-0049. This model is an instance segmentation network for 80 classes of objects.It is a Mask-RCNN-like model with ResNet50 backbone, Feature Pyramid Networks block for feature maps refinement and relatively light segmentation head.
  • [NEW] person-detection-action-recognition-teacher-0002. This is an action detector for the Smart Classroom scenario. It is based on the RMNet backbone that includes depth-wise convolutions to reduce the amount of computations for the 3x3 convolution block. The first SSD head from 1/16 scale feature map has four clustered prior boxes and outputs detected persons (two class detector). The second SSD-based head predicts actions of the detected persons. Possible actions: standing, writing, demonstrating.
  • [NEW] text-recognition-0012. This is a network for text recognition scenario. It consists of VGG16-like backbone and bidirectional LSTM encoder-decoder.The network is able to recognize case-insensitive alpha-numeric text (36 unique symbols).
  • [NEW] text-detection-0002. This is a text detector based on PixelLink architecture with MobileNetV2 as a backbone for indoor/outdoor scenes.
  • [NEW] face-detection-adas-binary-0001. This is a face detector for driver monitoring and similar scenarios. The network features a default MobileNet backbone that includes depth-wise convolutions to reduce the amount of computation for the 3x3 convolution block.
  • [NEW] pedestrian-detection-adas-binary-0001. Pedestrian detection network based on SSD framework with tuned MobileNet v1 as a feature extractor.Some layers of MobileNet v1 are binary and use I1 arithm.
  • [NEW] vehicle-detection-adas-binary-0001. This is a vehicle detection network based on an SSD framework with tuned MobileNet v1 as a feature extractor and using binary layer for speedup.This detecector was created by binarization the vehicle-detection-adas-0002.
  • [NEW] resnet50-binary-0001. This is a classical classification network for 1000 classes trained on ImageNet.The difference is that most convolutional layers were replaced by binary once that can be implemented as XNOR+POPCOUN operations.Only input, final and shortcut layers were kept as FP32, all the rest convolutional layers are replaced by BinaryConvolution layers.

  • facial-landmarks-35-adas-0002. This is a custom-architecture convolution neural network for 35 facial landmarks estimation.

  • person-attributes-recognition-crossroad-0230. This model presents a person attributes classification algorithm analysis scenario. It produces probability of person attributions existing on the sample and a position of two point on sample, whiches can be used for color prob (like, color picker in graphical editors).

  • person-detection-action-recognition-0005. This is an action detector for the Smart Classroom scenario. It is based on the RMNet backbone that includes depth-wise convolutions to reduce the amount of computations for the 3x3 convolution block. The first SSD head from 1/16 scale feature map has four clustered prior boxes and outputs detected persons (two class detector). The second SSD-based head predicts actions of the detected persons. Possible actions: sitting, standing, raising hand.

  • single-image-super-resolution-1032. An Attention-Based Approach for Single Image Super Resolution but with reduced number of channels and changes in network architecture. It enhances the resolution of the input image by a factor of 4.

  • single-image-super-resolution-1033. An Attention-Based Approach for Single Image Super Resolution but with reduced number of channels and changes in network architecture. It enhances the resolution of the input image by a factor of 3.

Model Downloader

Model Downloader configuration file is extended to support the following public models in Caffe*, TensorFlow* and MXNet* formats:

ModelFormat
brain_tumor_segmentationMXNet
mobilenet-v1-1.0-224Caffe
mobilenet-v2-1.4-224TensorFlow
mobilenet-v2Caffe
faster_rcnn_inception_v2_cocoTensorFlow
ctpn (LSTM: CTPN)TensorFlow
deeplabv3 (DeepLab-v3+)TensorFlow
ssd_mobilenet_v1_cocoTensorFlow
faster_rcnn_resnet101_cocoTensorFlow

Preview Features Terminology

    A preview feature is functionality that is being introduced to gain early developer feedback. Comments, questions, and suggestions related to preview features are encouraged and should be submitted to the forum at https://software.intel.com/en-us/forums/computer-vision.

    The key properties of a preview feature are:

    •     It is intended to have a high quality implementation
    •     There is no guarantee of future existence or compatibility.

    NOTE: A preview feature is subject to change in the future. It may be removed or altered in future releases. Changes to a preview feature do NOT require the a deprecation and deletion process. Using a preview feature in a production code base is discouraged.

     

    Known Issues

    IDDescriptionComponentWorkaround
    1A number of OpenVX* issues are not addressed yet, please see "Known issue" section in the Release Notes for Intel® Distribution of OpenVINO™ toolkit v.2018OpenVX*N/A
    2Unsupported Dynamic Shapes for Caffe* layersModel OptimizerN/A
    3Some TensorFlow operations are not supported, but only a limited set of different operations can be successfully converted.Model OptimizerEnable unsupported ops through Model Optimizer  extensions and IE custom layers
    4Only TensorFlow models with FP32 Placeholders. If there is non FP32 Placeholder, the next immediate operation after this Placeholder should be a Cast operation that converts to FP32.Model OptimizerRebuild your model to include a FP32 placeholder only or add cast operations
    5Only TensorFlow models with FP32 weights are supported.Model OptimizerRebuild your model to have FP32 weights only
    6Embedded preprocessing in Caffe models is not supported and ignored.Model OptimizerPass preprocessing parameters through Model Optimizer CLI parameters
    7Releasing the the plugin's pointer before inference completion might cause a crash.Inference EngineRelease the plugin pointer at the end of the application, when inference is done.
    8If Intel OpenMP was initialized before OpenCL, OpenCL will hang. This means initialization or executing the FPGA will hang too.Inference EngineInitialize FPGA or Heterogeneous with the FPGA plugin priority before the CPU plugin.
    9The performance of the first iteration of the samples for networks executing on FPGA is much lower than the performance of the next iterations.Inference EngineUse the -ni <number> -pc to tet the real performance of inference on FPGA.
    10To select the best bitstream for a custom network, evaluate all available bitstreams and choose the bitstream with the best performance and accuracy. Use validation_app to collect accuracy and performance data for the validation dataset.Inference Engine 
    11The setBatch method works only for topology which has batch as first dimension for all tensorsInference EngineUse the reshape() method. It is applicable for all topology types.
    12Multiple OpenMP runtime initialization is possible if you are using MKL and Inference Engine simultaneouslyInference EngineUse apreloaded iomp5 dynamic library
    13Completion Callback is called in case of successfull execution of infer request onlyInference EngineUse Wait to get notified about errors in infer request
    14Resize feature works only on machines which support SSE4.2 instruction set.Inference EngineN/A
    15GPU plugin fails to execute several synchronous requests for the same Executable Network from multiple threads simultaneously.Inference EngineN/A
    16While loading extension modules, the Model Optimizer reports an "No module named 'extensions.<module_name>'" internal error and does not load any extensions from the specified directory. It happens only if you use the --extensions command line option with a directory with the base name extensions but that is not the <INSTALL_DIR>/deployment_tools/model_optimizer/extensions directory.Model OptimizerUse different base name for the directory with custom extensions.
    17

    If you have the Intel® Media Server Studio installed on your CentOS* 7.4 machine, the installation of the OpenCV dependencies may cause a libva.so version conflict.

    Installation

    Remove libva and reinstall it manually from the Intel Media Server Studio RPM package:

    # yum remove libva
    # yum install ffmpeg-libs
    # rpm -ivh rpm -ivh /path/to/mss/distributive/libva-*.rpm
    18

    Low-precision 8-bit integer inference on CPU might have problems with calibration and final big accuracy drop for a calibrated ResNet* model if the model is converted with default options of the Model Optimizer.

    Inference Engine

    Use --disable_resnet_optimization for ResNet models which are going to be calibrated and executed in low-precision 8-bit integer inference on CPU
    19

    Shape Inference for Reshape layer might not work correctly for TensorFlow models if its shape and parameters are dynamically depend on other layers (for example, for the pre-trainned vehicle-license-plate-detection-barrier-0107 model).

    Inference EngineGenerate IR using MO with --input_shape option
    20

    Models with fixed dimensions in the `dim` attribute of the Reshape layer can't be resized.

    Inference EngineGenerate IR using MO with --input_shape option
    21

    Shape inference for Interp layer works for almost all cases, except for Caffe models with fixed width and height parameters (for example, semantic-segmentation-adas-0001).

    Inference EngineGenerate IR using MO with --input_shape option
    22

    Keyboard layout issue running GUI installer from vnc client (user can experience symbols mismatch by typing text in installer GUI due to QT and VNC compatibility issue)

    InstallerUse the CLI installer version by running ./install.sh from the package directory instead of the GUI installer (./install_GUI.sh) 
    23Media SDK samples build failure due to issue with the environment.Media SDK

    Run the following command:

    export PKG_CONFIG_PATH=/opt/intel/mediasdk/lib64/pkgconfig:$PKG_CONFIG_PATH

     

    24

    Model Optimizer does not support element-wise operations supported by CPU/GPU plugin: Div, Squared_diff, Floor_mod, Logical_AND, Logical_OR, Logical_XOR, Less, Less_equal, Greater, Greater_equal, Equal, Not_equal.

    Model Optimizer

    Implement extractor for these operations and add them to the "<MO_INSTALL_DIR>/extensions/front/<framework>/" directory.

    25Accuracy loss in applications which use locale different from en_USInference EngineForce set locale in user's application to en_US.UTF-8
    26InferenceEngine::CNNNetwork::reshape fails for YoloV3Inference EngineN/A
    27Blue screen crash on Windows Server 2016 (Multistreaming)Inference EngineN/A
    28Poor performance on some GEMM and FullyConnected layersInference EngineBuild IE with OMP threading  or continue using TBB threading, but build IE with mkl-dnn's SGEMM implementation (-DGEMM=JIT).
    29TensorFlow mobilenet models accuracy drop is more than 1% for INT8 in comparison with FP32Inference EngineN/A
    30Exception at load model to plugin while import IE and pytorch at same timeInference Engine

    Build PyTorch* without the MKL-DNN support:

    git clone https://github.com/pytorch/pytorch.git
    cd pytorch
    git submodule update --init --recursive
    export USE_MKLDNN=OFF
    sudo setup.py install                                
    31CPU plugin shows different performance on Windows 10 and Ubuntu 16Inference EngineN/A
    32FP32 and INT8 XceptionSoftmax return different results on AVX2Inference EngineN/A
    33Calibration Tool can not handle FullyConnected layer during accuracy drop per layer collection with -fcf command line optionCalibration ToolN/A
    34Calibration Tool supports metrics in percent onlyCalibration ToolMultiply threshold by 100.
    35Passing empty filename to cv::VideoWriter::open causes crash in GStreamer backendOpenCVCheck filename for emptiness before passing it to VideoWriter
    36TensorFlow faster-rcnn-inception-v2 accuracy is lower than CPU reference (3.7% of relative difference) on Intel® Movidius™ Neural Compute Stick, Intel® Neural Compute Stick 2 and Intel® Vision Accelerator Design with Intel® Movidius™ VPUsInference EngineN/A
    37Intel® Neural Compute Stick 2 and Intel® Vision Accelerator Design with Intel® Movidius™ VPUs sometimes works unstable when executing multiple graphs on the same deviceInference EngineChange the order of executable networks creation on the device.
    38Performance drop (~10% relative difference) on TensorFlow tiny-yolo-v2 vs OpenVINO 2018R5 on Intel® Neural Compute Stick 2 and Intel® Vision Accelerator Design with Intel® Movidius™ VPUsInference EngineUse the previous version of OpenVINO for maximum performance on this topology.
    39Topologies containing LSTMCell layer with batch dimension > 1 demonstrate non-optimal performance on GPUInference EngineN/A
    40Inference on FPGA freezes when run in Docker*Inference EngineN/A
    41Performance drop on FPGA (all platforms) vs OpenVINO 2018 R5 on a set of topologies: Caffe mobilenet v1 224, Caffe mobilenet v2, Caffe ssd512, Caffe ssd300, Caffe squeezenet 1.0, Caffe googlenet v1, DenseNet familyInference EngineUse previous version of the OpenVINO toolkit for maximum performance on this topologies
    42FPGA plugin shows low accuracy for TensorFlow MobileNet topologies on FP11 bitstreams on Intel® Arria® 10 FPGA GX Development KitInference EngineRun these topologies in FP16 version
    43When running the speech_sample application, GNA plugin shows low accuracy on convolution based topologies if the input scale factor is about 16535 (half of the int16 maximum value).Inference EngineRun the speech_sample application with a smaller custom scale factor (-q). Values less than 1000 should improve accuracy.
     GNA plugin does not provide full support for Concat layer - it only supports Concat layers that have two inputs, and one of them is a network input.Inference EngineN/A

    Included in This Release

    The Intel® Distribution of OpenVINO™ toolkit is available in three versions:

    • Intel® Distribution of OpenVINO™ toolkit for Windows*
    • Intel® Distribution of OpenVINO™ toolkit for Linux*
    • Intel® Distribution of OpenVINO™ toolkit for Linux* with FPGA Support
    • Intel® Distribution of OpenVINO™ toolkit for macOS*
    Install Location/File NameDescription
    Deep Learning Model OptimizerModel optimization tool for your trained models
    Deep Learning Inference EngineUnified API to integrate the inference with application logic
    OpenCV*OpenCV Community version compiled for Intel hardware. Includes PVL libraries for computer vision
    Intel® Media SDK libraries (open source version)Eases the integration between the OpenVINO™ toolkit and the Intel® Media SDK.
    Intel OpenVX* runtimeIntel's implementation of the OpenVX* run-time optimized for running on Intel® hardware (CPU, GPU, IPU)
    Intel® Graphics Compute Runtime for OpenCL™ Enables OpenCL™ on the GPU/CPU for Intel® processors
    Intel® FPGA Deep Learning Acceleration Suite, including pre-compiled bitstreams

    Implementations of the most common CNN topologies to enable image classification and ease the adoption of FPGAs for AI developers.

    Includes pre-compiled bitstream samples for the Intel® Programmable Acceleration Card with Intel® Arria® 10 GX FPGA and the Arria® 10 GX FPGA Development Kit.

    Intel® FPGA SDK for OpenCL™ software technologyThe Intel® FPGA RTE for OpenCL™ provides utilities, host runtime libraries, drivers, and RTE-specific libraries and files
    Intel® Distribution of OpenVINO™ toolkit documentationDeveloper guides and other documentation. Available from the Intel® Distribution of OpenVINO™ toolkit product site
    Open Model ZooThis component includes documentation for the latest stable set of pre-trained models from https://github.com/opencv/open_model_zoo. The models in a binary form can be downloaded using the Model Downloader tool
    Computer Vision Samples and DemosSamples that illustrate Inference Engine API usage and demos that demonstrate how you can use features of Intel® Distribution of OpenVINO™ toolkit in your application.

     

    Where to Download This Release

    https://software.intel.com/en-us/OpenVINO-toolkit/choose-download

     

    System Requirements

    Development Platform

    Hardware

    • 6th-8th Generation Intel® Core™
    • Intel® Xeon® v5 family
    • Intel® Xeon® v6 family

    Operating Systems

    • Ubuntu* 16.04.3 long-term support (LTS), 64-bit
    • CentOS* 7.4, 64-bit
    • Windows* 10, 64-bit
    • macOS* 10.14, 64-bit

    Target Platform (choose one processor with one corresponding operating system)

    Your requirements may vary, depending on which product version you use.

    Intel® CPU processors with corresponding operating systems

    • 6th-8th Generation Intel® Core™ and Intel® Xeon® processor with operating system options:
      • Ubuntu* 16.04.3 long-term support (LTS), 64-bit
      • CentOS* 7.4, 64-bit
      • Windows* 10, 64-bit
    • Intel® Pentium® processor N4200/5, N3350/5, N3450/5 with Intel® HD Graphics
      • Ubuntu* 16.04.3 long-term support (LTS), 64-bit
      • Yocto Project* Poky Jethro* v2.0.3, 64-bit
      • macOS* 10.14, 64-bit

    Intel® Integrated Graphics processors with corresponding operating systems

    NOTE: This installation requires drivers that are not included in the Intel® Distribution of OpenVINO™ toolkit package

    • 6th - 8th Generation Intel® Core™ processor with Intel® Iris® Pro graphics and Intel® HD Graphics
      • Ubuntu* 16.04.3 long-term support (LTS), 64-bit
      • CentOS* 7.4, 64-bit
    • 6th - 8th Generation Intel® Xeon® processor with Intel® Iris® Pro graphics and Intel® HD Graphics

      NOTE: A chipset that supports processor graphics is required for Intel® Xeon® processors. Processor graphics are not included in all processors. See https://ark.intel.com/ for information about your processor.

      • Ubuntu* 16.04.3 long-term support (LTS), 64-bit
      • CentOS* 7.4, 64-bit
    • Intel® Pentium® processor N4200/5, N3350/5, N3450/5 with Intel® HD Graphics
      • Ubuntu* 16.04.3 long-term support (LTS), 64-bit
      • Yocto Project* Poky Jethro* v2.0.3, 64-bit

    Intel® FPGA processors with corresponding operating systems

    NOTES:
    Only for the Intel® Distribution of OpenVINO™ toolkit for Linux with FPGA Support
    OpenCV* and OpenVX functions must be run against the CPU or Intel® Integrated Graphics to get all required drivers and tools

    • Intel® Arria® FPGA 10 GX development kit and Intel® Programmable Acceleration Card with Intel® Arria® 10 GX FPGA
      • Ubuntu* 16.04.3 long-term support (LTS), 64-bit
      • CentOS* 7.4, 64-bit

    Intel® Movidius™ Neural Compute Stick, Intel® Neural Compute Stick 2, and  Intel® Vision Accelerator Design with Intel® Movidius™ VPUs with corresponding operating systems

    • Ubuntu* 16.04.3 long-term support (LTS), 64-bit
    • CentOS* 7.4, 64-bit
    • Windows* 10, 64-bit

    Helpful Links

    Note: Links open in a new window.

    OpenVINO™ toolkit Home Page: https://software.intel.com/en-us/OpenVINO-toolkit

    OpenVINO™ toolkit Documentation: https://software.intel.com/en-us/OpenVINO-toolkit/documentation/featured

     

    Legal Information

    You may not use or facilitate the use of this document in connection with any infringement or other legal analysis concerning Intel products described herein. You agree to grant Intel a non-exclusive, royalty-free license to any patent claim thereafter drafted which includes subject matter disclosed herein.

    No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.

    All information provided here is subject to change without notice. Contact your Intel representative to obtain the latest Intel product specifications and roadmaps.

    The products described may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request.

    Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn more at http://www.intel.com/ or from the OEM or retailer.

    No computer system can be absolutely secure.

    Intel, Arria, Core, Movidius, Xeon, OpenVINO, and the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries.

    OpenCL and the OpenCL logo are trademarks of Apple Inc. used by permission by Khronos

    *Other names and brands may be claimed as the property of others.

    Copyright © 2019, Intel Corporation. All rights reserved.

    For more complete information about compiler optimizations, see our Optimization Notice.