Optimize Networks for the Intel® Neural Compute Stick 2 (Intel® NCS 2)

签署人: Neal P Smith

已发布:11/14/2018   最后更新时间:11/14/2018

This document pertains to the Intel® Distribution of OpenVINO™ toolkit and neural compute devices based on Intel® Movidius™ Myriad™ X such as the Intel® Neural Compute Stick 2 (Intel®  NCS 2).


The Neural Compute Engine (NCE) is an on-chip hardware block available in neural compute devices based on Intel® Movidius™ Myriad™ X. It is designed to run deep neural networks in hardware at much higher speeds than possible with previous generations of the Myriad VPU, still with low power and without compromising accuracy. With two NCEs, the Intel® Movidius™ Myriad™ X architecture is capable of 1 TOPS (1 trillion operations per second) of compute performance on deep neural network inferences.

The model optimizer in OpenVINO™ tookit automatically optimizes networks such that the device can process appropriate layers to take advantage of the NCEs onboard.

Supported Hardware Features:

Networks utilizing the following supported features can be compiled to run as hardware networks on the NCEs. If your network has other non-hardware features, it can still partially run in hardware on the NCE.

  • Multichannel convolution
    •  Matrix-Matrix Multiply/Accumulate
    •  Optional non-overlapping Max and Avg pooling
  • Pooling
    • Overlapping Max and Avg pooling
  • Fully connected
    • Vector-Matrix Multiplay/Accumulate
  • Post processing
    • Bias, Scale, Relu-x, PRelu

Supported Hardware Networks

To see the list of networks that validated to compile and run as hardware networks in this release please refer to the Release Notes.

Using the Intel® Distribution of OpenVINO™ toolkit Inference Engine API with Hardware Networks

No application changes are required to use OpenVINO™ toolkit with hardware networks.

Hardware acceleration with network configuration, HW_STAGES_OPTIMIZATION, is on by default. This can be turned off or back on. The Inference Engine supports different layers for different hardware targets. For a list of supported devices and layers, refer to the Inference Engine Guide.



英特尔的编译器针对非英特尔微处理器的优化程度可能与英特尔微处理器相同(或不同)。这些优化包括 SSE2、SSE3 和 SSSE3 指令集和其他优化。对于在非英特尔制造的微处理器上进行的优化,英特尔不对相应的可用性、功能或有效性提供担保。该产品中依赖于微处理器的优化仅适用于英特尔微处理器。某些非特定于英特尔微架构的优化保留用于英特尔微处理器。关于此通知涵盖的特定指令集的更多信息,请参阅适用产品的用户指南和参考指南。

通知版本 #20110804