This paper addresses how the Computer Vision system architecture is increasing in complexity and evolving into new industries and use cases. It articulates the advantage of using Intel® software tools such as Intel® Media SDK, Intel® Distribution of OpenVINO™ Toolkit, software libraries, Intel® Active Management Technology (Intel® AMT), Intel® vPro™ technology and Intel® hardware to enable these computer vision systems. The main areas this article covers are Smart cameras that hardware accelerate encoding, transcoding and decoding, computer vision gateway’s hardware and software, video analytics, deep learning and integration with the Internet of Things.
The next generation of Computer Vision Systems have complex and integrated computer architectures that are rapidly evolving. Technologies from computer fields such as HD graphics, real-time video encoding and transcoding, scalable video streaming and storage, artificial intelligence and the Internet of Things are expanding the abilities of modern Computer Vision Systems.
Computer Vision Systems are also expanding into new market segments such as autonomous vehicles, smart lighting, industrial machine vision and drones. They are increasingly used in city surveillance, retail interactions, banking and finance surveillance, traffic monitoring, railway supervision, education management, airport security and many other market applications (some are industry markets i.e. retail some are application i.e. airports). For example, an automotive dealership may use a computer vision system in order to ensure the security of property on their auto lot. A factory may use a computer vision system to monitor equipment and determine if there are any malfunctions that need to be serviced. A hospital or a government building may use a V system to monitor the people that are entering restricted areas. And retail stores often use them to monitor customer transactions and provide reporting to police and authorities in the case of a robbery.
Each of these scenarios not only changes how the SV system will be architected and installed, but also they affect the policies and procedures enforced during the use of the system.
Intel focuses on enabling technologies that are flexible, horizontal and scalable so that businesses are able to implement their own regulations and procedures when deciding how to use the recorded video.
Current video systems use camera sources to create video and stream it to local or cloud-based storage where it is analyzed and any necessary actions are taken. The scalability of the system is limited by the amount of available bandwidth and storage, as well as, the ability to rapidly analyze and determine if alerts or actions need to be taken.
Intel® technologies address customer challenges by focusing on each area of the technology stack and providing solutions that can be mixed and integrated to build next-generation systems.
Let’s focus on four areas where Computer Vision System technology is rapidly evolving:
Smart cameras powered by Intel technology will continue to increase resolutions to HD and 4K/UHD. Intel® processors based on next generation Intel® architecture, such as SkyLake, provide hardware acceleration for media operations in HEVC, H.264 to 4K30, VP9, and JPEG based on Intel® Quick Sync Video technology (GPU). Higher resolutions are achieved with lower power and bandwidth requirements.
The Intel® Media SDK delivers a higher level set of libraries, tools, and sample code that define cross-platform APIs. This SDK provides a unified abstraction layer for media decode and encode. That is capable of accessing hardware acceleration and includes an OpenCL interface to allow custom media processing capabilities.
Higher resolutions and video pipelines enable the creation of multiple video streams that lower bandwidth requirements, enabling more cameras to be deployed in a computer vision system ultimately producing an increase in the number of simultaneous video feeds.
Computer Vision Gateways serve as aggregation points for analytics services, network management, security management, and integration with automation services and the Internet of Things.
By moving high-end computation closer to the video sources, Intel based gateways will be able to analyze the video onsite without moving video over the Internet or to centralized cloud based data centers. This reduces Computer Vision System bandwidth requirements and increases scalability from tens of video streams to hundreds or thousands of simultaneous video streams.
Intel offers computer vision systems based on a range of processors so the computational needs of your Computer Vision System can grow with your business needs. Systems based on the Intel Atom® platform, the Intel® Core™ processor family and the Intel® Xeon® platform span the range of video computational needs.
Here is a profile of a potential mid-level and an advanced gateway.
Intel® Celeron® processor family +24ch 1080p decoding
APL: 4k H.265, 16-20ch 1080p decode capability and VA
(Data Center / High-Performance NVR )
Intel® Xeon® processor E3-1225v5: Hi End NVR/Transcoder/VA Server
Intel® Core™ i3/5/7 family-based NVR with VA
Intel provides a large number of libraries that system integrators and vendors can use to create computer vision services. The algorithms span a range of applications from providing statistical information about the video to allowing higher level descriptions and actionable information.
These computer vision libraries make it possible to build custom analytics algorithms that are hardware optimized for image processing, cryptography, math processing & neural networks routines, data analytics and high availability threading:
• Intel® Integrated Performance Primitives (IPP)
• Intel® Math Kernel Library (MKL)
• Intel® Data Analytics Acceleration Library (DAAL)
• Intel® Threading Building Blocks (TBB)
Higher level analytics can be customize for different situations. For example, if several smart cameras are monitoring an area that only allows restricted personal, artificial intelligence and deep learning can be used to train the system to identify authorized people and sound the alarm for unauthorized people.
Intel provides the Intel® Distribution of OpenVINO™ Toolkit-based OpenVX*, OpenCL™ software technology, and OpenCV*. Intel Distribution of OpenVINO Toolkit is a specialized set of libraries for enabling computer vision processing.
Computational power near the video source is also important in reducing latency in timing sensitive application. In autonomous vehicle applications, pedestrian recognition needs to happen immediately and avoid the latencies associated with the cloud.
Enabling analysis of video nearer the smart camera reduces network transit for high-bandwidth video streams. Low bandwidth alerts and automatic actions can be triggered.
Intel® Distribution for Caffe* is a set of libraries from machine learning to enable deep learning applications that can be deployed on a Computer Vision System. By using advanced smart cameras and video gateways, multiple video streams can be encoded in an efficient manner.
Deep learning techniques are used in tasks such as object recognition, object classification, object or person tracking, intrusion detection, scene analysis, counting people, facial detection, facial recognition, recognition of age gender or behavior, license plate recognition, vehicle detection, 3D depth analysis, navigation and many other useful applications of video analysis.
Once you have the correct hardware and software layers you also need a manageability layer. Intel provides Intel® AMT technology, enabled by Intel® vPro™ Technology, which will allow you to remotely administer your entire computer vision system. This includes upgrading firmware, managing deployment and decommissioning lifecycle steps, and monitoring in real-time the status of each of the components in your DDS system.
Intel silicon and software is enabling next-generation architecture of computer vision systems and security and surveillance systems.
Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.
Notice revision #20110804