Accelerating Media, Video & Computer Vision Processing: Which Tool Do I Use?

 

Intel has many awesome software development tools to optimize media and video solutions and applications across edge to the cloud. Common usages that developers need include:

  • Accelerating video transcoding performance and taking full advantage Intel hardware capabilities and accelerators
  • Transitioning to higher frame rates and more efficient formats like HEVC
  • Integrating intelligent vision and deep learning
  • Innovating immersive experiences like 360 video, real-time cloud gaming, and AR/VR

But sometimes, it's hard to figure out just which development tool or combinartion of tools are best for your particular needs and usages. Below see a list of Intel® Software Development Tools that developers can use to build their media and video applications. 


For Fast Video or Image Processing

  • Use Intel® Media SDK for Embedded Linux* for Client, Mobile, Embedded Applications & Devices
  • Use Intel® Media SDK for Windows* for Server, Desktop, Client, Mobile, Embedded Applications & Devices

Developing for:

  • Intel® Core™ or Intel® Core™ M processors 
  • Select SKUs of Intel® Celeron™, Intel® Pentium® and Intel Atom® processors with Intel® HD Graphics supporting Intel® Quick Sync Video
  • Client, mobile and embedded devices - desktop or mobile media applications
  • OS - Windows* and Embedded Linux*
  • An Open Source version is also available at Github under the MIT license 

Uses and Needs

  • Fast video playback, encode, processing, media formats conversion or video conferencing
  • Accelerated processing of RAW video or images
  • Screen capture
  • Used with smart cameras across drones, phones, editors/players, network video recorders, and connected cars
  • Supports HEVC, AVC, MPEG-2 and audio codecs
  • Depending on the solution and usage, this software tool is commonly also used with Intel® Computer Vision SDK, Intel® SDK for OpenCL™ Applications, Intel® System Studio.

Free Download


intel media server studioFor Enterprise, Data Center/Visual Cloud, Broadcasting & Embedded Video Transcoding - Use Intel® Media Server Studio for Linux*

Three editions are available:

FREE Community

Essentials

Professional

Developing for:

Format Support HEVC, AVC, MPEG-2 and more

Uses and Needs

  • Used in enterprise, data center, cloud-based media solutions. Common usages includes broadcast, over-the-top (OTT), video-on-demand (VOD), video streaming, video conferencing, visual cloud, cloud gaming, and more.
  • High-density and fast video decode, encode, transcode
  • Optimize performance of Media/GPU pipeline 
  • Enhanced graphics programmability or visual analytics (for use with OpenCL™ applications)
  • Low-level control over encode quality
  • Debug, analysis and performance/quality optimization tools
  • Speed transition to real-time 4K HEVC
  • Need to measure visual quality (Video Quality Caliper)
  • Screen capture

 Free Download & Paid Edition Options 


For Real-time Communications via Web Applications - Use Intel® Collaboration Suite for WebRTC

This Client SDK builds on top of the W3C standard WebRTC APIs to accelerate development of real-time communications (RTC), including broadcast, peer-to-peer, conference mode communications, and online gaming/VR streaming. 

Use with Andrioid*, web (JavaScript* built), iOS* and Windows* applications. 

Free Download


To Customize Solutions, Balance CPU/GPU workloads and Utilize Intel® Processor Graphics - Use Intel® SDK for OpenCL™ Applications

Developing for:

General purpose GPU acceleration on select Intel® processors (see technical specifications). OpenCL primarily targets execution units. An increasing number of extensions are being added to Intel processors to make the benefits of Intel’s fixed function hardware blocks accessible to OpenCL applications.

  • Provides the ability to customize heterogeneous compute applications and accelerate performance
  • Balance media workloads between CPU and GPU - uniquely offload compute to Intel® Graphics Technology tailored to your application needs

Free Download


For Smart Video, Computer Vision & Deep Learning Inference - Use the Intel® Computer Vision SDK

To accelerate computer vision solutions and integrate deep learning inference, use the Intel Computer Vision SDK.

  • Easily harness the performance of computer vision accelerators from Intel
  • Add your own custom kernels into your workload pipeline
  • Quickly deploy computer vision algorithms with deep-learning support using the included Deep Learning Deployment Toolkit from Intel
  • Create OpenVX* workload graphs with the intuitive and easy-to-use Vision Algorithm Designer
  • Commonly used with Intel® Media SDK and Intel® Media Server Studio.
  • Usages include smart video, object recognition, computer vision, deep learning inference across vertical solutions like digital surveillance, retail, manufacturing, smart home/cities, healthcare, autonomous driving, and more.

Free Download


To Analyze Video Performance & Quality, use Intel® VTune™ Amplifier

It is a performance profiler that gathers CPU, GPU, threading, OpenCL, and bandwidth metrics to find media processing bottlenecks.Quickly analyze data - filter results on timeline, code source, and on a GPU architecture diagram showing VDBox, VEBox, EU utilization, bus bandwidth.


Intel System StudioTo Improve System Bring-up, Boost Performance, Power Usage, Reliability for System and IoT Device Applications - use Intel® System Studio

This cross-platform tool suite includes optimizing compilers, highly tuned libraries, analyzers, debug tools, and access to cloud connectors and 400+ sensors.

  • Helps system and IoT developers improve system bring-up, boost performance, power usage and reliability of system & IoT device applications.
  • Includes Intel® VTune™ Amplifier and debugger tools
Free Download

For FPGA - Use Video & Image Processing Suite MegaCore Functions 

(part of Intel® Quartus® Prime Software Suite IP Catalog)
 

Developing for:

  • All Altera FPGA families
  • Video and image processing applications, such as video surveillance, broadcast, video conferencing, medical and military imaging and automotive displays

Uses and Needs

  • For design, simulation, verification of hardware bit streams for FPGA devices
  • optimized building blocks for deinterlacing, color space conversion, alpha blending, scaling and more

Intel® C for Media and Related Tools

We recently open-sourced CM runtime together with the Intel media driver and CM compiler. We also list the CM package in 01.org.

The source code can be accessed at these locations:

  • Intel Media Driver for VAAPI and Intel® C for Media Runtime: on GitHub 
  • Intel C for Media Compiler and examples: on GitHub 
  • Intel Graphics Compiler: on GitHub 

 

OpenCL and the OpenCL logo are trademarks of Apple Inc. used by permission by Khronos.

For more complete information about compiler optimizations, see our Optimization Notice.