Upgrading from Intel® Media SDK to Intel® oneAPI Video Processing Library

By Ekaterina Mekhnetsova,

Published:03/22/2021   Last Updated:03/26/2021

This guide describes the 2021.2.2 release of oneVPL. Future behavior may change.

Overview

Intel® oneAPI Video Processing Library (oneVPL) is the successor to Intel® Media SDK.

Intel® Media SDK has a new name, oneVPL, but it is not a new software development kit. oneVPL is the 2.x API continuation of Intel® Media SDK API. The API upgrade provides an enhanced programming model and access to new features available on future hardware not yet released. In general, API updates and new hardware capabilities will be added only to the oneVPL API.

For currently available hardware, Intel® Media SDK and oneVPL provide the same access. Existing applications may continue to use Intel® Media SDK. Developers with existing Intel® Media SDK code should start to evaluate upgrading to be ready for new hardware features. New work, even for currently available hardware, should start with oneVPL.

Please check back for additional information on releases, updates, and more transition guidance.

Intended audience

This guide is for developers who have applications written using Intel® Media SDK.

If you are a developer starting a new project that deals with video processing and you want guidance on choosing the best approach and an appropriate hardware, check out the following:

Where to get oneVPL

You can obtain oneVPL through many channels. Refer to oneVPL Installation Guide for more information.

How long will Intel® Media SDK be available?

In general, new features, API updates and improvements will be added only to oneVPL API. This also includes updates to enable future hardware features.

Intel® Media SDK API is nearing its final update. Implementation updates, such as security patches and critical bug fixes, will continue as needed. This document will be updated as more information becomes available.

Customer feedback is an important part of this process, so please contact your Intel representative or write to the Intel Media Products Forum with any questions or concerns.

Changes in API functional scope

Intel® oneAPI Video Processing Library streamlines and clarifies the scope defined by Intel® Media SDK. Some less used Intel® Media SDK API functionality is removed from oneVPL, while the library continues to retain its focus on video and hardware acceleration. oneVPL provides high control access to hardware accelerated video encode, decode, and frame processing capabilities.

For the library to move forward, it was necessary to re-focus the scope to the core competencies of elementary stream video decode/encode operations plus a few common frame processing algorithms. Functionality outside of the video processing scope is removed. This allows oneVPL to focus on the most important features and functionality of  future target hardware. To work with audio, containers, and other deprecated functionality, consider the hardware-enabled FFmpeg and GStreamer frameworks as described in How to Choose Video Processing Software for Intel Hardware.

Feature and functionality availability in oneVPL and Intel® Media SDK
  Audio Decode Audio Encode Video Decode Video Frame Processing Video Encode GPU driver-specific controls  Opaque memory and plugins
Intel® Media SDK scope Audio Decode is a part of SDK but audio library has not been shipped for years Audio Encode is a part of SDK but audio library has not been shipped for years Yes Yes Yes ENC/PAK for AVC/HEVC Yes
oneVPL scope No No Yes Yes Yes No No

Find more information in the Additional details chapter of this guide as well as in oneVPL Specification.

Clarifying the scope allows oneVPL to grow in new directions. Enhanced programming model features added to oneVPL include:

Enhanced programming model feature Available for
Improved session initialization with MFXLoad All implementations (2021.2.2 release and newer)
Internal allocation All implementations >= API 2.0
Fused decode operation with VPP All implementations >= API 2.1
Simplified decode operation API>=2.0.  Optional feature in CPU implementation

For more details, see the oneVPL Specification, an open specification that oneVPL is based on. This specification describes how new oneVPL implementations can be created, which allows more hardware types and codecs to work with oneVPL.

Note: The specification is still actively evolving. The published spec may not reflect the latest updates published in oneVPL headers. Refer to the oneAPI Specification repository to find the latest updates.

Architecture Summary

Applications link to the dispatcher instead of directly linking to an implementation library. The dynamic loading of the dispatcher allows the application to choose between many library implementations to load at runtime and route API calls to the chosen implementation. On the scheme below, the implementation layer illustrates this choice. Note that static linkage is no longer available on Windows.

oneVPL Architecture
oneVPL Architecture

Different library implementations can target different hardware, which allows each implementation to map hardware capabilities to the oneVPL API. In the diagram above, the available libraries could be designed for different generations of GPUs as well as other hardware types. This design means that you can write an application using the API that will work on many hardware types, even though the code that implements the API functions in each of these libraries is very different. This allows applications to focus on functionality instead of maintaining multiple device-specific code pathways. It also future-proofs the application code since new hardware can be enabled by adding another implementation library without changes to the application code.

In some cases, a single hardware type can have multiple API implementations. For Intel GPUs, the legacy Intel® Media SDK implementation provides access to legacy GPU hardware acceleration, while the oneVPL implementation enables newer hardware. Both oneVPL and Intel® Media SDK implementations are included in the GPU runtime packages. Legacy Intel® Media SDK GPU implementations are most widely available today. Where multiple implementations for a hardware type are available, the dispatcher determines which one to load based on the runtime environment.

Hardware supported by oneVPL and Intel® Media SDK implementations

This guide describes the behavior of oneVPL in 2021.2.2 release. More platforms will be supported by the oneVPL GPU implementation in future releases.

Implementation Supported Hardware
CPU
vplswref64
Intel® Core, Xeon, and Core X-series processors
GPU (Intel® Media SDK)
mfxhw64
Integrated and discrete graphics up to/including Intel® Iris® Xe Graphics
GPU (oneVPL)
mfx-gen
Upcoming hardware

For currently available Intel GPUs, you may use both the Intel® Media SDK and oneVPL dispatchers, depending on the functionality you need. For applications that use removed features, you can continue to use the Intel® Media SDK dispatcher, though its lifetime will not continue indefinitely. All current hardware features are covered by both dispatchers. In the future, new hardware features will only be added to oneVPL implementations and will only be usable with the oneVPL dispatcher. In addition to new hardware features, new enhanced programming model API features will also only be available if you are using the oneVPL dispatcher.

Integrated graphics products covered by the legacy Intel® Media SDK implementation:

Supported APIs

For the remainder of this guide, the following terms will be used to describe APIs available in various configurations.

API Definition
Intel® Media SDK API 1.x API version of Intel® Media SDK
Removed API Intel® Media SDK functionality removed from oneVPL
Core API API functions and features that are shared by both Intel® Media SDK and oneVPL. These API functions do not include removed API functions or features from Intel® Media SDK API, nor do they include new features from oneVPL.
oneVPL 2.x+ API New enhanced programming features introduced in oneVPL
oneVPL API oneVPL API version 2.x, as described above
2.x enhanced API New enhanced programming features added to oneVPL API. oneVPL API consists of core API and 2.x enhanced API.

 

The relationship between these API groups is shown on this Venn diagram:

For details about removed APIs and 2.x enhanced APIs refer to the Additional Details section. Core APIs are supported by both Intel® Media SDK implementation and oneVPL implementation, as shown in this table:

Implementation Hardware Supported Removed APIs Core API 2.x enhanced APIs
CPU
vplswref64
Intel® Core, Xeon, and Core X-series processors No Yes Yes
GPU (Intel® Media SDK)
mfxhw64
Integrated and discrete graphics up to/including Intel® Iris® Xe Graphics Yes* Yes No
GPU (oneVPL)
mfx-gen
Upcoming Intel Graphics hardware No Yes Yes

Note: (*) The use of removed APIs is only possible if the application is compiled with legacy Intel® Media SDK headers, linked with the legacy Intel® Media SDK dispatcher, and targets legacy GPUs.

Note that it is also possible to use the Intel® Media SDK dispatcher with the Intel® Media SDK and oneVPL runtime plugins. This can be a good choice for legacy applications to work on legacy hardware without code changes. However, for the hardware covered by the oneVPL runtime plugin, it is only possible to use Core APIs, as these are the only functions and options that are common between Intel® Media SDK dispatcher and the oneVPL runtime implementation. The use of removed APIs on hardware targeted by the oneVPL plugin will result in a runtime error when the requested symbol cannot be found.

Application portability is maximized with Core API. Core API applications are portable:

  • If compiled with legacy Intel® Media SDK headers or oneVPL headers
  • If used on any hardware supported by Intel® Media SDK or oneVPL
  • With any implementation supporting Intel® Media SDK or oneVPL Specification

Codecs provided by platform and implementation

For GPU hardware, new codec capabilities are added as new hardware becomes available. This is shown in the table below, where:

  • “D” stands for Hardware Decoding
  • “E” stands for Hardware Encoding, Low Power Encoding (VDEnc/Huc)
  • “Es” stands for Hardware (PAK) and Shader (media kernel and VME) Encoding

For currently available GPU hardware, both the legacy Intel® Media SDK and the new oneVPL implementations provide identical hardware access.  However, since Intel® Media SDK is nearing its final update, new features for new hardware will be added only to oneVPL from now on.

Tutorial: Core API portability

A tutorial showcasing transition steps is available in the oneVPL open source repository.

In the tutorial, you will set up a Docker environment with Intel® Media SDK and oneVPL installed, and then compile and run a oneVPL example with both Intel® Media SDK and oneVPL.

For applications that use core API, the steps can be as simple as this:

  1. Include the headers from the oneVPL API directory instead of Intel® Media SDK API directory.
  2. Use MFXLoad instead of MFXInit (as shown in the example code).
  3. Link to the oneVPL dispatcher instead of the Intel® Media SDK dispatcher: 
    • On Linux, use libvpl.so (-lvpl) instead of libmfx.so (-lmfx)
    • On Windows, use vpl.lib instead of libmfx.lib

Note: For oneVPL 2021.1 release, the oneVPL and Intel® Media SDK dispatchers shared the same name (libmfx). The oneVPL dispatcher name was changed since then. For backward compatibility with the oneVPL 2021.1 release, the CPU-only oneVPL dispatcher from 2021.1 release, libmfx, will be provided with the upcoming oneVPL release for a limited time. New applications or applications that can be recompiled should use the new oneVPL dispatcher.

Conclusion

The sections above are intended as a high-level summary of the Intel® Media SDK upgrade and renaming to oneVPL.

To summarize, oneVPL is not a new SDK. It is the 2.x API continuation of Intel® Media SDK API with some scope clarifications to enable future functionality. Intel® Media SDK is nearing its final update. Legacy GPU hardware is supported via the core API, but without the benefits of the new 2.x enhanced API features. Use core API for greatest portability, including support for currently available GPU hardware. Use the new enhanced API features with new hardware for greatest capability and developer convenience.

Additional Details

The following sections provide additional developer details to assist with the transitioning to oneVPL. This still only represents a brief overview of changes introduced with the upgrade from Intel® Media SDK to oneVPL.

Introducing MFXLoad vs. MFXInit

Intel® Media SDK and oneVPL are initialized via their dispatcher:

The dispatcher provides the API interface symbol lookup tables, which are filled in when the implementation library is loaded. Implementation libraries enable oneVPL operations for specific devices as described in Architecture Summary.

In Intel® Media SDK, there were two initialization functions: MFXInit was used in basic cases , while MFXInitEx provided extended parameters. These functions are deprecated starting from API 2.0. They are currently available in oneVPL but marked with deprecation pragmas that cause compiler warnings. In future versions these functions will be completely removed from oneVPL.

MFXLoad replaces MFXInit/MFXInitEx in oneVPL as the entry point function for the dispatcher.  This function enumerates and initializes all available runtimes. This upgrade to the initialization process means that runtimes are able to report capabilities so that applications can choose implementations providing capabilities needed. Transitioning to MFXLoad is the first step towards using the new 2.x enhanced API features, as this allows the use of capability queries.

For example, in the code below, the capabilities for all available runtimes are filtered so that only implementations providing HEVC decode will be considered. The session is created with the highest-ranking implementation able to decode HEVC.

mfxLoader loader = MFXLoad();
mfxConfig cfg = MFXCreateConfig(loader);

// Implementation must provide an HEVC decoder
inCodec.Type     = MFX_VARIANT_TYPE_U32;
inCodec.Data.U32 = MFX_CODEC_HEVC;
MFXSetConfigFilterProperty(
	cfg,
	(mfxU8 *)"mfxImplDescription.mfxDecoderDescription.decoder.CodecID",
	inCodec);

MFXCreateSession(loader, 0, &session);

The following tables map the use of MFXLoad and MFXInit to hardware for applications that use either oneVPL or Intel® Media SDK.

  • For applications that use oneVPL:
    Implementation MFXLoad supported for apps using only Core API MFXLoad supported for 2.x enhanced APIs
    CPU
    vplswref64
    Yes Yes
    GPU (Intel® Media SDK)
    mfxhw64
    Yes No
    GPU (oneVPL)
    mfx-gen
    Yes Yes
  • For applications that use Intel® Media SDK:
    Implementation MFXInit supported (Including features removed in oneVPL) MFXInit supported with Core API MFXLoad supported 

    GPU (Intel® Media SDK)

    mfxhw64

    Yes Yes No
    GPU (oneVPL)
    mfx-gen
    No Yes No

Functionality change: CPU operations

In legacy Intel® Media SDK, operations on CPU were available via two pathways:

  1. Sessions initialized to use the software implementation
  2. Automatic software fallback for hardware implementation sessions 

For option 1, the Intel® Media SDK software implementation is currently only available for Windows.

Option 2 is no longer available with oneVPL. To provide developers with a greater flexibility, automatic software fallback was removed from oneVPL hardware implementations. Hardware capabilities are determined at session initialization, which clarifies expected behavior.

Sessions that are initialized to use software implementation have access to open source codecs, so the use of oneVPL does not mean taking steps back in CPU quality and performance.

Removed Features: Details and Mitigations

This section provides additional details for the set of features that were removed from Intel® Media SDK API and are not present in the Core APIs. The oneVPL Specification contains the full list of Intel® Media SDK functions that are not included in oneVPL.

Removed functions and capabilities fall into these categories:

  • Audio
  • Flexible Encode Infrastructure (FEI)
  • Plugins
  • Opaque memory
  • Obsolete functions

Audio Functions

Audio support is an obsolete feature in Intel® Media SDK and is not included in oneVPL.

Audio support is offered by a variety of other libraries distributed under a wide range of different licenses. Some of the alternatives are available in open source.

If your application needs audio support, FFmpeg and GStreamer offer support for full-featured media pipelines that also include containers, network protocols, and other features. FFmpeg and GStreamer are enabled with hardware acceleration, refer to How to Choose Video Processing Software for Intel Hardware for more details.

 

Flexible Encode Infrastructure (FEI)

Flexible Encode Infrastructure (FEI) is an extension to Intel® Media SDK intended to give additional control over the encoding process for AVC and HEVC encoders.

 

Plugins

oneVPL enables robust video acceleration through API implementations of many different video processing frameworks. Support of an SDK user plugin framework is obsolete. oneVPL is backward compatible with in-house plugins (HEVC decode/encode, VP9 decode/encode, and VP8 decode).

For GPU, H.265 plugin support is now a part of the graphics driver and can be accessed as any other codec.

 

Opaque Memory

Opaque memory support is removed in oneVPL due to its high complexity and a limited number of supported scenarios.

Opaque memory support is replaced with internal memory allocation concept. Refer to oneVPL hello example for the illustration. 

Note that internal memory allocation is not available for legacy hardware. Applications will need to use external allocation if code needs to support hardware which targets the legacy Intel® Media SDK implementation.

 

Obsolete Functions

These functions have been removed. Replacements in code are straightforward.

Function Mitigation Strategy
MFXVideoCORE_SetBufferAllocator() All required non-shared buffers are allocated internally. 
MFXVideoVPP_RunFrameVPPAsyncEx() Use MFXVideoVPP_RunFrameVPPAsync() function instead.

 

Intel® oneVPL Enhanced Programming Model Features

oneVPL introduces new functionality that is not available in Intel® Media SDK. This functionality includes new oneVPL dispatcher functions, oneVPL memory management functions, oneVPL implementation capabilities retrieval functions, and oneVPL session initialization.

Refer to oneVPL specification for more details.

New dispatcher initialization and device enumeration

In oneVPL, MFXInit/MFXInitEx is replaced by MFXLoad(). Refer to legacy-createsession and hello-createsession in the examples directory to compare them. The use of a side-by-side diff tool is recommended to see what changes in the 2.x enhanced API compared to the previous version of API.

Legacy approach to initialization of a oneVPL session:

// Initialize oneVPL session
sts = MFXInit(cliParams.impl, &version, &session);

With the enhanced programming model in oneVPL 2.x API, the dispatcher is initialized like this:

// Initialize oneVPL session
loader = MFXLoad();
cfg = MFXCreateConfig(loader);
 

sts = MFXSetConfigFilterProperty(
	cfg,
	reinterpret_cast<mfxU8 *>(const_cast<char *>("mfxImplDescription.Impl")),
	cliParams.implValue);

sts = MFXCreateSession(loader, 0, &session);

In the code above, MFXLoad() finds available implementations and then MFXSetConfigFilterProperty filters them to find the implementation that supplies the needed functionality.

Applications can walk through implementation capabilities if a more complex selection algorithm is needed. An example is provided with the vpl-inspect tool that shows implementations found by MFXLoad() and their capabilities:

$ ./vpl-inspect

Implementation: oneAPI VPL CPU Reference Impl
  Version: 1.1
  Impl: MFX_IMPL_SOFTWARE
  AccelerationMode: 0
  ApiVersion: 2.1
  License:
  Keywords:
  VendorID: 0x8086
  VendorImplID: 0x0000

 …

Using internal memory

The simple use of external memory allocation is shown in legacy-decode:

sts = MFXVideoDECODE_Init(session, &mfxDecParams);


// Query number required surfaces for decoder
MFXVideoDECODE_QueryIOSurf(session, &mfxDecParams, &decRequest);

// External (application) allocation of decode surfaces
decSurfPool =
	(mfxFrameSurface1 *)calloc(sizeof(mfxFrameSurface1), decRequest.NumFrameSuggested);
sts = AllocateExternalSystemMemorySurfacePool(&decOutBuf,
											  decSurfPool,
											  mfxDecParams.mfx.FrameInfo,
											  decRequest.NumFrameSuggested);

When processing with externally allocated frames, the application must find a free surface in the pool and supply it to oneVPL. It must also use 1.x API synchronization to access the data.

For a decode example using internal allocation, see hello-decode. Note that the application does not need to allocate a surface pool or find free surfaces.

sts = MFXVideoDECODE_DecodeFrameAsync(session,
									  (isDraining) ? NULL :          &bitstream,
									  NULL,
									  &decSurfaceOut,
									  &syncp);

Using Fused Decode + VPP

A new oneVPL feature that is not found in Intel® Media SDK is a combined decode and VPP function:

MFXVideoDECODE_VPP_DecodeFrameAsync(session,
                                  (isDraining) ? NULL : &bitstream,
                                     NULL,
                                     0,
                                     &outSurfaces);

For more information, refer to the hello-decvpp example.

 

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.