SDK provides the USER class of functions to allow user-defined functions, also known as plug-ins, to participate in transcoding operations. When combined with DECODE, VPP and ENCODE, USER provides additional functionality beyond what SDK defines. Figure 1 shows three usage examples. In the first example, the application uses custom decoder in the transcoding pipeline. In the second one, the application adds rotation into the pipeline. In the third example, the application opens two sessions to decode two video streams and then calls the USER class of functions to form a composite stream for encoding.

user-defined functions

Note, that currently USER class of functions supports only video processing functionality. Audio functionality will be added in one of the future releases of the SDK. Splitters and muxers have straightforward API that does not require plug-in extension and developer can create his own component based on provided samples. See “Intel® Media Software Development Kit Splitters and Muxers Sample” manual for more details.

The SDK supports two kinds of plug-in. First one was introduced in version 1.1 of the SDK API. It was called general plug-in and it was intended for general kind of video processing. Although it can support decode and encode functionality its major goal was to support complex video processing filters. It has loosely defined interface and requires significant changes in application to implement.

Second kind of plug-ins has been added in version 1.8 of the SDK API. It is called codec plug-in and it is intended to completely replace one of the internal SDK components: decode, encode or VPP. Codec plug-in uses the same API functions as native SDK component and application can use the same code path for both native SDK component and codec plug-in. For example, to replace AVC decoder in the existent application by HEVC one, all that application developer has to do is to load plugin and to change codec ID during decoder initialization.

There are four different types of plugin. One for general plug-in and three for codec plug-ins:

· general – this is general type that can be used to implement any video processing functionality. It does not replace any SDK class of functions.

  • decode – replaces the SDK DECODE class of functions,
  • encode - replaces the SDK ENCODE class of functions,
  • VPP - replaces the SDK VPP class of functions.

There are two different ways to insert plug-in into the SDK session. First one uses MFXVideoUSER_Register function and gives the application complete control over plugin code location. It can be in separate DLL or part of the application. All types of plug-ins can be loaded this way. Second one uses MFXVideoUSER_Load function and loads one of the preinstalled plug-ins directly from DLL. General types of plug-ins cannot be loaded by this method.

The SDK session can hold only one component of any given class of functions. Therefore, the application could not insert plug-in if the same component has been initialized, or plug-in with the same type has been inserted. For example, if application has initialized native SDK decoder, any attempts to insert decoder plugin in the SDK session fails. The application should use multiple session and session joining mechanism to deal with such pipelines.

The USER class of functions requires the application to use an additional include file, mfxplugin.h, besides the regular SDK include files. No additional library is required at link time.

Include these files:

#include “mfxvideo.h” /* SDK functions in C */
                #include “mfxvideo++.h” /* optional for C++ development */
                #include “mfxplugin.h” /* plugin development */

Link these libraries:

libmfx.lib /* The SDK static dispatcher library */
libmfx.a /* The SDK static dispatcher library */
The following sections describe the USER class of functions  including rules that application developers must follow when programming  with USER functions.