Surface Type Neutral Transcoding

Performance wise, software SDK library (running CPU instructions) prefers system memory I/O, and SDK platform implementation (accelerated by platform graphic devices) prefers video memory surface I/O. The application needs to manage both surface types (thus two data paths in a transcoding AàB) to achieve the best performance in both cases.

The SDK provides a third surface type: opaque surface. With opaque surface, the SDK will map the surface type to either system memory buffer or video memory surface at runtime. The application only needs to manage one surface type, or one transcoding data path.

It is recommended the application use opaque surfaces for any transcoding intermediate data. For example, the transcoding pipeline can be DECODE à Opaque Surfaces à VPP à Opaque Surfaces à ENCODE. It is possible to copy an opaque surface to a “real” surface through a VPP operation.

The application uses the following procedure to use opaque surface, assuming a transcoding pipeline SDK A à SDK B:

·        As described in section Surface Pool Allocation, the application queries SDK component A and B and calculates the surface pool size. The application needs to use MFX_IOPATTERN_IN_OPAQUE_MEMORY and/or MFX_IOPATTERN_OUT_OPAQUE_MEMORY while specifying the I/O pattern. It is possible that SDK component A returns a different memory type than SDK component B, as the QueryIOSurf function returns the native allocation type and size. In this case, the surface pool type and size should follow only one SDK component: either A or B.

·        The application allocates the surface pool, which is an array of the mfxFrameSurface1 structures. Within the structure, specify Data.Y= Data.U= Data.V= Data.A= Data.MemId=0 for all array members.

·        During initialization, the application communicates the allocated surface pool to both SDK components by attaching the mfxExtOpaqueSurfaceAlloc structure as part of the initialization parameters. The application needs to use MFX_IOPATTERN_IN_OPAQUE_MEMORY and/or MFX_IOPATTERN_OUT_OPAQUE_MEMORY while specifying the I/O pattern.

·        During decoding, encoding, and video processing, the application manages the surface pool and passes individual frame surface to SDK component A and B as described in section Decoding Procedures, section Encoding Procedures, and section Video Processing Procedures, respectively.

Example 12 shows the opaque procedure sample code.

Since the SDK manages the association of opaque surface to “real” surface types internally, the application cannot read the content of opaque surfaces. Also the application does not get any opaque-type surface allocation requests if the application specifies an external frame allocator.

If the application shares opaque surfaces among different SDK sessions, the application must join the sessions before SDK component initialization and ensure that all joined sessions have the same hardware acceleration device handle. Setting device handle is optional only if all components in pipeline belong to the same session.


Text Box: mfxExtOpqueSurfaceAlloc osa, *posa=&osa; memset(&osa,0,sizeof(osa)); // query frame surface allocation needs MFXVideoDECODE_QueryIOSurf(session, &decode_param, &request_decode); MFXVideoENCODE_QueryIOSurf(session, &encode_param, &request_encode); // calculate the surface pool surface type and numbers if (MFX_MEMTYPE_BASE(request_decode.Type) == MFX_MEMTYPE_BASE(request_encode.Type)) { osa.Out.NumSurface = request_decode.NumFrameSuggested + request_encode.NumFrameSuggested - decode_param.AsyncDepth; osa.Out.Type=request_decode.Type; } else { // it is also ok to use decode's NumFrameSuggested and Type. osa.Out.NumSurface=request_encode.NumFrameSuggested; osa.Out.Type=request_encode.Type; } // allocate surface pool and zero MemId/Y/U/V/A pointers osa.Out.Surfaces=alloc_mfxFrameSurface1(osa.Out.NumSurface); // attach the surface pool during decode & encode initialization osa.Header.BufferId=MFX_EXTBUFF_OPAQUE_SURFACE_ALLOCATION; osa.Header.BufferSz=sizeof(osa); decode_param.NumExtParam=1; decode_param.ExtParam=&posa; MFXVideoDECODE_Init(session, &decode_param); memcpy(&osa.In, &osa.Out, sizeof(osa.Out)); encode_param.NumExtParam=1; encode_param.ExtParam=&posa; MFXVideoENCODE_Init(session, &encode_param);

Example 12: Pseudo-Code of Opaque Surface Procedure


For more complete information about compiler optimizations, see our Optimization Notice.