As we mentioned before, the main plugin initializations, OpenVX graph construction and verification are performed during the very first Execute method invocation. The frame format retrieval is the very first task resolved here. The current implementation operates only with NV12 data format for simplicity. But from the other hand it unveils multi planar data sharing and OpenVX processing caveats which may be important for the real life scenarios. NV12 data layout is a standard in various media processing tasks in contrast to single planar RGB or RGBX pixel formats. RGB and RGBX processing in OpenVX may look simpler but has less practical usage for DSS.

Here are code snippets illustrating Intel® Media Server Studio surface mapping and OpenVX input image creation and initialization. We are skipping functionality validation code (check API return values) for simplicity in these code snippets.

mfxFrameSurface1 *frame_in;   //Input Media Server Studio (MSS) surface.
mfxFrameAllocator* pAlloc = &m_mfxCore.FrameAllocator();   
//Lock Media Server Studio (MSS) input frame.  
pAlloc->Lock(pAlloc->pthis, frame_in->Data.MemId, &frame_in->Data);    

//Lock MSS output frame.  
mfxFrameSurface1 *frame_out;   //Output MSS surface.
mfxFrameAllocator* pAlloc = &m_mfxCore.FrameAllocator();    
pAlloc->Lock(pAlloc->pthis, frame_out->Data.MemId, &frame_out->Data);  

//Get data size, pitches and pointers.
mfxU32 in_pitch = frame_in->Data.Pitch; 
mfxU32 w = frame_in->Info.Width; 
mfxU32 h = frame_in->Info.Height;

//Use locked MSS surface pointers. No copy.
//Two pointers for NV12: Y plane and UV plane.
void *in_ptrs[] = { frame_in->Data.Y, frame_in->Data.UV };
void *out_ptrs[] = { frame_out->Data.Y, frame_out->Data.UV };

//Fill OpenVX image addressing structure and create OpenVX image using MSS data //pointers and frame format.
vx_imagepatch_addressing_t imagepatchNV12[2]; //2 for two planes. Y and UV
//For Y plane.
imagepatchNV12 [0].dim_x = w; //Use input frame width.
imagepatchNV12 [0].dim_y = h; //Use input frame height.
imagepatchNV12 [0].stride_y = in_pitch; //Use MSS horizontal data pitch.
//For UV plane.
imagepatchNV12 [1].dim_x = w/2; //Use input frame width divided by 2 (subsampled).
imagepatchNV12 [1].dim_y = h/2; //Use input frame height divided by 2 (subsampled).
imagepatchNV12 [1].stride_y = in_pitch; //Use MSS horizontal data pitch.
... //fill the rest of the fields.

//Create OpenVX image using addressing structures imagepatchNV12  
//and data pointers in_ptrs received from MSS.
vx_image inputNV12_image = vxCreateImageFromHandle (ovx_context, VX_DF_IMAGE_NV12, imagepatchNV12, in_ptrs, VX_MEMORY_TYPE_HOST);

Now we can create the rest of the OpenVX images

//Output NV12 image.  
vx_image outputNV12_image = vxCreateImageFromHandle (ovx_context, VX_DF_IMAGE_NV12, imagepatchNV12, out_ptrs, VX_MEMORY_TYPE_HOST);//Intermediate images for separate Y, U //and V channels.
vx_image y_image = vxCreateVirtualImage(ovx_graph_handle, w, h, VX_DF_IMAGE_U8);
vx_image u_image = vxCreateVirtualImage(ovx_graph_handle, w/2, h/2, VX_DF_IMAGE_U8);
vx_image v_image = vxCreateVirtualImage(ovx_graph_handle, w/2, h/2, VX_DF_IMAGE_U8);
//Canny edge detection result for Y channel image.
vx_image canny_image = vxCreateVirtualImage(ovx_graph_handle, w, h, VX_DF_IMAGE_U8);

and threshold data object for Canny edge detector. Initialize gradient size and normalization type as well.

vx_threshold thr = vxCreateThreshold(ovx_context, VX_THRESHOLD_TYPE_RANGE, VX_TYPE_UINT8);
vx_int32 hystLo = 80, hystUp = 150;
vxSetThresholdAttribute(thr, VX_THRESHOLD_THRESHOLD_LOWER, &hystLo, sizeof(hystLo));
vxSetThresholdAttribute(thr, VX_THRESHOLD_THRESHOLD_UPPER, &hystUp, sizeof(hystUp));

vx_int32 gradient_size = 3;
vx_enum norm_type = VX_NORM_L1;

Finally, construct and verify OpenVX graph.

//Construct graph
node1=vxChannelExtractNode(ovx_graph_handle, inputNV12_image, VX_CHANNEL_Y, y_image);
node2=vxChannelExtractNode(ovx_graph_handle, inputNV12_image, VX_CHANNEL_U, u_image);
node3=vxChannelExtractNode(ovx_graph_handle, inputNV12_image, VX_CHANNEL_V, v_image);
node4=vxCannyEdgeDetectorNode(ovx_graph_handle, y_image, hyst, gradient_size, norm_type, canny_image);
node5=vxChannelCombineNode(ovx_graph_handle, canny_image, u_image, v_image, 0, outputNV12_image);

//Validating the Graph

Now we are done with OpenVX data structures initialization and graph construction. So, we can unlock previously locked Intel® Media Server Studio resource, returning ownership for the surfaces back to the Intel ® Media Server Studio.

//Unlock MSS input frame surfaces
pAlloc->Unlock(pAlloc->pthis, frame_in->Data.MemId, &frame_in->Data);
pAlloc->Unlock(pAlloc->pthis, frame_out->Data.MemId, &frame_out->Data);
Для получения подробной информации о возможностях оптимизации компилятора обратитесь к нашему Уведомлению об оптимизации.