Media SDK with IDirect3DSurface9 + h.264 encoder

Lu N.

Di, 15/01/2013 - 19:42

Hi,

I want to do the screen capture(1920*1080) via DirectX function GetFrontBufferData, and use IDirect3DSurface9 with input for h.264 encoder,

but the encoder is without any output video file, here is the my code(modification by smaple code) and the log data by mediasdk_tracer.

Could you help to check which items I need to modify, thanks.

//Run Encoder

while (MFX_ERR_NONE <= sts || MFX_ERR_MORE_DATA == sts )
{
sts = GetFreeTask(&pCurrentTask);
MSDK_BREAK_ON_ERROR(sts);
nEncSurfIdx = GetFreeSurface(m_pEncSurfaces, m_EncResponse.NumFrameActual);
MSDK_CHECK_ERROR(nEncSurfIdx, MSDK_INVALID_SURF_IDX, MFX_ERR_MEMORY_ALLOC);
pSurf = &m_pEncSurfaces[nEncSurfIdx];
if (m_bExternalAlloc)
{

sts = m_pMFXAllocator->Lock(m_pMFXAllocator->pthis, pSurf->Data.MemId, &(pSurf->Data));
}
MSDK_BREAK_ON_ERROR(sts);
pSurf->Info.FrameId.ViewId = currViewNum;

g_pd3dDevice->GetFrontBufferData(0,pSurf->Data.MemId);

if (m_bExternalAlloc)
{
sts = m_pMFXAllocator->Unlock(m_pMFXAllocator->pthis, pSurf->Data.MemId, &(pSurf->Data));
}
MSDK_BREAK_ON_ERROR(sts);
for (;;)
{
sts = m_pmfxENC->EncodeFrameAsync(NULL, &m_pEncSurfaces[nEncSurfIdx], &pCurrentTask->mfxBS, &pCurrentTask->EncSyncP);
if (MFX_ERR_NONE < sts && !pCurrentTask->EncSyncP)
{
if (MFX_WRN_DEVICE_BUSY == sts)
MSDK_SLEEP(1);

}

else if (MFX_ERR_NONE < sts && pCurrentTask->EncSyncP)
{
sts = MFX_ERR_NONE; // ignore warnings if output is available
break;
}
else if (MFX_ERR_NOT_ENOUGH_BUFFER == sts)
{
sts = AllocateSufficientBuffer(&pCurrentTask->mfxBS);
MSDK_CHECK_RESULT(sts, MFX_ERR_NONE, sts);
}
else
{
MSDK_IGNORE_MFX_STS(sts, MFX_ERR_MORE_BITSTREAM);
break;
}
}
}

Anhang	Größe
Herunterladen hw-encode-0115-1.txt	4.05 MB

8 Beiträge / 0 neu

Letzter Beitrag

Nähere Informationen zur Compiler-Optimierung finden Sie in unserem Optimierungshinweis.

Zum Verfassen von Kommentaren bitte anmelden.

Petter Larsson (Intel)

Mi, 16/01/2013 - 14:09

Hi Lu,

GetFrontBufferData captures the front buffer in native format which is: D3DFMT_A8R8G8B8
http://msdn.microsoft.com/en-us/library/windows/desktop/bb174388(v=vs.85).aspx

But Media SDK EncodeFrameAsync can only process surface types of type NV12.So from the code it appears that you are trying to have the GetFrontBufferData call do an implicit color conversion from RGB32 type surface to an NV12. This is very likely not supported. If you check the return value from GetFrontBufferData you will probably see that it returns an error.

So, to achieve what you want you will have to do color conversion before feeding to EncodeFrameAsync. RGB32 to NV12 converison can be done using Media SDK VPP component or via other custom color converison routines you may have.

Media SDK sample_encode showcases how to use VPP ahead of encode,

Regards,
Petter

Zum Verfassen von Kommentaren bitte anmelden.

Lu N.

Do, 17/01/2013 - 01:14

Hi Petter,

Thansk for your help.

So we may use VPP to do the surface color conversion(from RGB32 to NV12) , and use it as the input of EncodeFrameAsync,right?

As below:

IDirect3DSurface9* g_pSurface;

Surface_Lock();

g_pd3dDevice->GetFrontBufferData(0, g_pSurface); //RGB32 format

Vpp_color_convert(g_pSurface, NV12_Surface); //Vpp color convert function

EncodeFrameAsync(NV12_Surface); //Encode

Surface_UnLock();

But I have checked the sample_VPP, it seems not have the surface color conversion, could you give some suggstion or hint for the surface color conversion, thanks.

Zum Verfassen von Kommentaren bitte anmelden.

Petter Larsson (Intel)

Do, 17/01/2013 - 10:05

Hi Lu,

Correct, for RGB32 to NV12 color space conversion you can use VPP.

sample_vpp does showcase RGB32 to NV12 color converson.

Also sample_encode sample has functionality for VPP processing before encode. But the sample code only illustrates how to use it for resize. But it could easily be extended to support RGB32.

Regards,
Petter

Zum Verfassen von Kommentaren bitte anmelden.

Lu N.

Do, 17/01/2013 - 18:32

Hi Petter,

Thanks.

I see the partial code for the read RGB4 file to memory, as below:

Due to I don't see any others ,does it do the conversion from file(RGB4) to video memory(NV12)?

if (pInfo->FourCC == MFX_FOURCC_RGB4)
{
ptr = MSDK_MIN( MSDK_MIN(pData->R, pData->G), pData->B );

ptr = ptr + pInfo->CropX + pInfo->CropY * pitch;

for(i = 0; i < h; i++)
{
nBytesRead = (mfxU32)fread(ptr + i * pitch, 1, 4*w, m_fSrc); //File reader
}

}

Zum Verfassen von Kommentaren bitte anmelden.

Petter Larsson (Intel)

Fr, 18/01/2013 - 15:57

Hi Lu,

RGB4 to NV12 color space conversion happen inside Media SDK VPP component.

The code section you refer to is just the file reader that reads a RGB4 surface from file into D3D or system memory buffer. The next step performed by the sample code is to feed the RGB4 surface to VPP for processing into NV12 surface.

By the way. Some things to keep in mind when using "GetFrontBufferData" in this context:
- Not only does GetFrontBufferData require output surface to be of type D3DFMT_A8R8G8B8. The surface must also be stored in system memory (D3DPOOL_SYSTEMMEM)
- This forces you to store the surface retrieved from "GetFrontBufferData" in a intermediate surface. Such surface can be created using "CreateOffscreenPlainSurface".
- You can then copy the intermediate surface into the Media SDK RGB4 surface using "UpdateSurface".

Regards,
Petter

Zum Verfassen von Kommentaren bitte anmelden.

Lu N.

So, 20/01/2013 - 19:56

Hi Petter,

Thanks a lot, know how to do the color coversion.

And I have used "CreateOffscreenPlainSurface" to store the surface retrieved from "GetFrontBufferData".

But when I use the "UpdateSurface" for GetFrontBufferData-> VPP RGB4 surface, it will not succeed:

//************************************************************************************

Lock(VPP_surface);

GetFrontBufferData(0, Getfrontdata_surface);

UpdateSurface(Getfrontdata_surface, NULL, (IDirect3DSurface9*)VPP_surface->Data.MemId, NULL); // not succeed

Unlock(VPP_surface);

RunFrameVPPAsync(&VPP_surface, &Encode_surface,NULL, &VppSyncPoint); // Do VPP color conversion

EncodeFrameAsync(NULL, &Encode_surface, &pCurrentTask->mfxBS, &pCurrentTask->EncSyncP); // Encode

//************************************************************************************

After check the Microsoft MSDN for UpdateSurface , this function has the some restriction, ex:

-The destination surface must have been created with D3DPOOL_DEFAULT => It seems the VPP_surface not created with D3DPOOL_DEFAULT, does any other need to be modified, thanks.

Zum Verfassen von Kommentaren bitte anmelden.

Petter Larsson (Intel)

Mo, 21/01/2013 - 12:35

Hi Lu,

"UpdateSurface" will likely fail since the surface is locked by the "Lock" call. For DirectX calls such as this you do not need to call the Media SDK locking operation.

Media SDK D3D surfaces are created using D3DPOOL_DEFAULT.

Also, I suggest you use DirectX control panel to enable DirectX debugging information. By enabling DirectX debug and singlestepping though your code you can get much more details on what and why specific DirectX calls fail.

Regards,
Petter

Zurück zum ursprünglichen Beitrag

Intel® Developer Zone

Media SDK with IDirect3DSurface9 + h.264 encoder

Media SDK with IDirect3DSurface9 + h.264 encoder

Melden Sie sich an, um einen Kommentar zu hinterlassen.