video conferencing Mpeg4 encoder/decoder

video conferencing Mpeg4 encoder/decoder

Hi,I want to use Mpeg4 encoder to encodecapture image comes from a video capture card and transmit compressed images by network. I work with aintel P4 and windows 2000.Is it possible to encode and decode Mpeg4 stream wihtout using avi file ? I want to use this codec for video conferencing application.Best regards

26 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.
vladimir-dudnik (Intel)'s picture

Hello,

sure, you not necessary need AVI container to decode MPEG4 from or encode to. If you will take a look on IPP media sample, you can find video_enc_con application which demonstrates how to encode YUV data to raw video bitstream. What you need is to write some additional code to wrap YUV frames you captured from card to input of MPEG4 (or whatever) video encoder.

Regards,
Vladimir

Thank you for your help,

Is it possible to use a decoder without using a splitter. So, I encode in MPEG4 a video cameradata and I want to transmit the encoded images through the network. How can I decode the raw video bitstream without using a raw video bitstream file ? Is it possible ?

Best regardes.

Johan.

vladimir-dudnik (Intel)'s picture

Hi Johan,

yes, it is possible to decode pure video stream without using splitter. In fact, what does splitter do - it extracts pure video stream from program or transport stream. The only additional service it provides, it split stream to separate frames.

Regards,
Vladimir

Ok,

So, I have modified the "video_enc_con" program to encodein MPEG4 avideo sequence.Each encoded frameis stored in a raw video bitstream filewith its size (Sizeframe1+ frame1+Sizeframe2+ frame2 etc....). I have modified "video_enc_con" to decode the previous raw video bitstream file. I initialize a MPEG4 decoder,I split raw video bitstream file andI call Getframe() method.

I post code source in a attach file.

Best regards.

vladimir-dudnik (Intel)'s picture

Thanks for sharing sources, but what I actually mean under pure video stream was just ++...+. UMC MPEG4 decoder should be able to decode this. Simplest way is when whole stream are located in memory and pointed by MediaData object. More complicated organization is requiredwhen streams you are working on are big in size (or just live video). In this case you need tomap at least one compressed video framein MediaDatabufferand keep it filled after each call of GetFrame until end of stream.

Vladimir

Message Edited by vdudnik on 06-01-2006 06:06 PM

Hi vdudnik,

Excuse me I have forgotten to ask my question in my previous post. When I use the source code, I attach to this post, the function Getframe() return "can't find sync word in buffer" => status -897. I thinks I do operation at the same way you describe me.

Best regards.

Johan

Hi Vladimir,

I've some problem with decoding video frames.
I convert picture obtained from capture card from RGB to YUV420 format and code by h.264 codec. After this, I send the data on the network obtained from the codec. On another computer I try to decode obtained data, bug I get UMC_NOT_ENOUGH_DATA responce from GetFrame function.

void DisplayRemoteFrame(unsigned char *data,int size)
{
Status umcRes;
Ipp8u *pY = NULL, *pU = NULL, *pV = NULL, *pBS = NULL;
MediaData in;
VideoData out;

int mSize = IMAGE_WIDTH * IMAGE_HEIGHT;
pY = pU = pV = pBS = NULL;
pY = new Ipp8u[mSize];
pU = new Ipp8u[mSize];
pV = new Ipp8u[mSize];
pBS = new Ipp8u[2*mSize];

memcpy(cdata,data,size);

out.Init(IMAGE_WIDTH, IMAGE_HEIGHT,YUV420,NULL);
out.SetDest(pY,pU,pV);
out.SetPitch(IMAGE_WIDTH, IMAGE_WIDTH/2, IMAGE_WIDTH/2);
out.SetColorFormat(YUV420);
out.SetDataSize(IMAGE_HEIGHT*IMAGE_WIDTH + (IMAGE_HEIGHT/2)*(IMAGE_WIDTH/2) + (IMAGE_HEIGHT/2)*(IMAGE_WIDTH/2));

in.SetBufferPointer(cdata, size);
in.SetDataSize(size);

if(!m_bInitDecoder)
{
InitH264Decoder(&in);
}

in.SetBufferPointer(cdata, size);
in.SetDataSize(size);

umcRes = m_pH264Decoder->GetFrame(&in,&out);
}

Decoder was init with following code:

VideoStreamInfo video_info;
video_info.color_format = UMC::YUV420;
video_info.stream_type = UMC::H264_VIDEO;
video_info.stream_subtype = UMC::UNDEF_VIDEO_SUBTYPE;
video_info.clip_info.width = IMAGE_WIDTH;
video_info.clip_info.height = IMAGE_HEIGHT;

UMC::ColorConversionInfo ColorInit;
ColorInit.SizeSource.width = video_info.clip_info.width;
ColorInit.SizeSource.height = video_info.clip_info.height;
vm_var32 uiFactor = 1;
ColorInit.FormatDest = UMC::YUV420;
ColorInit.SizeDest.width = video_info.clip_info.width;
ColorInit.SizeDest.height = video_info.clip_info.height;
ColorInit.lFlags = 1;
ColorInit.lDeinterlace = 0;
ColorInit.lInterpolation = UMC::FLAG_CCNV_NONE;

pParams->info = video_info;
pParams->cformat = UMC::YUV420;
pParams->lFlags = 0;
pParams->lpConverter = ColorDecoder;
pParams->lpConvertInit = &ColorInit
pParams->uiLimitThreads = 1;

What data it is missing for the decoding? Help me please!!!

Best regards,
Alexander

vladimir-dudnik (Intel)'s picture

When you try to decode, did you get all encoded frame?

Vladimir

hi,Vladimir!

Wish you can see this discuss agian.cause i have some question to decode mpeg4 with ipp,i hope you can help me.I download v5.1 for Windows* on Intel IXP4XX & Simple Media Player and complied the mpeg4 decode library successfuly.My target platform is wince5.0 cpu is pxa250/255/270.I read the related discuss which tell me this version can work.Instead of playing avi files,my target is recieving video stream from network and display it.

Similar to what Alexander did,iinitilized my MPEG-4 decoder as following code:

#define DEST_C_FORMAT UMC::RGB24

void InitilizeMpeg4Decoder(char *pIFrameData, int nFrameSize)

{

VideoStreamInfo video_info;
m_video_info.color_format = UMC::YUV420;
m_video_info.stream_type = UMC::MPEG4;
m_video_info.stream_subtype = UMC::UNDEF_VIDEO_SUBTYPE;
m_video_info.clip_info.width = 0;
m_video_info.clip_info.height = 0;

UMC::ColorConversionInfo ColorInit;
ColorInit.SizeSource.width = video_info.clip_info.width;
ColorInit.SizeSource.height = video_info.clip_info.height;
ColorInit.FormatDest = DEST_C_FORMAT;
ColorInit.SizeDest.width = video_info.clip_info.width;
ColorInit.SizeDest.height = video_info.clip_info.height;
ColorInit.lFlags = 1;
ColorInit.lDeinterlace = 0;
ColorInit.lInterpolation = UMC::FLAG_CCNV_NONE;

m_pInMediaData->SetBufferPointer((vm_byte*)pIFrameData,nFrameSize );

m_pVideoDecoderParams->info = video_info;
m_pVideoDecoderParams->cformat = DEST_C_FORMAT;
m_pVideoDecoderParams->lFlags = 0;
m_pVideoDecoderParams->lpConverter = ColorDecoder;
m_pVideoDecoderParams->lpConvertInit = &ColorInit
m_pVideoDecoderParams->uiLimitThreads = 1;
m_pVideoDecoderParams->m_pData = m_pInMediaData;

UMC::Status status = m_Mpeg4Decoder.Init(m_pVideoDecoderParams);
if (UMC_OK == status)
{
m_videoWidth = m_Param.info.clip_info.width;
m_videoHeight = m_Param.info.clip_info.height;
m_pOutMediaData->SetVideoParameters(m_videoWidth, m_videoHeight, DEST_C_FORMAT);
}

}

After recieved the first I frame video stream,i initilized my decoder by call InitilizeMpeg4Decoder function,it seems that this funciont works well,because i got the right video width and height,but by call decode function i can's get right decoded data:

m_Mpeg4Decoder.GetFrame(m_pInMediaData, m_pOutMediaData);

It seems that the decoded data in m_pOutMediaData is not right,i render these data but obviously not right.I am sure that the mpeg4 data is correct,but strangly,if i decode mpeg4 stream into UMC::RGB32(#define DEST_C_FORMAT UMC::RGB32),all of the decoded data in m_pOutMediaDatais 0,if ((#define DEST_C_FORMAT UMC::RGB565),the decoded data is wrong too(not all of the data is 0).I have read the discussio
ng "Using MPEG-4 Decoder",and i notice that you decode mpeg4 stream into UMC::YV12 format.Is this differenceleads tomy failure to decode?And could you tell me what mistake i made?

vladimir-dudnik (Intel)'s picture

I think the reason is that UMC MPEG4 decoder (and some other UMC video decoders) do not output first decoded frame immediately (in first call of GetFrame). It accumulates frames internally for usingfor prediction subsequent frames. You can see how loop over frames made in simple player.

Regards,
Vladimir

Vladimir:
Thanks for your reply!But i don't think the MPEG4 decoder didn't output first decoded frame immediately since i did a test to write the first decoded data into a file when i got the first I-Frame.It seems that the decoded data is just not right.As mentioned above,i got difference results.when i decode the first I-Frame into different format(into rgb24 and rgb565 i got wrong data,into rgb32 i got empty data).I have validated my video render funcition,it works well.As long as the decoder get one decodedI-Frame then it can produce the fllowingP-Frames,isn't it?
Maybe i should tell more details about what i have done.By now,i have compiled three versions of "Simple Media Player" with different wince sdk(wince4.0 standard sdk,wince5.0 standard sdk and Windows Mobile 5.0 Pocket PC SDK) use Intel IPP for IXP4XX:
For "wince4.0 standard sdk" i can't compile "vm_plus" library successfully cause the "winbase.h" doesnot declare the "GetIdleTime" function.And the "simple_player" can't compile successfully too (it need "aygshell.lib" but wince4.0 sdk doesnot contain this library).
For "wince5.0 standard sdk" and "Windows Mobile 5.0 Pocket PC SDK" i can't compile "simple_player" successfully,i got "error LNK2005: sin already defined in x0__ar10.lib(xsc_sin.obj)" I have search this error in the forum and found your reply,so it is to say that because of the confliction of x0__ar10.lib and coredll.lib we can't complie the "simple_player" in this condition,isn't it?
By the way,i modified some "Makefile.win" files(get ride of the /YX) in "Simple Media Player" in order to complie successfully.It doesn't matter,isn't it?
There is another strange thing:If i use the "mpeg4_dec" library in the real mainboard(cpu is pxa255 or pxa270,os is wince4.2 or wince5.0),the mpeg4 decoder seems works,but produce wrong datas(as motioned above).While using "mpeg4_dec" in emulator environment(that is running my program in "Microsoft Visual Studio 2005"),the program crashed when GetFrame():
Do you still remember the discussion "Using MPEG-4 Decoder"?You posted a sample named "umc_mpeg4_dec",i don't know whether the sample can works with IXP4XX.I tried to complie this sample with "Microsoft Visual Studio 2005",first i create a new "MFC Smart Device Application" project,then add the file "umc_mpeg4_dec.cpp" to the project,i deleted the main() function and create new function as substitute:

void CEPlayerDlg::OnBnClickedButton2()
{
ippStaticInit();
copyright();
ipp_version();
DecodeMPEG4Video(L"Storage CardVideo20060107_050829.avi");
}

When i debug this program(run the debug version in Windows Mobile 5.0 Pocket PC Emulator),it crashed when decode frame:
UMC::Status DecodeMPEG4Video(vm_char* input_file)
{
....
t0 = vm_time_get_tick();

umcRes = mpeg4_dec.GetFrame(&in, &out); <---- crashed
if (umcRes != UMC::UMC_OK && umcRes != UMC::UMC_NOT_ENOUGH_DATA)
break;
...
}

Before GetFrame the data "UMC::MediaData in" seems good,but "UMC::VideoData out" seems strange,Since we are ready to decode the video frame into UMC::YV12 format,why the out.m_lpDest[1] and out.m_lpDest[2] is 0,out.m_lPitch[1]
and out.m_lPitch[2] is 0 too?I try to decode video stream into UMC::RGB32 format,crash in the same place!
The last question:I notice that the MPEG4VideoDecoder::Init() function set the undecoded video frame format to YV12 as default:
m_Convert.ConversionInit.FormatSource = YV12;
i don't understand why?

Maybe you are very busy recently,sorry to disturb you.

Hi Vladimir:

I have seen the simple player many times,and i noticed that there was a repeat decode procedure when try to decode video frame.If the decoder accumulates frames internally,it will return UMC_NOT_ENOUGHT_DATA sometimes when call GetFrame() funtionand return UMC_OK when it get enough data to decode one frame,isn't it?So,if the GetFrame() funtion return UMC_OK,i can deal with the decoded frames?But when i call GetFrame(),it never return UMC_NOT_ENOUGHT_DATA,every time i got UMC_OK,but i can't get decoded video frame in the output.What's wrong?

Thank you.

vladimir-dudnik (Intel)'s picture

Hi,

you might be interested to see modification of simple MJPEG decoder sample (to make it simple MPEG4 decoder), described in thread Using MPEG4 decoder. As it is much simplier than simple_player (well, should we rename it to complex_player? :) ) you might find it useful for understanding how UMC components work.

Regards,
Vladimir

Hi Vladimir:

Thanks for your advise.I have download the "umc_mpeg4_dec sample " several days ago.And it runs well at windows 2000/xp(it output an "output.yuv" file),but this sample decode video frames into YV12 format.Can i modify this sample to make it to decode video frames into RGB32 format?I tried many times,howerver i can't got right decoded data.Every time i call GetFrame() function, it return UMC_OK,but the output is empty.

In fact,my target platform is wince5.0(pxa255/270).I have been complied the mpeg4 decoder successfully with Intel IPP v5.1 for Windows* on Intel? IXP4XX and Standard wince5.0 sdk.I recive video stream from the network and try to decompress it.

Here is my initialize function:

void InitilizeMpeg4Decoder(char *pIFrameData, int nFrameSize)

{

m_video_info.color_format = UMC::YUV420;
m_video_info.stream_type = UMC::MPEG4_VIDEO;
m_video_info.stream_subtype = UMC::UNDEF_VIDEO_SUBTYPE;
m_video_info.clip_info.width = 0;
m_video_info.clip_info.height = 0;

UMC::ColorConversionInfo ColorInit;
ColorInit.FormatSource = UMC::UMC::YUV420;
ColorInit.SizeSource.width = video_info.clip_info.width;
ColorInit.SizeSource.height = video_info.clip_info.height;
ColorInit.FormatDest = UMC::UMC_RGB32;
ColorInit.SizeDest.width = video_info.clip_info.width;
ColorInit.SizeDest.height = video_info.clip_info.height;
ColorInit.lFlags = 1;
ColorInit.lDeinterlace = 0;
ColorInit.lInterpolation = UMC::FLAG_CCNV_CONVERT;

m_pInMediaData->SetBufferPointer((vm_byte*)pIFrameData,nFrameSize );

m_pVideoDecoderParams->info = m_video_info;
m_pVideoDecoderParams->cformat = UMC::UMC_RGB32;
m_pVideoDecoderParams->lFlags = UMC::FLAG_VDEC_UVMERGED | UMC::FLAG_VDEC_REORDER;
m_pVideoDecoderParams->lpConverter = m_ColorDecoder;
m_pVideoDecoderParams->lpConvertInit = &ColorInit
m_pVideoDecoderParams->uiLimitThreads = 1;
m_pVideoDecoderParams->m_pData = m_pInMediaData;

UMC::Status status = m_Mpeg4Decoder.Init(m_pVideoDecoderParams);
if (UMC::UMC_OK == status)
{
m_videoWidth = m_Param.info.clip_info.width;
m_videoHeight = m_Param.info.clip_info.height;
m_bInitialized = true;
}
}

And here is my decoder funtion:

int DecodeVideoFrame(char *pVideoData, unsigned int nDataLen, bool bIFrame);

{

if(!m_bInitialized)
{
if(bIFrame) InitilizeMpeg4Decoder(pVideoData, nDataLen);
else return -4;
}
m_pInMediaData->SetBufferPointer((vm_byte*)pVideoData,nDataLen);
m_pOutVideoData->SetVideoParameters(m_videoWidth, m_videoHeight, UMC::UMC_RGB32);
m_nOutLen = m_pOutVideoData->m_lPitch[0]*m_videoHeight;
m_pOutVideoData->SetBufferPointer((vm_byte*)m_pOutBuf,m_nOutLen);
umcRes = m_Mpeg4Decoder.GetFrame(m_pInMediaData, m_pOutVideoData);
if(umcRes != UMC::UMC_OK && umc
Res != UMC::UMC_NOT_ENOUGH_DATA) return -2;
if(umcRes == UMC::UMC_OK) return 0;
if(umcRes == UMC::UMC_NOT_ENOUGH_DATA) return -1;
}

Well,i call DecodeVideoFrame() function in the reciving video stream thread,it call InitilizeMpeg4Decoder() function when it get the first I-Frame and i can get the video width and height,it's all right.But the GetFrame() function never return UMC_NOT_ENOUGH_DATA,it always return UMC_OK,but when i check the output data in m_pOutVideoData,there is nothing.

Thanks for your patiance,could you tell me what's wrong with these codes?

vladimir-dudnik (Intel)'s picture

Hello,

our experts found a mistake in VideoData.cpp file, which affect decoding to RGB24 format.

Instead of

enum
{
ALIGN_VALUE = 128
};

should be

enum
{
ALIGN_VALUE = 1
};

Please also notice that the following functions are not implemented in IPP 5.1 sample

ConvertFrom_RGB565_To_RGB24(ColorConversionParams &/*ConvertParam*/)ConvertFrom_RGB32_To_RGB24(ColorConversionParams &/*ConvertParam*/)

You probably can try to decode to RGB24 with corrected VideoData.cpp file.

Regards,
Vladimir

Hi Vladimir:

That's right!!!This morningi foundthis mistake too,but at first i do not think this is a mistake,i consider that there are some special need for pxa255/pxa277 platform to set the ALIGN_VALUE to 128.So i have to make a additional memory copy for the decoded data whichresovledmy problem.Now i can decode the video stream into RGB24 format correctly,but it seems thatRGB32 data is still unreachable,but it doesn't matter,RGB24 is OK.Thanks for your reply!

vladimir-dudnik (Intel)'s picture

Great, I'm glad we finally found at least some solution. Regarding RGB32, as I said this functionality is not implemented in current version of UMC ColorConverter. Hope, you'll find UMC framework useful for your purposes. Please feel free to provide us info about any issues (you can do it her or through Premier Intel Support service).

Regards,
Vladimir

Hello,

Thanks for all your contribution... I have got my umc_doc_sample program (to decode mpeg4) working finally........ But it expects an avi file.... what if i get encoded mpeg4 stream ina file with *.mp4... I tried the same code with no for *.mp4 file....... initializatoin of avisplitter failed with error code 895 ( end of stream)....

FYI, i created this *.mp4 file by converting avi_file using Editgraph ( directx filters).... may be this is not the right way to do..... can some one pass any sample *.mp4 files....

Also, should i replace avisplitter with mp4splitter to make it work with *.mp4 file?

Thanks in advance for your help.

with regards,

+rajpras+

vladimir-dudnik (Intel)'s picture

please look through simple_player, UMC provides several splitters, AVISplitter work with AVI containers, MP4Splitter works with MP4 containers, MPEG2Splitter work with MPEG streams and so on...

Vladimir

Hello Vladimir,

could you please help me out how can i create a playble MPEG4 file.

FYI, i get RTP Header + Rtp payload from Bosch MPEG4 encoder........ Rtp Payload is comprised of series of Acess Units. Below is the format

+---------+-----------+-----------+---------------+
| RTP | AU Header | Auxiliary | Access Unit |
| Header | Section | Section | Data Section |

+---------+-----------+-----------+---------------+

<----------RTP Packet Payload----------->

Here is the link for RFC on MPEG4 format, http://www.networksorcery.com/enp/rfc/rfc3640.txt

I was wonderingjust writing RTPPlayload into a sequestial file makes it playable by other mpeg4 players..... Bosch has some specific player which interprets above mentioned file format..... But i would like to make it more generic file format....which can be decoded by most of the mpeg4 players....

Appreciate if someone can help me what exact the mpeg4 file format would/should be? what elements should be written into *.mp4 file.....

Thanks

+rajpras+

vladimir-dudnik (Intel)'s picture

Hello, you probably can use UMC muxer component to pack payload data from your stream to MP4. I think if you first extract raw video stream from your RTP stream then you can mux it to MP4. Please look at MPEG4 standard for details about MP4 file format

Vladimir

Hi!

Vladimir, you mentioned that we have to change in VideoData.cpp the ALIGN_VALUE from 128 to 1. What are the effects of this parameter? I convert YV12 to RGB24 with my program but everything looks good. Sometimes I have a few badly decoded(or converted) frame, but I thougt it is the mpeg4 camera's fault. Do I have to change that parameter too?

Thanks in advance,

Bendeguy

vladimir-dudnik (Intel)'s picture

Yes, according our experts, it is better to change for IPP 5.1.1 sample. Setting this to 1 you turn off alignment of VideoData's color planes pitches.

Vladimir

hi Raj ,
Raj i have some problem to getting the live stream of mpeg4 from IP Camera ...through rtsp as it supports only it ....

Please can u give that URL by which i can get the live video stream from Axis 214 IP camera.....

and also i have got the stream from Sony IP cam and it is in Mpeg4 format but for Axis camera i cant get it..

i am asking for that url only and aloso the decoding methos to decode ans to show this live stream coming from camera..

Regards
--n@yi

Hi Raj,

We are trying to do a similar thing (i.e. view a Bosch VideoJet 10 stream with a standard MPEG-4 viewer), but with no success.

Just wondering if you had had any success at your end? Send me an email, maybe we can help each other out?

Jono

Login to leave a comment.