Key Frame (I/IDR) insertion

Key Frame (I/IDR) insertion

Hello,

I'm trying to implement key frame insertion into my video encoding process. However, I'm facing issues with it.

When calling function MFXVideoENCODE_EncodeFrameAsync (&gop_ctrl, ...), I provide on requested key-frame in structure gop_ctrl.FrameType = FRAME_I | FRAME_IDR | FRAME_REF.

However, I'm not getting I-frame on the expected position ! (Or not at all).
(Please note I'm using code similar to sample_encode or sample_videoconf)

Other parameters in mfxInfoMFX structure which I'm using, are:

VBR encoding, NumSlice=4, NumRefFrame=0 (or 1), EncodedOrder=0, GopPicSize=24, IdrInterval=0, GopOptFlag=0, GopRefDist=1 or 4

I observed the following:

1. when using SW library, no extra key frame is inserted - never !!!
2. when using HW library, and GopRefDist=1 (i.e. no B-frames), then Key Frame is inserted in expected position
3. when using HW library, and GopRefDist=4, the Key Frame is inserted but on wrong position (shifted few frames earlier)

Could you please advice what else should I check? Which other parameters may influence key frame insertion process?

Or is there something wrong with Key Frame Insetion (known bug in Intel Media SDK R2)?
It looks to me like KeyFrame Insertion works only for P frames and only in HW library. That would be a pity!

Thanks

Frank

8 post / 0 nuovi
Ultimo contenuto
Per informazioni complete sulle ottimizzazioni del compilatore, consultare l'Avviso sull'ottimizzazione

Hi Frank,

I believe the Media SDK implementation may be optimizing your GOP structure based on what it feels is best, and to force the insertion you will need to include MFX_GOP_STRICT in your code.

Please see the "Forced Key Frames" section of this doc:  http://software.intel.com/en-us/articles/video-conferencing-features-of-intel-media-software-development-kit/

-Tony

Hello,

unfortunately setting MFX_GOP_STRICT has no impact on key frame insertion.
I performed the above mentioned test with setting MFX_GOP_STRICT but result is the same:

1. when using SW library, no extra key frame is inserted - not at all !
2. when using HW library, then Key Frame is inserted on expected position only when GopRefDist=1,

This still leads me to think there is some bug/gap in the core SDK routines... :(
(unless I need to do something else than only setting MFXVideoENCODE_EncodeFrameAsync (&gop_ctrl, ...) on proper frame)

Remark:
GopRefDist=1 means no B-frames = bad efficiency of video compression.

Btw. the article you have mentioned also uses only mfxInfoMFX::GopRefDist = 1  (for "optimal low latency").

This is valid for videoconferencing, but practically useless for effective HD video encoding (which is my aim, btw.).

"Optimization" of GOP structure is welcome for general HD video, that's right.
But if I want to insert a key frame, I want to do it on the exact position (scene cut), in such case I expect encoder to follow my instruction :)
Otherwise it makes no sense to define manually keyframes.

Frank

 

Hi Frank,

Using low latency configuration, like the encoder config you are using, immediate I-frame insertion is possible for both SW and HW mode. Just to make sure I just tried I-frame insertion locally for both HW and SW mode and it works fine.

Not sure why you're encountering issues with SW mode.  Could you please share your exact encoder configuration (you can capture this using the media SDK tracer tool) and also the version of Media SDK SW DLL (from 2013 R2?) you are using.

The behavior you describe for the case when GopRefDist > 1  is expected. Please see the chapter in the Media SDK reference manual which describes "Forced Key Frame Generation".

Regards,
Petter

Hi Petter,

thanks for your response.

I'm using libraries:
HW: file version 4.13.11.22, product version 4.0.1533432.75720
SW: file version 4.13.6.21,  product version 4.0.0000760.60139  (from SDK 2013 R2)

I run and encoded four times the same video (275 frames), just changing HW vs. SW library and GopRefDist = 1 vs. 4 (hence 4 combinations),
all other parameters were identical.

I requested frames 9, 40 and 55 to be keyframes (IDR frames, counting from 0)

Resulting gop-structure is displayed in file *_gop.log, and associated trace analysis in *_analysis.log
As you can see, only HW library and GopRefDist=1 fulfilled the request.

SW library ignored forced keyframes completely.

HW library with GopRefDist=4 placed them to different positions (38 and 53), which is wrong.
Key frame must be placed to exactly requested position (scene cut), this is the reason and purpose why do we want to force keyframes :)
(And obviously, for HD video encoding B-frames are a necessity for achieving high quality with reasonable bitrate)

Any advice is welcome.

Thanks (and Happy New Year 2014 - additionally :) )

Frank

Allegati: 

AllegatoDimensione
Download gop.zip23.12 KB

Hi Frank,

not sure what is going on with regards to your SW execution case. Locally, using the exact same configuration and SW DLL (from 2013 R2), I frames are inserted at the expected locations.  Can you please revisit your implementation to make sure Iframe/IDR/REF are inserted correctly in your code for the SW execution path.

Regarding the case when you're using B-frames. The behavior you observe is the expected one as stated in my earlier post.

You may be able to achieve your goal  by moving to "EncodedOrder" mode, which requires you to manually specify the exact frame type for each encoded frame.

Regards,
Petter

Hi Petter

The actual encoding pipeline is identical both for HW and SW, basically it is practically identical code to "sample_encode" example
- modified for insertion of I/IDR frames as mentioned in the first post above.
Selection of HW/SW is done in the initializaton - all the rest of the code is identical.
So it is strange for me why SW library ignores it.

Btw. I noticed that SW and HW encoding for the same yuv-source and same parameters produce different .h264 streams, so internal encoding logic in HW and SW is apparently slightly different.
(HW is much faster but SW gives slightly better picture - I noticed in HW some macroblocking artefacts in case of too long GOP)

Regarding B-frames: It might be expected behavior, but honestly - it is not logical and useless for practical video-encoding :(
If I want to insert keyframe (=I/IDR-frame), I want to do it intentionally on some sharp scene cut to avoid unwanted artefacts (macroblocing).
This is the reason for forced key-frames.
And in such case encoder must place it on exactly this frame and adjust GOP structure of other surrounding frames accordingly.
Otherwise I would leave it completely on encoder and not use forcing key frames.

I'd suggest to Intel Media Development Team to reconsider it and change this behavior to "natural", logical way...
 

Thanks

Frank

 

Hi Frank,

SW and HW encoder are implemented in very different ways, and thus the resulting encoded stream does have different properties. This is the expected behavior.

We have no plans of changing the behavior for the case of manual insertion I-frames.  Please explore using "EncodedOrder" mode instead.

Regards,
Petter

Lascia un commento

Eseguire l'accesso per aggiungere un commento. Non siete membri? Iscriviti oggi