Very high latency in the decoder side

Very high latency in the decoder side

I've been building a video conferencing application based on the encoder & decoder in the SDK (namely, H.264 codec, using 3.0 beta 2).
The encoder works wonders, with extremely low latency, However, the decoder appears to have extremely high latency ! I know it is the decoder (decoding to system memory, by the way), because when I connect another decoder to the encoder's output, I receive almost no latency (however, CPU utilization suffers).

I've gone over all the tricks listed in the manual and the videoconf samples, including setting AsyncDepth to 1 (on both encoder & decoder), setting the MaxDecFrameBuffering to 1, GopRefDist to 1...

When encoding 640x480 30fps, I see a latency of about 1/2 second. What I've noticed was that when I call
Status=MFXVideoDECODE_QueryIOSurf (m_EncodingSession,&m_DecodedVideoParameters,&AllocRequest);
on the decoder, I get a frame-count of 17, which almost exactly matches the perceived latency.
The only paramter I've changed which has any effect over it is the codec level, but this is tied to the resolution passed and framrate, which I can not change. I hardly understand why these parameters affect the latency so.

What I'm askin in short is : what am I missing here, in shortening the latency ? I doubt this latency is inherent to the decoder, which is otherwise very good !


Jack Chimasera

6 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.

Hi Jack,
You are doing everything right, but unfortunately low latency mode of software H.294 decoder doesn't work properly as of MSDK 3.0 Beta2 version. Also, since we've discovered this limitation too late, the Beta3 version which is to be available pretty soon will suffer from the same defect.
So here is a workaround we suggest to use with Beta2 and Beta3: in addition to everything you did before set NumThread=1. Please note, that the MSDK manual lists this parameter as deprecated, so this is only a temporary solution we offer to let the customers go on with testing of VC features. You will need to drop this setting when MSDK gets fixed.
MSDK 3.0 Beta4 release will function without this hack, totally complying with the manual with regards to low latency configuration.
Best regards,Nina

Hello Nina
Setting ThreadNum=1 worked perfectly when it concerns the software decoder, lowering the latency to barely perceptible levels, thank you very much.
However, just note that this workaround does absolutely nothing when using SandyBridge hardware-based decoding (i3 2100 CPU). Latency remains in the realm of 500ms.


Jack Chimasera

Best Reply

Hi Jack,
That's right, the workaround is for the sw decoder only (I thought I've mentioned that). Unfortunately there's no such easy workaround for the current SNB decoder. MSDK 3.0 features (API version 1.3)are targeted for the next generation of Intel platforms code named Ivy Bridge. Its hardware decoders will fully support video conferencing features. Sandy Bridge version of Media SDK will hopefully add this support in future as well.

Hi Nina,


Do you mean, MSDK 2.0 can't support video conferencing features? Thus, it'snot goodto use MSDK 2.0 to develop video conference application. Is that right?


Another question:

As you said, the next generation of Intel Platforms is Ivy Bridge, and MSDK 3.0 is targeted at Ivy Bridge. At this time, Sandy Bridge version of Media SDK can't support video conferencing features. Is that right? And what is the Sandy Bridge version of Media SDK now, 2.0 or 3.0 beta2?




Hi Qiang,
That's correct. Media SDK 2.0 has the API version 1.1 which doesn't include many of important VC related features (low latency, dynamic bitrate change etc.). Sandy Bridge version corresponds to MSDK 2.0 and has the API 1.1.
So it's not recommended to program VC apps with MSDK 2.0 (maybe except for some very limited models) and with SandyBridge version as of now.
MSDK 3.0 and Ivy Bridge library have the API version 1.3 which fully supports VC features.
It's best to start prototyping using 1.3 API and SW library from MSDK 3.0 Beta (taking into account all the workarounds :)). When Ivy Bridge platforms with specific MSDK library are available your app will smoothly switch to using hardware acceleration.

Login to leave a comment.