VPP or h.264 encode reducing mean intensity

VPP or h.264 encode reducing mean intensity

I have modifiedsample codeto read from an RGB3 AVI and encode to h.264 using VPP for RGB4 to NV12 color format conversion. I'm using hw API 1.3. The output has lower mean intensity than the input (109 down to 105 for the frame I'm looking at).

I have tried high bit rates where the output is essentially identical to the input except for mean intensity, so this is independent of bit rate, and by eye appears to be uniform, so it is not highlight blurring/clipping. I have tried both baseline and main profiles with no change in result. I have turned off VPP denoise, scene, detail, and procamp with VppDoNotUse with no change in result.

I am not comparing raw h.264 frames, rather I am muxing to mp4 and then extracting frames with ffmpeg, but doing this on h.264 data from a different encoder does not show the same reduction in intensity.

My system:
- Media SDK 2012 R2 (x64)
- Core i7 3770K (HD 4000)
- Windows 7 x64 SP1
- latest intel graphics driver (

Does 1.3 hw VPP or h.264 encode drop intensity?


10 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.

Hi Ryan,
Thanks for sharing your findings with us.

For us to make comparable investigation could you please share with us how you calculate the intensity. I'dlike to make sure we agree on the method.

Do you seesimilar intensity delta for all frames in your raw input stream and is the delta in the same range independent of what input you are using?
Are you only observing this issue when using RGB4->NV12 VPP conversion?


My intensity metric is mean pixel value after converting to 8u grayscale.

Here are some stats on per-frame mean pixel value differences for a short clip. So for 327 frames I compared the original to the transcoded frame. I converted both frames to grayscale, then computed the mean ofeach image, and the subtracted (originalMean - transcodedMean):

Mean 1.912494914

Standard Error 0.056628696

Median 1.742355

Mode #N/A

Standard Deviation 1.024024712

Sample Variance 1.048626612

Kurtosis -0.510323895

Skewness 0.102027743

Range 5.714798

Minimum -0.827156

Maximum 4.887642

Sum 625.385837

Count 327

Sofor theaverage frame, its grayscale mean intensity dropped by 1.9.For one frame,its grayscale meanintensity dropped by 4.88.

A different encoder has a nearly identical range magnitude, but the range is close to centered around 0, so the mean is near 0. Its standard deviation is 0.7, so a little lower but same ballpark.

For your second question, here are stats for a different clip:

Mean 1.036684283

Standard Error 0.062302422

Median 1.271118

Mode #N/A

Standard Deviation 0.861036598

Sample Variance 0.741384023

Kurtosis -0.191864263

Skewness -0.262728729

Range 4.726814

Minimum -1.107979

Maximum 3.618835

Sum 198.006698

Count 191

I understand your last question but haven't run that test yet.

For the last question, when I use IPP for the RGB to YUV colorspace conversion (to avoid using VPP), the intensity is greatly increased in the output. So IPP and Quicksync seem to have a different RGB->YUV conversion regarding intensity so this isolation is not instructive.

Hi Ryan,

This is quite strange, I do not see the same effect on my side. I tried both the driver version you used and a more recent internal driver with the same results.

First I tested the intensity difference when using VPP conversion from RGB to NV12. For the workloads I tried the greatest intensity delta I observe is ~0.35.

Using Media SDK I also produced a few H.264 video streams with varying bitrate using different content, then decoded the streams. The calculated intensity delta is very low, the greatest average intensity delta comparing encoder inoput to decoder output was ~0.30, for most workloads the delta was much smaller.
It could be that the content you are using is especially challenging? Can you share with us the content that showcases the worst intensity delta?

Tocalculate intensity, for comparable results, I used a method similar to what you describe. YUV->RGB->grayscale->intensity. This was performed on the raw input to encoder and the raw output from decoder


Here is a fairly high bitrate encode using x264:
This is not identical to what I'm running (because I'm running from the original, uncompressed rgb3), but should be close enough.

Hi Ryan,

I tried your content but did not see much difference compared to the content I used. For encode I do not really see any intensity deviation at all. For VPP RGB->NV12 color conversion slightly greater (~0.40) than what I had seen earlier.

First, I noticed now that you refer to RGB3. Does that mean you are using MFX_FOURCC_RGB3 as the input format to VPP? Note that this Media SDK capability is deprecated and not recommended to use. If you are using this please migrate to using the RGB4 type instead.

If that's not the case my guess is that we are having different results due to the algorithm you use to convert from YUV (NVC12) to RGB color space.


My input file is RGB3, I am converting to RGB4 manually during AVI read so VPP is receiving RGB4. RGB3 is apparently not supported by VPP in version 1.3 API, judging by the MFX_ERR_UNSUPPORTED code I get.

I am using ffmpeg toextract frames fromthe transcoded mp4 (so that is where theresulting h.264 is decoded), and am using the same command for both quicksync and x264 transcodes, so I wouldn't think they would be different, but could be.

I'll post again if I get any more relevant results.


I ran the processing (both VPP and encode) using sw 1.4 and hw 1.3 implementations, and compared them.

Here is (original - sw):
Mean -0.474762046 Standard Error 0.038104205 Median -0.466354 Mode #N/A Standard Deviation 0.689043729 Sample Variance 0.474781261 Kurtosis 2.310864995 Skewness -0.007642003 Range 5.659698 Minimum -3.123901 Maximum 2.535797 Sum -155.247189 Count 327

Sothe sw range is fairly balanced around 0, so mean is near 0, and in this case the output is a little bit brighter on average.

Here is (sw - hw):
Mean 2.294726037 Standard Error 0.024461352 Median 2.075493 Mode #N/A Standard Deviation 0.442338092 Sample Variance 0.195662987 Kurtosis -1.357276659 Skewness 0.371498899 Range 1.679047 Minimum 1.452484 Maximum 3.131531 Sum 750.375414 Count 327

Thisreconfirms the previous resultbydirectly comparingthe sw and hw outputs. The sw implementation makes the content alittle brighter and the hw implementation makes the content aboutdarker.

I think my hw and sw code paths are supplying the same params.


Hi Ryan,

sorry for the delayed response. We have done some more internal investigations into this.
We found some potential issues that are being looked into but not specifically linked to the intensity skew that you reported.

A topic that came up was that different YUV-RGB color conversion algorithms may be used in Media SDK vs. the method you use?

Media SDK usesBT601 for SD video and
BT709 for HD video (source
height greater than 576 lines). The content you provided falls into the former category. If you use BT709 for converting the frames from ffmpeg to RGB this may introduce a skew?

Also, we compared the input vs. output from VPP and separately the input vs. output from encode and did not see much intensity deviation in either case.

You could you try the same on your side by using sample_vpp and sample_encode.

Also, if there is a way for you to provide the raw RGB3 or RGB4 input then we can take a look at that case to cover all bases.


Leave a Comment

Please sign in to add a comment. Not a member? Join today