Saving Frames Of Streaming H264 Video Into JPEGs

Saving Frames Of Streaming H264 Video Into JPEGs

I have streaming H264 video that I want to create jpeg still images of every X frame.  I am using the UMC video decoder example to decode the H.264 video to a "raw" format (I'm using RGB).  Once I want to encode one of the frames into a JPEG, I use the UIC library to copy that  frame of RBG data into a CIppImage object and then encode it using code based off the uic transcode example.

When I save the jpeg to a file and open it, I see the content correctly but the colors are all wrong (mostly shades of pink and green).  I also tried decoding the video frames to RGBA and encode a JPEG from that type, but again the colors are wrong (in this case, the image looks like a negative of the correct colors).  I'm not familiar with image and video processing, so I'm having a hard time determing where something is wrong based upon the results I'm seeing.  Any suggestions of what could be wrong when I have the content correct, just the colors are not? Is there a better way to accomplish my goal rather than using the method I defined in the first paragraph?  Part of my problem is finding a color format that is in common with what the video codec can decode into and what the UIC codec can encode from.  The only options that I see are either RGB or RGBA, does the decoded video frames of RGB type align with the format of RGB data when encoding a JPEG?

Another thing I would like to verify (but can't) is if the decoded frames are correct.  I was unable to find a program that could open a "raw" frame of just the RGB data.  My only guess that the decoded video is correct is that I was able to save the decoded video and then used the umc_video_enc_con sample binary to encode that raw video.  When doing that, the newly encoded video looks good, so I'm assuming (perhaps incorrectly?) that the decoded video is in a proper format.  Any suggested program that can open this "raw" RGB data format so I can at least verify the decoding is correct and it's just the JPEG encoding is wrong (which I think is the case).

4 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.

Hi Stephen,

first let's verify if the decode frames are correct. Have you tried to use simple_player.exe or third party player to see the decoded image? by default they are YUV420 format.

You mentioned you get "raw" RGB , but the color is wrong after RGB and jpeg. From H.2664 decoder readme , the supported output color format only --f. It seems the problem come here. as "BRG24" is not same as "RGB24". It may need to add some color conversion function like BRG24 to RGB24 , then use UIC.

The UMC color conversion are done by ipp-samples\audio-video-codecs\codec\color_space_converter\umc_color_space_conversion.cpp. You may obverse it and find the solution.

Best Regards,

I am using IPP samples, I think some of the things you reference must apply to a different version, as I was unable to find them.

simple_player is only for videos, I can't view images with it. I tried playing the decoded video with it, but the player continually gave errors, I don't think it knows how to play the raw decoded video format.

Where is the H.264 readme you mentioned? What are the supported output colors (your post just had --f but not colors listed)? I've been looking if there is a limitation of decode color formats, but the umc_video_dec_con example applicaiton lists rgb24 as a valid color format. Here's the output of the usage print:
Color formats: gray yv12 nv12 yuy2 uyvy yuv420 yuv422 yuv444
rgb24 rgb32 bgr24 bgr32 bgr565 bgr555 bgr444

Is RGB24 color format for videos the same as the JC_RGB format used in the JPEG encoder of UIC?

The file umc_color_space_conversion.cpp does not exist, but I found what I believe is the same thing you mentioned at audio-video-codecs/codec/video/common/cc/src/umc_filter_color_conversion.cpp. I put prints in the color conversion routines and verified that when I run the umc_video_dec_con function with a parameter of -f rgb (as well as my own functions) I see that the color conversion routine to convert from YUV420 to RGB does get run. That confirms that I am getting out RGB frames from the H264 decoding, so I guess the question is how come UIC doesn't seem to be able to utilize that RGB format to encode the image correctly?

Best Reply

I have it working now. The problem was when the function "SaveImageJPEG" gets called, the "image" that is passed in must have the color set. So now before calling SaveImageJPEG I just added the line: image.Color(IC_RGB) and the colors are coming out correctly.

Also, I found a bug in the sample code which ends up not causing a problem, but is wrong. The enumeration values for "JCOLOR" are not 1-for-1 matches for the enumeration values for "IM_COLOR". In the "SaveImageJPEG" function, for each different JPEG encoding type it has the line: je_params.SetColor((JCOLOR)param.color); This is incorrect to cast param.color (an IC_IMAGE) to JCOLOR as those enumerations do not line up. As the code is implemented now, it doesn't actually end up causing an issue, but I think there should be a conversion done to the right enum before making this function call.

Leave a Comment

Please sign in to add a comment. Not a member? Join today