Saving Frames Of Streaming H264 Video Into JPEGs

Saving Frames Of Streaming H264 Video Into JPEGs

I have streaming H264 video that I want to create jpeg still images of every X frame.  I am using the UMC video decoder example to decode the H.264 video to a "raw" format (I'm using RGB).  Once I want to encode one of the frames into a JPEG, I use the UIC library to copy that  frame of RBG data into a CIppImage object and then encode it using code based off the uic transcode example.

When I save the jpeg to a file and open it, I see the content correctly but the colors are all wrong (mostly shades of pink and green).  I also tried decoding the video frames to RGBA and encode a JPEG from that type, but again the colors are wrong (in this case, the image looks like a negative of the correct colors).  I'm not familiar with image and video processing, so I'm having a hard time determing where something is wrong based upon the results I'm seeing.  Any suggestions of what could be wrong when I have the content correct, just the colors are not? Is there a better way to accomplish my goal rather than using the method I defined in the first paragraph?  Part of my problem is finding a color format that is in common with what the video codec can decode into and what the UIC codec can encode from.  The only options that I see are either RGB or RGBA, does the decoded video frames of RGB type align with the format of RGB data when encoding a JPEG?

Another thing I would like to verify (but can't) is if the decoded frames are correct.  I was unable to find a program that could open a "raw" frame of just the RGB data.  My only guess that the decoded video is correct is that I was able to save the decoded video and then used the umc_video_enc_con sample binary to encode that raw video.  When doing that, the newly encoded video looks good, so I'm assuming (perhaps incorrectly?) that the decoded video is in a proper format.  Any suggested program that can open this "raw" RGB data format so I can at least verify the decoding is correct and it's just the JPEG encoding is wrong (which I think is the case).

4 post / 0 nuovi
Ultimo contenuto
Per informazioni complete sulle ottimizzazioni del compilatore, consultare l'Avviso sull'ottimizzazione

Hi Stephen,

first let's verify if the decode frames are correct. Have you tried to use simple_player.exe or third party player to see the decoded image? by default they are YUV420 format.

You mentioned you get "raw" RGB , but the color is wrong after RGB and jpeg. From H.2664 decoder readme , the supported output color format only --f. It seems the problem come here. as "BRG24" is not same as "RGB24". It may need to add some color conversion function like BRG24 to RGB24 , then use UIC.

The UMC color conversion are done by ipp-samples\audio-video-codecs\codec\color_space_converter\umc_color_space_conversion.cpp. You may obverse it and find the solution.

Best Regards,
Ying

I am using IPP samples 7.1.0.011, I think some of the things you reference must apply to a different version, as I was unable to find them.

simple_player is only for videos, I can't view images with it. I tried playing the decoded video with it, but the player continually gave errors, I don't think it knows how to play the raw decoded video format.

Where is the H.264 readme you mentioned? What are the supported output colors (your post just had --f but not colors listed)? I've been looking if there is a limitation of decode color formats, but the umc_video_dec_con example applicaiton lists rgb24 as a valid color format. Here's the output of the usage print:
Color formats: gray yv12 nv12 yuy2 uyvy yuv420 yuv422 yuv444
rgb24 rgb32 bgr24 bgr32 bgr565 bgr555 bgr444

Is RGB24 color format for videos the same as the JC_RGB format used in the JPEG encoder of UIC?

The file umc_color_space_conversion.cpp does not exist, but I found what I believe is the same thing you mentioned at audio-video-codecs/codec/video/common/cc/src/umc_filter_color_conversion.cpp. I put prints in the color conversion routines and verified that when I run the umc_video_dec_con function with a parameter of -f rgb (as well as my own functions) I see that the color conversion routine to convert from YUV420 to RGB does get run. That confirms that I am getting out RGB frames from the H264 decoding, so I guess the question is how come UIC doesn't seem to be able to utilize that RGB format to encode the image correctly?

Best Reply

I have it working now. The problem was when the function "SaveImageJPEG" gets called, the "image" that is passed in must have the color set. So now before calling SaveImageJPEG I just added the line: image.Color(IC_RGB) and the colors are coming out correctly.

Also, I found a bug in the sample code which ends up not causing a problem, but is wrong. The enumeration values for "JCOLOR" are not 1-for-1 matches for the enumeration values for "IM_COLOR". In the "SaveImageJPEG" function, for each different JPEG encoding type it has the line: je_params.SetColor((JCOLOR)param.color); This is incorrect to cast param.color (an IC_IMAGE) to JCOLOR as those enumerations do not line up. As the code is implemented now, it doesn't actually end up causing an issue, but I think there should be a conversion done to the right enum before making this function call.

Accedere per lasciare un commento.