Ffmpeg doesn't quite work either. The resulting mp4 file plays back in quicktime player with a severe time stutter for the quicksync h.264 input (not for the x264 h.264 input). It looks like it plays 5 frames them jumps back a couple frames, then maybe forward again several frames. It makes forward progress through the clip at about the right rate, but is jumping forwards and backwards, maybe due to B frame decode order.
I'm using: ffmpeg -i q.264 -vcodec copy fq.mp4
Am I missing mfxExtPictureTimingSEI or something like that?
Thanks,
Ryan

EDIT: Correction, Quicktime also plays the x264-produced stream in decode order, not presentation order. It is just less noticeable because B frames are more rare with default settings.

Here is the quicksync h.264 file.

Yes, please, I would like to try the hacked source you mentioned. I did notice that ffmpeg complains about missing PTS on every frame ("pts has no value").

I did try to hack in timestamps, using constant incrementing 90KHz units into mfxBitstream.TimeStamp just before CSmplBitstreamWriter::WriteNextFrame. And also by setting the same timestamp into mfxFrameSurface1.Data.TimeStamp. But neither of those fixed the problem. I did not try to account for frame reordering.

mp4box has an option to dump pts values. (mp4box -std -dts fq.mp4) I do not see any substantive difference between a working and non-working mp4 file. Can you recommend any other (inexpensive) tool to give visibility into the h.264 stream to look for differences? I think I need to see frame type and dts/pts.

Thanks,
Ryan

(In this post I am disregarding your integrated ffmpeg mux, I am still just dealing with my code that writes a raw h.264 file.) There seem to be several things I don't know about dts/pts handling. You are setting TimeStamp values to successive integers, so I tried that too. But ffmpeg -dump -debug_ts still shows 90KHz dts/pts values. Where are the 90KHz values coming from?

Note that ffmpeg displays very similar dts/pts values, and also the "pts has no value" message, for the x264 mux and that one plays correctly in quicktime.

Ryan

EDIT: Correction, the x264 version also plays incorrectly. So the timestamps are not the problem.

In CSmplBitstreamWriter::WriteNextFrame I printed the TimeStamp of each mfxBitstream as it is written and found that the frames are written in decode order (not presentation order). The decode order looks good (B frames come after the future frames they reference). Then I rendered frame numbers onto the frames themselves, and now I see that after ffmpeg mux Quicktime is presenting the frames in the order in which they are in the h.264 stream (file), which is the decode order, not the presentation order. Hence the temporal jitter.

So it seems like the presentation order is represented in the h.264 stream in a way that works for other decoders but not a way that Quicktime honors (after muxing by ffmpeg).

Note my correction above. The x264/ffmpeg file also plays in the incorrect order in quicktime. So the frame order issue is not a media sdk sample problem (or at least not exclusively a media sdk sample problem). It is most likely an ffmpeg problem. I am using the Oct 10 2012 build of ffmpeg.

So for quicktime playback, the ffmpeg mux doesn't work due to B-frame decode vs presentation order problem for both the media sdk sample encoded stream and the x264 stream. And mp4box mux works for x264 streams but not media sdk streams.

This may be the relevant ffmpeg bug:
https://ffmpeg.org/trac/ffmpeg/ticket/502

Which ffmpeg version are you using? Can you provide a sample (working) mp4 that I can use for comparison?

EDIT: One confusion is talking about ffmpeg.exe vs the libs associated with it (libavformat etc). Another is using ffprobe to look at packets vs frames. This post is looking at ffprobe frames display, not packets. Second edit: fixed the explanation below.

The command-line ffmpeg.exe (as opposed to your code that uses the libs that compose ffmpeg) does not mux correctly for quicktime. When looking at frames using ffprobe, it looks like ffmpeg.exe sets PTS to the decode order, which is not correct. I tried 3 versions of ffmpeg.exe and they all give different PTS/DTS values, but the PTS values are always in decode order, so it never works for quicktime.

Your sample code produces an mp4 that plays correctly in quicktime. Except quicktime swaps frames 2 and 3, maybe due to both having DTS=0? I noticed that for the other two muxers (ffmpeg.exe and mp4box), when looking at frames using ffprobe -show_frames, the dts is always greater than the pts, but not in your mux.