I'm writing a media streaming client based on the IPP 5.0 Media Samples. Currently, I want to play MPEG-4 and MP3 media over RTP. I've written a splitter that receives data over RTP and my program (currently just a slightly modified version of simple_player + my splitter) plays the video.
However, video playback is choppy. I have a cache which is just a UMC::LinearBuffer which is filled as frames come in from RTP. A call to GetNextVideoData() unlocks the used output data (thus allowing incoming data to be put there) and locks the not-yet-processed output data. For the first few seconds or so, the video will play fine, but then it pauses, then speeds up to catch up, then pauses, and so on. If I use UMC::SampleBuffer, playback is slow - about 1 frame per second.
I'm not sure what's causing this exactly. I've set the framerate correctly (and also tried setting it to zero). I'm setting the presentation time correctly, although I'm not sure if the MPEG-4 decoder uses my splitter-specified PTS anyway. AVI splitter, MPEG2-PS splitter, and MP4 splitter all have different ways of handling their data, so I'm not sure which one is the best example, although it seems to me that MP4 splitter should be just like using UMC::SampleBuffer for my cache, so I'm perplexed.
Any tips for properly feeding the MPEG-4 decoder (or decoders in general)?