Practical Motion Estimation (Part II, And Yet It Moves!)

«People, we've talked about this. Acting is reacting.

Mr. Timms? You were good. Perhaps a little too good.»

--Rango (a chameleon)

In my previous post we tried to make a deep dive into the anatomy of motion estimation algorithms in video processing. Perhaps a little too deep ;)…

Now, let’s have a look at how it “looks.” I mean literally, how do the motion vectors look? The question is: do motion vectors in the video below trace the handle knob movement trajectory?

Or, to put it bluntly: how “smart” is the motion estimate algorithm? If it is not very smart, the rotating knob image will be stored in each frame. On the other hand, if it is really smart, the knob will be stored in the key frames only, while other frames will contain the information on its movement trajectory.

To cut a long story short. Here is the illustrated answer:

I hope it is obvious enough. The image is a bit small to recognize its tiny details so I’ll provide some explanations. The black lines forming something like a mesh or a maze represent division into macroblocks. Initial block size is 16x16 pixels square. We can also see a number of various subsequent splits: in two 16x8 or 8x16 blocks, in four 8x8 blocks, or in 16 4x4 blocks. The decision on the split is taken by the motion estimation unit. After searching through all kinds of options, it decided that it’s better to split the block into, say, four block of 8x8. It’ll help to find similar blocks at another frame.

The chaotic red lines are motion vectors. You may note that motion vectors are messed up here and there, although that’s often the real life case.

Let’s go back to the handle knob. That’s really interesting! The macroblocks have mostly been split into four 8x8 blocks. As for the motion vectors, they are in the flush of enthusiasm aimed to the knob’s location at the previous frame.

So the answer is positive! And yet it Moves!

A new question, purely in scientific interest… Why do you think some blocks are split into small subblocks without any motion vector?

For more complete information about compiler optimizations, see our Optimization Notice.


anonymous's picture

No. I have negligible experience in video coding but my research interests are in image processing and graphics.

anonymous's picture

Yes. As I see finely subdivided blocks lie adjacent to blocks with mv.

dmitry.serkin (Intel)'s picture

Alok, I am not sure that I understand you correctly. Do you mean that empty (no mv) blocks are intended for prediction within current frame?

anonymous's picture

Blocks, without containing motion vectors, are subdivided more to be used as reference frames lying adjacent to blocks with motion vectors.
The finer division is useful for accuracy to motion estimation. I guess for comparison bases vectors are extracted from image spaces, represented by the smaller blocks, and the portion of frame to be matched to do estimation. In case of no change, next portion of the frame is considered thereby speeding up the process.

Add a Comment

Have a technical question? Visit our forums. Have site or software product issues? Contact support.