Practical Motion Estimation (Part II, And Yet It Moves!)

«People, we've talked about this. Acting is reacting.

Mr. Timms? You were good. Perhaps a little too good.»

--Rango (a chameleon)

In my previous post we tried to make a deep dive into the anatomy of motion estimation algorithms in video processing. Perhaps a little too deep ;)…

Now, let’s have a look at how it “looks.” I mean literally, how do the motion vectors look? The question is: do motion vectors in the video below trace the handle knob movement trajectory?

Or, to put it bluntly: how “smart” is the motion estimate algorithm? If it is not very smart, the rotating knob image will be stored in each frame. On the other hand, if it is really smart, the knob will be stored in the key frames only, while other frames will contain the information on its movement trajectory.

To cut a long story short. Here is the illustrated answer:

I hope it is obvious enough. The image is a bit small to recognize its tiny details so I’ll provide some explanations. The black lines forming something like a mesh or a maze represent division into macroblocks. Initial block size is 16x16 pixels square. We can also see a number of various subsequent splits: in two 16x8 or 8x16 blocks, in four 8x8 blocks, or in 16 4x4 blocks. The decision on the split is taken by the motion estimation unit. After searching through all kinds of options, it decided that it’s better to split the block into, say, four block of 8x8. It’ll help to find similar blocks at another frame.

The chaotic red lines are motion vectors. You may note that motion vectors are messed up here and there, although that’s often the real life case.

Let’s go back to the handle knob. That’s really interesting! The macroblocks have mostly been split into four 8x8 blocks. As for the motion vectors, they are in the flush of enthusiasm aimed to the knob’s location at the previous frame.

So the answer is positive! And yet it Moves!

A new question, purely in scientific interest… Why do you think some blocks are split into small subblocks without any motion vector?

For more complete information about compiler optimizations, see our Optimization Notice.