New kernels for fast mesh-based motion estimation

• Published in: IEEE transactions on circuits and systems for video technology Vol. 11; no. 1; pp. 40 - 51
• Main Author: Nosratinia, A.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.01.2001; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Applied sciences; Computation; Computational efficiency; Deformable models; Errors; Estimation error; Exact sciences and technology; Finite element method; Fundamental areas of phenomenology (including applications); Image forming and processing; Image processing; Imaging and optical processing; Information, signal and communications theory; Interpolation; Iterative algorithms; Kernel; Kernels; Mathematical analysis; Motion analysis; Motion control; Motion estimation; Noise reduction; Optics; Optimization; Physics; Signal processing; Studies; Telecommunications and information theory; Vectors (mathematics); Video coding; Image analysis; Image coding; Motion estimation; Image processing; Data compression; Image sequences; Experimental study; Mesh method
• DOI: 10.1109/76.894281
• ISSN: 1051-8215

Mesh-based motion estimation-also known as control grid interpolation or warping-provides a smoother estimated intensity field compared to the traditional block-matching algorithm (BMA), resulting in most cases in a more realistic motion field and smaller estimation error. In mesh-based motion, unlike BMA, the computation of a motion vector is affected by its neighboring vectors. This interdependence necessitates a costly, iterative computation of motion vectors. The computational cost of mesh-based motion has been a main drawback of this otherwise powerful technique. We propose to use noniteratively computed motion vectors, such as BMA motion vectors, for node motions in the mesh model. However, we found that a straightforward insertion of BMA motion vectors in the deformable mesh leads to unpredictable and erratic results, and were thus motivated to carefully analyze the interaction of motion vectors and interpolation kernels in mesh models. This analysis leads to a methodology for computing optimal motion interpolation kernels for a given set of motion vectors (e.g., BMA motion vectors). We find a generalized orthogonality condition for these kernels; optimality is achieved only if the projections of vertex motions on the local intensity gradients are statistically orthogonal to mesh-based estimation errors. Experiments show that optimal kernels are often very different from the traditional bilinear kernels, and exhibit interesting variations. The new kernels benefit a variety of applications, including motion estimated interpolation, denoising, and compression.