Recognizing Facial Expressions in Image Sequences Using Local Parameterized Models of Image Motion

• Published in: International journal of computer vision Vol. 25; no. 1; pp. 23 - 48
• Main Authors: Black, Michael J., Yacoob, Yaser
• Format: Journal Article
• Language: English
• Published: Heidelberg Springer 01.10.1997; Springer Nature B.V
• Subjects: Algorithmics. Computability. Computer arithmetics; Applied sciences; Artificial intelligence; Computer science; control theory; systems; Estimation; Exact sciences and technology; Human computer interaction; Image understanding; Mathematical models; Object recognition; Optical flows; Pattern recognition. Digital image processing. Computational geometry; Theoretical computing; Parameter estimation; Motion estimation; Image processing; Image sequence; Estimator robustness; Pattern recognition; Experimental study; Face; Optical flow
• ISSN: 0920-5691; 1573-1405
• DOI: 10.1023/A:1007977618277

This paper explores the use of local parametrized models of image motion for recovering and recognizing the non-rigid and articulated motion of human faces. Parametric flow models (for example affine) are popular for estimating motion in rigid scenes. We observe that within local regions in space and time, such models not only accurately model non-rigid facial motions but also provide a concise description of the motion in terms of a small number of parameters. These parameters are intuitively related to the motion of facial features during facial expressions and we show how expressions such as anger, happiness, surprise, fear, disgust, and sadness can be recognized from the local parametric motions in the presence of significant head motion. The motion tracking and expression recognition approach performed with high accuracy in extensive laboratory experiments involving 40 subjects as well as in television and movie sequences.