Automatic facial expression recognition using features of salient facial patches

• Published in: IEEE transactions on affective computing Vol. 6; no. 1; pp. 1 - 12
• Main Authors: Happy, S. L., Routray, Aurobinda
• Format: Journal Article
• Language: English
• Published: IEEE 01.01.2015
• Subjects: Eyebrows; Face; Face recognition; Facial expression analysis; facial landmark detection; Feature extraction; feature selection; Image edge detection; Image resolution; low resolution image; Nose; salient facial patches; Facial expression analysis; facial landmark detection; low resolution image; salient facial patches; feature selection
• ISSN: 1949-3045; 1949-3045
• DOI: 10.1109/TAFFC.2014.2386334

Extraction of discriminative features from salient facial patches plays a vital role in effective facial expression recognition. The accurate detection of facial landmarks improves the localization of the salient patches on face images. This paper proposes a novel framework for expression recognition by using appearance features of selected facial patches. A few prominent facial patches, depending on the position of facial landmarks, are extracted which are active during emotion elicitation. These active patches are further processed to obtain the salient patches which contain discriminative features for classification of each pair of expressions, thereby selecting different facial patches as salient for different pair of expression classes. One-against-one classification method is adopted using these features. In addition, an automated learning-free facial landmark detection technique has been proposed, which achieves similar performances as that of other state-of-art landmark detection methods, yet requires significantly less execution time. The proposed method is found to perform well consistently in different resolutions, hence, providing a solution for expression recognition in low resolution images. Experiments on CK+ and JAFFE facial expression databases show the effectiveness of the proposed system.