Voice and Facial Expression Based Classification of Emotion Using Linear Support Vector Machine

• Published in: 2009 Second International Conference on Developments in eSystems Engineering pp. 377 - 384
• Main Authors: Das, S., Halder, A., Bhowmik, P., Chakraborty, A., Konar, A., Nagar, A.K.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.12.2009
• Subjects: Automatic speech recognition; Computer science; Emotion recognition; Face recognition; Facial expression; Feature extraction; Hidden Markov models; Linear Classification; Linear Support Vector Machine; Neural networks; Psychology; Speech; Support vector machine classification; Support vector machines
• ISBN: 1424454018; 9781424454013
• DOI: 10.1109/DeSE.2009.9

The paper provides a novel approach to emotion recognition from facial expression and voice of subjects. The subjects are asked to manifest their emotional exposure in both facial expression and voice, while uttering a given sentence. Facial features including mouth-opening, eye-opening, eyebrow-constriction, and voice features including, first three formants: F1, F2, and F3, and respective powers at those formants, and pitch are extracted for 7 different emotional expressions of each subject. A linear Support Vector Machine classifier is used to classify the extracted feature vectors into different emotion classes. Sensitivity of the classifier to Gaussian noise is studied, and experimental results confirm that the recognition accuracy of emotion up to a level of 95% is maintained, even when the mean and standard deviation of noise are as high as 5% and 20% respectively over the individual features. A further analysis to identify the importance of individual features reveals that mouth-opening and eye-opening are primary features, in absence of which classification accuracy falls off by a large margin of more than 22%.