FAMOS: a framework for investigating the use of face features to identify spontaneous emotions

• Published in: Pattern analysis and applications : PAA Vol. 22; no. 2; pp. 683 - 701
• Main Authors: Costa-Abreu, Márjoy Da, Bezerra, Giuliana Silva
• Format: Journal Article
• Language: English
• Published: London Springer London 01.05.2019; Springer Nature B.V
• Subjects: Computer Science; Emotions; Ground truth; Man-machine interfaces; Pattern Recognition; Psychology; Theoretical Advances; Emotion analysis; Action units; Face biometrics; Facial expression analysis; Emotion models
• ISSN: 1433-7541; 1433-755X
• DOI: 10.1007/s10044-017-0675-y

Emotion-based analysis has raised a lot of interest, particularly in areas such as forensics, medicine, music, psychology, and human-machine interface. Following this trend, the use of facial analysis (either automatic or human-based) is the most common subject to be investigated once this type of data can easily be collected and is well accepted in the literature as a metric for inference of emotional states. Despite this popularity, due to several constraints found in real-world scenarios (e.g. lightning, complex backgrounds, facial hair and so on), automatically obtaining affective information from face accurately is a very challenging accomplishment. This work presents a framework which aims to analyse emotional experiences through spontaneous facial expressions. The method consists of a new four-dimensional model, called FAMOS, to describe emotional experiences in terms of appraisal, facial expressions, mood, and subjective experiences using a semi-automatic facial expression analyser as ground truth for describing the facial actions. In addition, we present an experiment using a new protocol proposed to obtain spontaneous emotional reactions. The results have suggested that the initial emotional state described by the participants of the experiment was different from that described after the exposure to the eliciting stimulus, thus showing that the used stimuli were capable of inducing the expected emotional states in most individuals. Moreover, our results pointed out that spontaneous facial reactions to emotions are very different from those in prototypic expressions, especially in terms of expressiveness.