Multimodal Spontaneous Emotion Corpus for Human Behavior Analysis

• Published in: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 3438 - 3446
• Main Authors: Zheng Zhang, Girard, Jeffrey M., Yue Wu, Xing Zhang, Peng Liu, Ciftci, Umur, Canavan, Shaun, Reale, Michael, Horowitz, Andrew, Huiyuan Yang, Cohn, Jeffrey F., Qiang Ji, Lijun Yin
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.06.2016
• Subjects: Blood pressure; Physiology; Sensor systems; Thermal sensors; Three-dimensional displays; Two dimensional displays
• ISSN: 1063-6919
• DOI: 10.1109/CVPR.2016.374

Emotion is expressed in multiple modalities, yet most research has considered at most one or two. This stems in part from the lack of large, diverse, well-annotated, multimodal databases with which to develop and test algorithms. We present a well-annotated, multimodal, multidimensional spontaneous emotion corpus of 140 participants. Emotion inductions were highly varied. Data were acquired from a variety of sensors of the face that included high-resolution 3D dynamic imaging, high-resolution 2D video, and thermal (infrared) sensing, and contact physiological sensors that included electrical conductivity of the skin, respiration, blood pressure, and heart rate. Facial expression was annotated for both the occurrence and intensity of facial action units from 2D video by experts in the Facial Action Coding System (FACS). The corpus further includes derived features from 3D, 2D, and IR (infrared) sensors and baseline results for facial expression and action unit detection. The entire corpus will be made available to the research community.