FEAD: Introduction to the fNIRS-EEG Affective Database - Video Stimuli

• Published in: IEEE transactions on affective computing Vol. 16; no. 1; pp. 15 - 27
• Main Authors: Nia, Alireza Farrokhi, Tang, Vanessa, Malyshau, Valery, Barde, Amit, Talou, Gonzalo Daniel Maso, Billinghurst, Mark
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.01.2025; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Affect (Psychology); Affective computing; Algorithms; Biomedical monitoring; Brain modeling; Electroencephalography; Emotion recognition; Emotions; Functional near-infrared spectroscopy; Hemodynamic responses; human-computer interaction; Infrared spectra; machine learning; Medical imaging; Near infrared radiation; Nervous system; neurovascular coupling; Ratings; Spectroscopy; video stimuli; Visual stimuli
• ISSN: 1949-3045; 1949-3045
• DOI: 10.1109/TAFFC.2024.3407380

This article presents FEAD, a fNIRS-EEG Affective Database that can be used for training emotion recognition models. The electrical activity and brain hemodynamic responses of 37 participants were recorded, as well as the categorical and dimensional emotion ratings they gave to 24 affective audio-visual stimuli. The relationship between the neurophysiological signals with the subjective ratings was investigated, with a significant correlation found in the prefrontal cortex region. A binary classification of affective states was performed using a subject-dependent approach, taking into account the fusion of both modalities, functional Near-Infrared Spectroscopy and Electroencephalography, and each single modality separately. In addition, we explored the temporal dynamics of the recorded data in shorter trials and found that the fusion of features from both modalities yielded significantly better results than using a single modality. This database will be made publicly available with the aim to encourage researchers to develop more advanced algorithms for affective computing and emotion recognition.