Building predictive models of emotion with functional near-infrared spectroscopy

• Published in: International journal of human-computer studies Vol. 110; pp. 75 - 85
• Main Authors: Bandara, Danushka, Velipasalar, Senem, Bratt, Sarah, Hirshfield, Leanne
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.02.2018
• Subjects: Affective computing; Arousal classification; Brain signal processing; Emotion classification; fNIRS; Valence classification; Affective computing; fNIRS; Brain signal processing; Arousal classification; Emotion classification; Valence classification
• DOI: 10.1016/j.ijhcs.2017.10.001
• ISSN: 1071-5819

•Goal of this work is to classify affective states of Valence and Arousal using functional Near Infra-Red Spectroscopy.•Use Russel's circumplex model as the representation of affect.•Suggest novel method of using Brain ``Regions of Interest” as a feature extraction technique for fNIRS based machine learning.•Across subject classification provided higher f1-score for Valence (0.739) than previous literature. We demonstrate the capability of discriminating between affective states on the valence and arousal dimensions using functional near-infrared spectroscopy (fNIRS), a practical non-invasive device that benefits from its ability to localize activation in functional brain regions with spatial resolution superior to the Electroencephalograph (EEG). The high spatial resolution of fNIRS enables us to identify the neural correlates of emotion with spatial precision comparable to fMRI, but without requiring the use of the constricting and impractical fMRI scanner. We make these predictions across subjects, creating the capacity to generalize the model to new participants. We designed the experiment and evaluated our results in the context of a prior experiment—based on the same basic protocol and stimulus materials—which used EEG to measure participants’ valence and arousal. The F1-scores achieved by our classifiers suggest that fNIRS is particularly useful at distinguishing between high and low levels of valence (F1-score of 0.739), which has proven to be difficult to measure with physiological sensors.