FusionSense: Emotion Classification Using Feature Fusion of Multimodal Data and Deep Learning in a Brain-Inspired Spiking Neural Network

• Published in: Sensors (Basel, Switzerland) Vol. 20; no. 18; p. 5328
• Main Authors: Tan, Clarence, Ceballos, Gerardo, Kasabov, Nikola, Puthanmadam Subramaniyam, Narayan
• Format: Journal Article
• Language: English
• Published: Switzerland MDPI 17.09.2020; MDPI AG
• Subjects: Brain - diagnostic imaging; Deep Learning; Electroencephalography; Emotions; Evolving Spiking Neural Networks (eSNNs); facial emotion recognition; Humans; multimodal data; NeuCube; Neural Networks, Computer; Spatio-temporal data; Evolving Spiking Neural Networks (eSNNs); Spatio-temporal data; facial emotion recognition; multimodal data; NeuCube

Using multimodal signals to solve the problem of emotion recognition is one of the emerging trends in affective computing. Several studies have utilized state of the art deep learning methods and combined physiological signals, such as the electrocardiogram (EEG), electroencephalogram (ECG), skin te...