Deep learning framework for subject-independent emotion detection using wireless signals

• Published in: PloS one Vol. 16; no. 2; p. e0242946
• Main Authors: Khan, Ahsan Noor, Ihalage, Achintha Avin, Ma, Yihan, Liu, Baiyang, Liu, Yujie, Hao, Yang
• Format: Journal Article
• Language: English
• Published: United States Public Library of Science 03.02.2021; Public Library of Science (PLoS)
• Subjects: Biology and Life Sciences; Computer and Information Sciences; Engineering and Technology; Facial expression; Machine learning; Methods; Observations; Physical Sciences; Research and Analysis Methods; Social Sciences; United Kingdom
• DOI: 10.1371/journal.pone.0242946
• ISSN: 1932-6203

Emotion states recognition using wireless signals is an emerging area of research that has an impact on neuroscientific studies of human behaviour and well-being monitoring. Currently, standoff emotion detection is mostly reliant on the analysis of facial expressions and/or eye movements acquired from optical or video cameras. Meanwhile, although they have been widely accepted for recognizing human emotions from the multimodal data, machine learning approaches have been mostly restricted to subject dependent analyses which lack of generality. In this paper, we report an experimental study which collects heartbeat and breathing signals of 15 participants from radio frequency (RF) reflections off the body followed by novel noise filtering techniques. We propose a novel deep neural network (DNN) architecture based on the fusion of raw RF data and the processed RF signal for classifying and visualising various emotion states. The proposed model achieves high classification accuracy of 71.67% for independent subjects with 0.71, 0.72 and 0.71 precision, recall and F1-score values respectively. We have compared our results with those obtained from five different classical ML algorithms and it is established that deep learning offers a superior performance even with limited amount of raw RF and post processed time-sequence data. The deep learning model has also been validated by comparing our results with those from ECG signals. Our results indicate that using wireless signals for stand-by emotion state detection is a better alternative to other technologies with high accuracy and have much wider applications in future studies of behavioural sciences.