Noise-Robust Deep Learning Model for Emotion Classification using Facial Expressions

• Published in: IEEE access Vol. 12; p. 1
• Main Authors: Oh, SeungJun, Kim, Dong-Keun
• Format: Journal Article
• Language: English
• Published: IEEE 01.01.2024
• Subjects: Data models; Emotion classification; Emotion recognition; Face recognition; Facial expression; Image recognition; Noise measurement; Noise-robust model; Physiology; Speech recognition
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2024.3436881

In emotion classification using facial expressions, noise in the data is the main factor hindering improvement in accuracy. In this study, a deep learning model was developed that is robust to noise in image data when classifying emotions using facial expressions. Fifty-three subjects were asked to make expressions corresponding to four emotions for 15 s each, which were then classified as positive or negative. Subsequently, facial expressions were extracted from the acquired images and stored as image data, and noise was applied in five steps. In addition, the randomness of the noise was confirmed during its application. A deep learning model that can classify emotions using preprocessed facial-expression data was constructed, and the classification metrics were compared. When facial-expression data with noise were applied step-by-step to the deep learning model, its robustness against noise increased by up to 15%. However, the results were unsatisfactory when more noise was used. The reason for the vulnerability to noise is discussed based on the results. This study confirmed that a deep learning model robust against noise at a certain level of data could be established for classifying emotions using facial expressions.