Emotion recognition while applying cosmetic cream using deep learning from EEG data; cross-subject analysis

• Published in: PloS one Vol. 17; no. 11; p. e0274203
• Main Authors: Kim, Jieun, Hwang, Dong-Uk, Son, Edwin J., Oh, Sang Hoon, Kim, Whansun, Kim, Youngkyung, Kwon, Gusang
• Format: Journal Article
• Language: English
• Published: San Francisco Public Library of Science 10.11.2022
• Subjects: Accuracy; Analysis; Biology and Life Sciences; Brain research; Classification; Computer and Information Sciences; Consumer behavior; Cosmetics; Deep learning; Electroencephalography; Emotion recognition; Emotional intelligence; Emotions; Feature extraction; Frequencies; Frequency dependence; Influence; Machine learning; Medicine and Health Sciences; Methods; Multimedia; Musical performances; Research and Analysis Methods; Social Sciences; Spatial data; Wavelet transforms; South Korea
• ISSN: 1932-6203; 1932-6203
• DOI: 10.1371/journal.pone.0274203

We report a deep learning-based emotion recognition method using EEG data collected while applying cosmetic creams. Four creams with different textures were randomly applied, and they were divided into two classes, “like (positive)” and “dislike (negative)”, according to the preference score given by the subject. We extracted frequency features using well-known frequency bands, i.e., alpha, beta and low and high gamma bands, and then we created a matrix including frequency and spatial information of the EEG data. We developed seven CNN-based models: (1) inception-like CNN with four-band merged input, (2) stacked CNN with four-band merged input, (3) stacked CNN with four-band parallel input, and stacked CNN with single-band input of (4) alpha, (5) beta, (6) low gamma, and (7) high gamma. The models were evaluated by the Leave-One-Subject-Out Cross-Validation method. In like/dislike two-class classification, the average accuracies of all subjects were 73.2%, 75.4%, 73.9%, 68.8%, 68.0%, 70.7%, and 69.7%, respectively. We found that the classification performance is higher when using multi-band features than when using single-band feature. This is the first study to apply a CNN-based deep learning method based on EEG data to evaluate preference for cosmetic creams.