Novel Emotion Recognition System Using Edge Computing Platform with Deep Convolutional Networks

• Published in: Journal of intelligent & fuzzy systems Vol. 45; no. 2; pp. 2669 - 2683
• Main Authors: Huang, Jr-Jen, Yang, Cheng-Ying, Lin, Yi-Nan, Shen, Victor R.L., Lin, Chia-Tsai, Shen, Frank H.C.
• Format: Journal Article
• Language: English
• Published: Amsterdam IOS Press BV 01.01.2023
• Subjects: Edge computing; Emotion recognition; Emotions; Frames per second; Microprocessors; Object recognition; Training
• ISSN: 1064-1246; 1875-8967
• DOI: 10.3233/JIFS-223801

Human faces have been naturally viewed as a central part in each image. One interesting task is to classify each face into different categories based on the emotion shown in the facial expression. In addition, an awareness of emotion during work on a project and how affective states are presented in the communication style might help system developers work more effectively, thus improving the performance of a collaborative team. Currently, the feasibility and portability of emotion recognition in the platform with Raspberry PI are insufficient. Hereby, a novel emotion recognition system in real time using the edge computing platform with deep learning has been implemented successfully. The feature values of objects are calculated by a high computing processor on the embedded platform. When an object with the matching features is detected, it is drawn as a rectangular bounding box and the results are displayed on the screen. In the proposed system, it first annotates the image datasets and saves them in the corresponding input data format for model training. Thus, the You Only Look Once (YOLOv5) model has been employed for training because it is a state-of-the-art object detection system. In other words, a fast and accurate emotion recognition is the main benefits of choosing YOLOv5 model. Then, the correctly trained YOLOv5 model file is loaded into an edge computing platform; and the feature values of objects are analyzed by a high computing processor. Finally, the experimental results show that the promising mean Average Precision (mAP), 92.6%, and recognition speed in Frames Per Second (FPS), 40, are obtained, which outperforms other existing systems.