Facial expression recognition algorithm based on parameter adaptive initialization of CNN and LSTM

• Published in: The Visual computer Vol. 36; no. 3; pp. 483 - 498
• Main Authors: An, Fengping, Liu, Zhiwen
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.03.2020; Springer Nature B.V
• Subjects: Accuracy; Algorithms; Artificial Intelligence; Artificial neural networks; Classification; Computer Graphics; Computer Science; Convex analysis; Deep learning; Emotions; Face recognition; Feature extraction; Image Processing and Computer Vision; Intelligence; Machine learning; Mathematical models; Multilayers; Neural networks; Original Article; Parameters; Support vector machines; Deep learning; Facial expression recognition; CNN; LSTM; SVM; Model parameter initialization method
• ISSN: 0178-2789; 1432-2315
• DOI: 10.1007/s00371-019-01635-4

In view of the high dimensionality, nonrigidity, multiscale variation and the influence of illumination and angle on facial expressions, it is quite difficult to obtain facial expression images or videos using computers and analyze facial morphology and changes to accurately obtain the emotional changes of the subjects. Existing facial expression recognition algorithms have the following problems in the application process: the existing shallow feature extraction model has lost a lot of effective feature information and low recognition accuracy. The facial expression recognition method based on deep learning has problems such as overfitting, gradient explosion and parameter initialization. Therefore, this paper develops a facial expression recognition algorithm based on the deep learning method. An adaptive model parameter initialization based on the multilayer maxout network linear activation function is proposed to initialize the convolutional neural network (CNN) and the long–short-term memory network (LSTM) method. It can effectively overcome the gradient disappearance and gradient explosion problems in the deep learning model training process. At the same time, the convolutional neural network with an LSTM memory unit is used to extract the related information from the image sequence, and the facial expression judgment is based on a single-frame image and historical-related information. However, the top-level structure of the CNN model is a fully connected feedforward neural network, which undertakes the task of expression classification. Therefore, the SVM classification method replaces the top-level classifier to further improve the expression classification accuracy. Experiments show that the facial expression recognition method proposed in this paper not only accurately identifies various expressions but also has good adaptive ability. This is because the method achieves the adaptive initialization of the parameters of the deep learning model construction process and also analyzes the relevance of the expression database expression, thereby improving the accuracy of expression recognition.