What Can Facial Movements Reveal? Depression Recognition and Analysis based on Optical Flow using Bayesian Networks

• Published in: IEEE transactions on neural systems and rehabilitation engineering Vol. 31; p. 1
• Main Authors: Ma, Yu, Shen, Jian, Zhao, Zeguang, Liang, Huajian, Tan, Yang, Liu, Zhenyu, Qian, Kun, Yang, Minqiang, Hu, Bin
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.01.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Action Units; Bayes methods; Bayesian analysis; Bayesian Networks; Construction; Correlation coefficient; Correlation coefficients; Data collection; Depression; Depression Recognition; Discrete Wavelet Transform; Discrete wavelet transforms; Face recognition; Facial Expressions; Feature extraction; Feature recognition; Medical services; Mental depression; Networks; Optical Flow; Optical flow (image analysis); Pattern recognition; Stimuli; Wavelet analysis
• ISSN: 1534-4320; 1558-0210; 1558-0210
• DOI: 10.1109/TNSRE.2023.3305351

Recent evidence have demonstrated that facial expressions could be a valid and important aspect for depression recognition. Although various works have been achieved in automatic depression recognition, it is a challenge to explore the inherent nuances of facial expressions that might reveal the underlying differences between depressed patients and healthy subjects under different stimuli. There is a lack of an undisturbed system that monitors depressive patients' mental states in various free-living scenarios, so this paper steps towards building a classification model where data collection, feature extraction, depression recognition and facial actions analysis are conducted to infer the differences of facial movements between depressive patients and healthy subjects. In this study, we firstly present a plan of dividing facial regions of interest to extract optical flow features of facial expressions for depression recognition. We then propose facial movements coefficients utilising discrete wavelet transformation. Specifically, Bayesian Networks equipped with construction of Pearson Correlation Coefficients based on discrete wavelet transformation is learnt, which allows for analysing movements of different facial regions. We evaluate our method on a clinically validated dataset of 30 depressed patients and 30 healthy control subjects, and experiments results obtained the accuracy and recall of 81.7 %, 96.7 %, respectively, outperforming other features for comparison. Most importantly, the Bayesian Networks we built on the coefficients under different stimuli may reveal some facial action patterns of depressed subjects, which have a potential to assist the automatic diagnosis of depression.