SFTNet: A microexpression-based method for depression detection

• Published in: Computer methods and programs in biomedicine Vol. 243; p. 107923
• Main Authors: Li, Xingyun, Yi, Xinyu, Ye, Jiayu, Zheng, Yunshao, Wang, Qingxiang
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.01.2024
• Subjects: Depression detection; Emotional stimuli; Facial expressions; Microexpression; Depression detection; Emotional stimuli; Facial expressions; Microexpression
• ISSN: 0169-2607; 1872-7565; 1872-7565
• DOI: 10.1016/j.cmpb.2023.107923

Depression is a typical mental illness, and early screening can effectively prevent exacerbation of the condition. Many studies have found that the expressions of depressed patients are different from those of other subjects, and microexpressions have been used in the clinical detection of mental illness. However, there are few methods for the automatic detection of depression based on microexpressions. A new dataset of 156 participants (76 in the case group and 80 in the control group) was created. All data were collected in the context of a new emotional stimulation experiment and doctor-patient conversation. We first analyzed the Average Number of Occurrences (ANO) and Average Duration (AD) of facial expressions in the case group and the control group. Then, we proposed a two-stream model SFTNet for identifying depression based on microexpressions, which consists of a single-temporal network (STNet) and a full-temporal network (FTNet). STNet is used to extract features from facial images at a single time node, FTNet is used to extract features from all-time nodes, and the decision network combines the two features to identify depression through decision fusion. The code for SFTNet is available at https://github.com/muzixingyun/SFTNet. We found that the AD of all subjects was less than 20 frames (2/3 seconds) and that the facial expressions of the control group were richer. SFTNet achieved excellent results on the emotional stimulus experimental dataset, with Accuracy, Precision and Recall of 0.873, 0.888 and 0.846, respectively. We also conducted experiments on the doctor-patient conversation dataset, and the Accuracy, Precision and Recall were 0.829, 0.817 and 0.837, respectively. SFTNet can also be applied to microexpression detection task with more accuracy than SOTA models. In the emotional stimulation experiment, the subjects in the case group are more likely to show negative emotions. Compared to SOTA models, our depression detection method is more accurate and can assist doctors in the diagnosis of depression. •A new experiment of emotional stimuli for expression evoking is proposed.•The Average Number of Occurrences (ANO) and the Average Duration (AD) of the subjects' seven basic expressions were analyzed.•A new microexpression-based depression recognition model SFTNet is proposed.