Depression Level Prediction Using Deep Spatiotemporal Features and Multilayer Bi-LTSM

• Published in: IEEE transactions on affective computing Vol. 13; no. 2; pp. 864 - 870
• Main Authors: Uddin, Md Azher, Joolee, Joolekha Bibi, Lee, Young-Koo
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.04.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; bidirectional long short-term memory; deep spatiotemporal network; Depression; Dynamics; Feature extraction; Feature maps; Histograms; Inception-ResNet-v2 network; Mental depression; Multilayers; Nonhomogeneous media; Optical imaging; Spatial data; Spatiotemporal phenomena; temporal median pooling; Video data; volume local directional number
• ISSN: 1949-3045; 1949-3045
• DOI: 10.1109/TAFFC.2020.2970418

Depression is a serious psychiatric disorder that restricts an individuals ability to work properly in both their daily and professional lives. Usually, the diagnosis of depression often needs a thorough assessment by an expert. Recently, significant consideration has been given to automatic depression prediction for more reliable and efficient depression investigation. In this article, we propose a novel framework to estimate the depression level from video data by employing a two-stream deep spatiotemporal network. Our approach extracts spatial information using the Inception-ResNet-v2 network. In contrast, we introduce a volume local directional number (VLDN) based dynamic feature descriptor to capture facial motions. Then, the feature map obtained from the VLDN is fed into a convolutional neural network (CNN) to obtain more discriminative features. Additionally, we designed a multilayer bidirectional long short-term memory (Bi-LSTM) model to obtain temporal information by integrating the temporal median pooling (TMP) approach into the model. The TMP approach is employed on the temporal fragments of spatial and temporal features. Finally, extensive experimental analysis of two challenging datasets, AVEC2013 and AVEC2014, demonstrates that the proposed approach shows promising performance compared to the existing approaches for depression level prediction.