Multimodal Approach for Detecting Depression Using Physiological and Behavioural Data

• Published in: 2023 3rd International Conference on Pervasive Computing and Social Networking (ICPCSN) pp. 53 - 65
• Main Authors: Kokkera, Ankita, Varsha, Narra, Vasanth, A.Vijay
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.06.2023
• Subjects: Computational modeling; Depression; Gating; Long Short-Term Memory; Mental health; Modalities; Neural Network; Random Forest; Road transportation; Social networking (online); Support Vector Machine; Support vector machines; Time-frequency analysis
• DOI: 10.1109/ICPCSN58827.2023.00016

Depression is a widespread mental illness that can have a significant impact on an individual's overall well-being. Therefore, the use of depression detection systems has become increasingly crucial in identifying and assessing the presence and severity of depression in various settings, such as primary care offices, mental health clinics, and online platforms. In the past, depression detection was typically performed through clinical interviews, where psychologists would evaluate the subject's responses to determine their mental state. In this study, a model has been developed that combines three modalities - text, audio, and video - to predict the patient's mental health status. The output is classified into different levels to consider the subject's level of depression. This fusion approach is designed to address several issues, such as handling noise in one modality and controlling the contribution level of a specific modality, to provide a more accurate and comprehensive assessment of a patient's mental state.