Depression and post traumatic stress disorder analysis with multi-modal data

• Published in: IAES international journal of artificial intelligence Vol. 14; no. 3; p. 2358
• Main Authors: Sivanaiah, Rajalakshmi, Suseelan, Angel Deborah, Thannickal, Krupa Elizabeth, Pirabahar, Sanmati
• Format: Journal Article
• Language: English
• Published: 01.06.2025
• ISSN: 2089-4872; 2252-8938
• DOI: 10.11591/ijai.v14.i3.pp2358-2366

With an increasing global population and more people living to the age when major depressive disorder (MDD) or post traumatic stress disorder (PTSD) commonly occurs, the number of those who suffer from such disorders is rising. Studies have also shown a high likelihood of comorbidity between these 2 disorders. This comorbidity can worsen symptoms, increase the risk of chronicity, and complicate treatment, significantly impacting patients’ emotional wellbeing and social and occupational functioning. There is a need to enable faster and reliable diagnosis methods, while taking into account the subjectivity of individuals and the role of behavioural cues. The proposed approach analyses the combination of audio, video and text input features (multi-modal data) of the subject to determine the severity class of MDD and PTSD. The DistilBERT transformer is used for learning and building a model with the textual modality and random forest classifiers for the audio and video modalities. An ensemble of these 3 models from 3 modalities performs better in the final classification of MDD and PTSD when compared to individual models. This work also covers a comparison of the models with different splits on the dataset. This ensembled system shows an improved accuracy of 2% to 7% for the MDD and PTSD multi class classification over the models tested on individual modalities.