Using speech recognition technology to investigate the association between timing-related speech features and depression severity

• Published in: PloS one Vol. 15; no. 9; p. e0238726
• Main Authors: Yamamoto, Mao, Takamiya, Akihiro, Sawada, Kyosuke, Yoshimura, Michitaka, Kitazawa, Momoko, Liang, Kuo-ching, Fujita, Takanori, Mimura, Masaru, Kishimoto, Taishiro
• Format: Journal Article
• Language: English
• Published: United States Public Library of Science 11.09.2020; Public Library of Science (PLoS)
• Subjects: Artificial Intelligence; Biology and Life Sciences; Biomarkers; Bipolar disorder; Bipolar Disorder - physiopathology; Case-Control Studies; Depression (Mood disorder); Depressive Disorder, Major - physiopathology; Engineering and Technology; Feature recognition; Female; Health aspects; Hospitals; Humans; Identification and classification; Intellectual disabilities; Interviews; Male; Medicine; Medicine and Health Sciences; Mental depression; Mental disorders; Middle Aged; Patients; Psychological aspects; Regression analysis; Response time; Signal processing; Social Sciences; Speech; Speech recognition; Speech recognition software; Technology; Technology utilization; Time Factors; Voice recognition; Japan
• ISSN: 1932-6203; 1932-6203
• DOI: 10.1371/journal.pone.0238726

There are no reliable and validated objective biomarkers for the assessment of depression severity. We aimed to investigate the association between depression severity and timing-related speech features using speech recognition technology. Patients with major depressive disorder (MDD), those with bipolar disorder (BP), and healthy controls (HC) were asked to engage in a non-structured interview with research psychologists. Using automated speech recognition technology, we measured three timing-related speech features: speech rate, pause time, and response time. The severity of depression was assessed using the Hamilton Depression Rating Scale 17-item version (HAMD-17). We conducted the current study to answer the following questions: 1) Are there differences in speech features among MDD, BP, and HC? 2) Do speech features correlate with depression severity? 3) Do changes in speech features correlate with within-subject changes in depression severity? We collected 1058 data sets from 241 individuals for the study (97 MDD, 68 BP, and 76 HC). There were significant differences in speech features among groups; depressed patients showed slower speech rate, longer pause time, and longer response time than HC. All timing-related speech features showed significant associations with HAMD-17 total scores. Longitudinal changes in speech rate correlated with changes in HAMD-17 total scores. Depressed individuals showed longer response time, longer pause time, and slower speech rate than healthy individuals, all of which were suggestive of psychomotor retardation. Our study suggests that speech features could be used as objective biomarkers for the assessment of depression severity.