Reflections on the nature of measurement in language-based automated assessments of patients' mental state and cognitive function

Modern advances in computational language processing methods have enabled new approaches to the measurement of mental processes. However, the field has primarily focused on model accuracy in predicting performance on a task or a diagnostic category. Instead the field should be more focused on determ...

Full description

Saved in:
Bibliographic Details
Published inSchizophrenia research Vol. 259; pp. 127 - 139
Main Authors Foltz, Peter W., Chandler, Chelsea, Diaz-Asper, Catherine, Cohen, Alex S., Rodriguez, Zachary, Holmlund, Terje B., Elvevåg, Brita
Format Journal Article
LanguageEnglish
Published Elsevier B.V 01.09.2023
Elsevier
Subjects
Online AccessGet full text

Cover

Loading…
More Information
Summary:Modern advances in computational language processing methods have enabled new approaches to the measurement of mental processes. However, the field has primarily focused on model accuracy in predicting performance on a task or a diagnostic category. Instead the field should be more focused on determining which computational analyses align best with the targeted neurocognitive/psychological functions that we want to assess. In this paper we reflect on two decades of experience with the application of language-based assessment to patients' mental state and cognitive function by addressing the questions of what we are measuring, how it should be measured and why we are measuring the phenomena. We address the questions by advocating for a principled framework for aligning computational models to the constructs being assessed and the tasks being used, as well as defining how those constructs relate to patient clinical states. We further examine the assumptions that go into the computational models and the effects that model design decisions may have on the accuracy, bias and generalizability of models for assessing clinical states. Finally, we describe how this principled approach can further the goal of transitioning language-based computational assessments to part of clinical practice while gaining the trust of critical stakeholders.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
Schizophrenia Research
ISSN:0920-9964
1573-2509
1573-2509
DOI:10.1016/j.schres.2022.07.011