Automating the Classification of Complexity of Medical Decision-Making in Patient-Provider Messaging in a Patient Portal

• Published in: The Journal of surgical research Vol. 255; pp. 224 - 232
• Main Authors: Sulieman, Lina, Robinson, Jamie R., Jackson, Gretchen P.
• Format: Journal Article
• Language: English
• Published: United States Elsevier Inc 01.11.2020
• Subjects: Clinical Decision-Making; Consumer health; Decision complexity; Informatics; Machine Learning; Medical billing; Medical decision; Patient Portals; Consumer health; Decision complexity; Patient portals; Medical billing; Medical decision; Machine learning
• ISSN: 0022-4804; 1095-8673
• DOI: 10.1016/j.jss.2020.05.039

Patient portals are consumer health applications that allow patients to view their health information. Portals facilitate the interactions between patients and their caregivers by offering secure messaging. Patients communicate different needs through portal messages. Medical needs contain requests for delivery of care (e.g. reporting new symptoms). Automating the classification of medical decision complexity in portal messages has not been investigated. We trained two multiclass classifiers, multinomial Naïve Bayes and random forest on 500 message threads, to quantify and label the complexity of decision-making into four classes: no decision, straightforward, low, and moderate. We compared the performance of the models to using only the number of medical terms without training a machine learning model. Our analysis demonstrated that machine learning models have better performance than the model that did not use machine learning. Moreover, machine learning models could quantify the complexity of decision-making that the messages contained with 0.59, 0.45, and 0.58 for macro, micro, and weighted precision and 0.63,0.41, and 0.63 for macro, micro, and weighted recall. This study is one of the first to attempt to classify patient portal messages by whether they involve medical decision-making and the complexity of that decision-making. Machine learning classifiers trained on message content resulted in better message thread classification than classifiers that employed medical terms in the messages alone.