Predicting patient outcomes in psychiatric hospitals with routine data: a machine learning approach

• Published in: BMC medical informatics and decision making Vol. 20; no. 1; pp. 21 - 9
• Main Authors: Wolff, J, Gary, A, Jung, D, Normann, C, Kaier, K, Binder, H, Domschke, K, Klimke, A, Franz, M
• Format: Journal Article
• Language: English
• Published: England BioMed Central Ltd 06.02.2020; BioMed Central; BMC
• Subjects: Adult; Aged; Calendars; Cancer; Child & adolescent psychiatry; Classifiers; Clinical decision making; Coercivity; Confidence intervals; Databases, Factual; Decision making; Decision Support Techniques; Diabetic retinopathy; Diagnostic systems; Electronic Health Records; Female; Germany; Health services administration; Hospitalization; Hospitals; Hospitals, Psychiatric; Humans; Information management; Learning algorithms; Logistic Models; Machine Learning; Male; Management; Medical research; Mental disorders; Mental health records; Mental health services; Middle Aged; Outcome and process assessment (Medical care); Patients; Performance prediction; Prognosis; Psychiatric hospitals; Psychiatry; ROC Curve; Schizophrenia; Standard deviation; Stochasticity; Variables; Germany; Health services administration; Decision support techniques; Hospitals; Psychiatry; Machine learning
• ISSN: 1472-6947; 1472-6947
• DOI: 10.1186/s12911-020-1042-2

A common problem in machine learning applications is availability of data at the point of decision making. The aim of the present study was to use routine data readily available at admission to predict aspects relevant to the organization of psychiatric hospital care. A further aim was to compare the results of a machine learning approach with those obtained through a traditional method and those obtained through a naive baseline classifier. The study included consecutively discharged patients between 1st of January 2017 and 31st of December 2018 from nine psychiatric hospitals in Hesse, Germany. We compared the predictive performance achieved by stochastic gradient boosting (GBM) with multiple logistic regression and a naive baseline classifier. We tested the performance of our final models on unseen patients from another calendar year and from different hospitals. The study included 45,388 inpatient episodes. The models' performance, as measured by the area under the Receiver Operating Characteristic curve, varied strongly between the predicted outcomes, with relatively high performance in the prediction of coercive treatment (area under the curve: 0.83) and 1:1 observations (0.80) and relatively poor performance in the prediction of short length of stay (0.69) and non-response to treatment (0.65). The GBM performed slightly better than logistic regression. Both approaches were substantially better than a naive prediction based solely on basic diagnostic grouping. The present study has shown that administrative routine data can be used to predict aspects relevant to the organisation of psychiatric hospital care. Future research should investigate the predictive performance that is necessary to provide effective assistance in clinical practice for the benefit of both staff and patients.