Automated model versus treating physician for predicting survival time of patients with metastatic cancer

• Published in: Journal of the American Medical Informatics Association : JAMIA Vol. 28; no. 6; pp. 1108 - 1116
• Main Authors: Gensheimer, Michael F, Aggarwal, Sonya, Benson, Kathryn R.K, Carter, Justin N, Henry, A. Solomon, Wood, Douglas J, Soltys, Scott G, Hancock, Steven, Pollom, Erqi, Shah, Nigam H, Chang, Daniel T
• Format: Journal Article
• Language: English
• Published: England Oxford University Press 12.06.2021
• Subjects: Aged; Area Under Curve; Electronic Health Records; Female; Humans; Kaplan-Meier Estimate; Life Expectancy; Machine Learning; Male; Middle Aged; Neoplasm Metastasis - radiotherapy; Neoplasms - mortality; Prognosis; Radiation Oncologists; Research and Applications; ROC Curve; radiotherapy; natural language processing; neoplasms; machine learning; prognosis
• ISSN: 1527-974X; 1067-5027; 1527-974X
• DOI: 10.1093/jamia/ocaa290

Abstract Objective Being able to predict a patient’s life expectancy can help doctors and patients prioritize treatments and supportive care. For predicting life expectancy, physicians have been shown to outperform traditional models that use only a few predictor variables. It is possible that a machine learning model that uses many predictor variables and diverse data sources from the electronic medical record can improve on physicians’ performance. For patients with metastatic cancer, we compared accuracy of life expectancy predictions by the treating physician, a machine learning model, and a traditional model. Materials and Methods A machine learning model was trained using 14 600 metastatic cancer patients’ data to predict each patient’s distribution of survival time. Data sources included note text, laboratory values, and vital signs. From 2015–2016, 899 patients receiving radiotherapy for metastatic cancer were enrolled in a study in which their radiation oncologist estimated life expectancy. Survival predictions were also made by the machine learning model and a traditional model using only performance status. Performance was assessed with area under the curve for 1-year survival and calibration plots. Results The radiotherapy study included 1190 treatment courses in 899 patients. A total of 879 treatment courses in 685 patients were included in this analysis. Median overall survival was 11.7 months. Physicians, machine learning model, and traditional model had area under the curve for 1-year survival of 0.72 (95% CI 0.63–0.81), 0.77 (0.73–0.81), and 0.68 (0.65–0.71), respectively. Conclusions The machine learning model’s predictions were more accurate than those of the treating physician or a traditional model.