Development and validation of an artificial intelligence mobile application for predicting 30-day mortality in critically ill patients with orthopaedic trauma

• Published in: International journal of medical informatics (Shannon, Ireland) Vol. 184; p. 105383
• Main Authors: Han, Tao, Xiong, Fan, Sun, Baisheng, Zhong, Lixia, Han, Zhencan, Lei, Mingxing
• Format: Journal Article
• Language: English
• Published: Ireland Elsevier B.V 01.04.2024
• Subjects: Artificial Intelligence; Critical fracture; Critical Illness; Early death; Humans; Machine learning; Mobile Applications; Neural Networks, Computer; Orthopedics; Web-based model; Early death; Critical fracture; Web-based model; Artificial intelligence; Machine learning
• ISSN: 1386-5056; 1872-8243
• DOI: 10.1016/j.ijmedinf.2024.105383

•Development and validation of an AI model for predicting 30-day mortality in critically ill orthopaedic trauma patients.•eXGBM algorithm demonstrated highest predictive performance, outperforming other models.•Model deployed as web-based AI application for easy access by healthcare professionals.•Successful external validation with AUC value of 0.913.•Model aids in identification and management of high-risk patients in ICU settings. Given the intricate and grave nature of trauma-related injuries in ICU settings, it is imperative to develop and deploy reliable predictive tools that can aid in the early identification of high-risk patients who are at risk of early death. The objective of this study is to create and validate an artificial intelligence (AI) model that can accurately predict early mortality among critical fracture patients. A total of 2662 critically ill patients with orthopaedic trauma were included from the MIMIC III database. Early mortality was defined as death within 30 days in this study. The patients were randomly divided into a model training cohort and a model validation cohort. Various algorithms, including logistic regression (LR), extreme gradient boosting machine (eXGBM), decision tree (DT), support vector machine (SVM), random forest (RF), and neural network (NN), were employed. Evaluation metrics, including discrimination and calibration, were used to develop a comprehensive scoring system ranging from 0 to 60, with higher scores indicating better prediction performance. Furthermore, external validation was carried out using 131 patients. The optimal model was deployed as an internet-based AI tool. Among all models, the eXGBM demonstrated the highest area under the curve (AUC) value (0.974, 95%CI: 0.959-0.983), followed by the RF model (0.951, 95%CI: 0.935-0.967) and the NN model (0.922, 95%CI: 0.905-0.941). Additionally, the eXGBM model outperformed other models in terms of accuracy (0.915), precision (0.906), recall (0.926), F1 score (0.916), Brier score (0.062), log loss (0.210), and discrimination slope (0.767). Based on the scoring system, the eXGBM model achieved the highest score (53), followed by RF (42) and NN (39). The LR, DT, and SVM models obtained scores of 28, 18, and 32, respectively. Decision curve analysis further confirmed the superior clinical net benefits of the eXGBM model. External validation of the model achieved an AUC value of 0.913 (95%CI: 0.878-0.948). Consequently, the model was deployed on the Internet at https://30-daymortalityincriticallyillpatients-fnfsynbpbp6rgineaspuim.streamlit.app/, allowing users to input patient features and obtain predicted risks of early mortality among critical fracture patients. Furthermore, the AI model successfully stratified patients into low or high risk of early mortality based on a predefined threshold and provided recommendations for appropriate therapeutic interventions. This study successfully develops and validates an AI model, with the eXGBM algorithm demonstrating the highest predictive performance for early mortality in critical fracture patients. By deploying the model as a web-based AI application, healthcare professionals can easily access the tool, enabling them to predict 30-day mortality and aiding in the identification and management of high-risk patients among those critically ill with orthopedic trauma.