Estimating time-to-total knee replacement on radiographs and MRI: a multimodal approach using self-supervised deep learning

• Published in: Radiology advances Vol. 1; no. 4; p. umae030
• Main Authors: Cigdem, Ozkan, Chen, Shengjia, Zhang, Chaojie, Cho, Kyunghyun, Kijowski, Richard, Deniz, Cem M
• Format: Journal Article
• Language: English
• Published: England Oxford University Press 01.01.2022
• Subjects: deep learning; self-supervised learning; multimodal learning; knee osteoarthritis; time-to-TKR prediction
• ISSN: 2976-9337
• DOI: 10.1093/radadv/umae030

Purpose Accurately predicting the expected duration of time until total knee replacement (time-to-TKR) is crucial for patient management and health care planning. Predicting when surgery may be needed, especially within shorter windows like 3 years, allows clinicians to plan timely interventions and health care systems to allocate resources more effectively. Existing models lack the precision for such time-based predictions. A survival analysis model for predicting time-to-TKR was developed using features from medical images and clinical measurements. Methods From the Osteoarthritis Initiative dataset, all knees with clinical variables, MRI scans, radiographs, and quantitative and semiquantitative assessments from images were identified. This resulted in 895 knees that underwent TKR within the 9-year follow-up period, as specified by the Osteoarthritis Initiative study design, and 786 control knees that did not undergo TKR (right-censored, indicating their status beyond the 9-year follow-up is unknown). These knees were used for model training and testing. Additionally, 518 and 164 subjects from the Multi-Center Osteoarthritis Study and internal hospital data were used for external testing, respectively. Deep learning models were utilized to extract features from radiographs and MR scans. Extracted features, clinical variables, and image assessments were used in survival analysis with Lasso Cox feature selection and a random survival forest model to predict time-to-TKR. Results The proposed model exhibited strong discrimination power by integrating self-supervised deep learning features with clinical variables (eg, age, body mass index, pain score) and image assessment measurements (eg, Kellgren-Lawrence grade, joint space narrowing, bone marrow lesion size, cartilage morphology) from multiple modalities. The model achieved an area under the curve of 94.5 (95% CI, 94.0-95.1) for predicting the time-to-TKR. Conclusions The proposed model demonstrated the potential of self-supervised learning and multimodal data fusion in accurately predicting time-to-TKR that may assist physicians to develop personalize treatment strategies.