Development and validation of medical record-based logistic regression and machine learning models to diagnose diabetic retinopathy

• Published in: Graefe's archive for clinical and experimental ophthalmology Vol. 261; no. 3; pp. 681 - 689
• Main Authors: Li, He-Yan, Dong, Li, Zhou, Wen-Da, Wu, Hao-Tian, Zhang, Rui-Heng, Li, Yi-Tong, Yu, Chu-Yao, Wei, Wen-Bin
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.03.2023; Springer Nature B.V
• Subjects: Creatine; Cross-Sectional Studies; Diabetes; Diabetes Mellitus; Diabetic retinopathy; Diabetic Retinopathy - diagnosis; Diabetic Retinopathy - epidemiology; Female; Humans; Learning algorithms; Logistic Models; Machine Learning; Male; Medical Records; Medicine; Medicine & Public Health; Nutrition Surveys; Ophthalmology; Regression analysis; Retinal Disorders; Retinopathy; National Health and Nutrition Examination Survey (NHANES); Diabetic retinopathy; Logistic regression; Model selection; Machine learning
• ISSN: 0721-832X; 1435-702X; 1435-702X
• DOI: 10.1007/s00417-022-05854-9

Purposes Many factors were reported to be associated with diabetic retinopathy (DR); however, their contributions remained unclear. We aimed to evaluate the prognostic and diagnostic accuracy of logistic regression and three machine learning models based on various medical records. Methods This was a cross-sectional study. We investigated the prevalence and associations of DR among 757 participants aged 40 years or older in the 2005–2006 National Health and Nutrition Examination Survey (NHANES). We trained the models to predict if the participants had DR with 15 predictor variables. Area under the receiver operating characteristic (AUROC) and mean squared error (MSE) of each algorithm were compared in the external validation dataset using a replicate cohort from NHANES 2007–2008. Results Among the 757 participants, 53 (7.00%) subjects had DR, the mean (standard deviation, SD) age was 57.7 (13.04), and 78.0% were male ( n  = 42). Logistic regression revealed that female gender (OR = 4.130, 95% CI: 1.820–9.380; P  < 0.05), HbA1c (OR = 1.665, 95% CI: 1.197–2.317; P  < 0.05), serum creatine level (OR = 2.952, 95% CI: 1.274–6.851; P  < 0.05), and eGFR level (OR = 1.009, 95% CI: 1.000–1.014, P  < 0.05) increased the risk of DR. The average performance obtained from internal validation was similar in all models (AUROC ≥ 0.945), and k-nearest neighbors (KNN) had the highest value with an AUROC of 0.984. In external validation, they remained robust or with modest reductions in discrimination with AUROC still ≥ 0.902, and KNN also performed the best with an AUROC of 0.982. Both logistic regression and machine learning models had good performance in the clinical diagnosis of DR. Conclusions This study highlights the utility of comparing traditional logistic regression to machine learning models. We found that logistic regression performed as well as optimized machine learning methods when classifying DR patients.