Confidence-Calibrated Clinical Decision Support System for Reliable Respiratory Disease Screening

• Published in: IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI ...) (Online) pp. 1 - 8
• Main Authors: Nnamdi, Micky C., Tamo, J. Ben, Shi, Wenqi, Chundru, Vivek, Marteau, Benoit, Patil, Oankar, Wang, May D.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 10.11.2024
• Subjects: Accuracy; Artificial neural networks; audio signal analysis; Calibration; clinical decision support; Computer network reliability; confidence calibration; Decision support systems; Medical services; Predictive models; Pulmonary diseases; Reliability; respiratory disease; Uncertainty
• ISSN: 2641-3604
• DOI: 10.1109/BHI62660.2024.10913797

With the growing adoption of computer-aided diagnostic and treatment recommendation systems in healthcare, it is essential to ensure both the accuracy and reliability of AI-enabled clinical decision support systems. In this study, we comprehensively examine existing model confidence calibration methods and propose an ensemble-based calibration approach for reliable predictions in clinical decision support systems (CDSSs). Specifically, we introduce an ENsemble-based Confidence-caLibrated deep neural network, ENCL-DNN, to improve respiratory disease screening using cough sounds. We also leverage local interpretable model-agnostic explanations to monitor the behavior of the CDSS, identifying the key features that contribute to its predictions and ensuring transparency in the diagnosis. By employing the ensemble-based calibration method, ENCL-DNN demonstrates superior performance on two publicly available respiratory audio datasets, Coswara and Cambridge, as evidenced by a 50% and a 28.74% reduction in Expected Calibration Error (ECE), respectively, compared to the uncalibrated baselines. Our experiments highlight the significance of well-calibrated deep neural networks in respiratory disease screening and the enhancement of reliability in mobile healthcare systems. By providing reliable and transparent predictions, ENCL-DNN has the potential to promote the wide adoption of AI-driven CDSSs and thereby improve patient outcomes through early diagnosis and intervention.