Propagating Uncertainty Across Cascaded Medical Imaging Tasks for Improved Deep Learning Inference

• Published in: IEEE transactions on medical imaging Vol. 41; no. 2; pp. 360 - 373
• Main Authors: Mehta, Raghav, Christinck, Thomas, Nair, Tanya, Bussy, Aurelie, Premasiri, Swapna, Costantino, Manuela, Chakravarthy, M. Mallar, Arnold, Douglas L., Gal, Yarin, Arbel, Tal
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.02.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Alzheimer's disease; Alzheimer’s; Bayesian deep learning; Biomedical imaging; Brain; Brain cancer; Brain Neoplasms - diagnostic imaging; Brain tumors; brain tumour; classification; Datasets; Deep Learning; detection; Embedding; Humans; Image Processing, Computer-Assisted - methods; Image segmentation; Inference; Lesions; Magnetic Resonance Imaging; Medical imaging; Multiple sclerosis; Neurodegenerative diseases; Neuroimaging; Performance evaluation; segmentation; synthesis; Task analysis; Tumors; Uncertainty
• ISSN: 0278-0062; 1558-254X
• DOI: 10.1109/TMI.2021.3114097

Although deep networks have been shown to perform very well on a variety of medical imaging tasks, inference in the presence of pathology presents several challenges to common models. These challenges impede the integration of deep learning models into real clinical workflows, where the customary process of cascading deterministic outputs from a sequence of image-based inference steps (e.g. registration, segmentation) generally leads to an accumulation of errors that impacts the accuracy of downstream inference tasks. In this paper, we propose that by embedding uncertainty estimates across cascaded inference tasks, performance on the downstream inference tasks should be improved. We demonstrate the effectiveness of the proposed approach in three different clinical contexts: (i) We demonstrate that by propagating T2 weighted lesion segmentation results and their associated uncertainties, subsequent T2 lesion detection performance is improved when evaluated on a proprietary large-scale, multi-site, clinical trial dataset acquired from patients with Multiple Sclerosis. (ii) We show an improvement in brain tumour segmentation performance when the uncertainty map associated with a synthesised missing MR volume is provided as an additional input to a follow-up brain tumour segmentation network, when evaluated on the publicly available BraTS-2018 dataset. (iii) We show that by propagating uncertainties from a voxel-level hippocampus segmentation task, the subsequent regression of the Alzheimer's disease clinical score is improved.