Combining physics‐based models with deep learning image synthesis and uncertainty in intraoperative cone‐beam CT of the brain

• Published in: Medical physics (Lancaster) Vol. 50; no. 5; pp. 2607 - 2624
• Main Authors: Zhang, Xiaoxuan, Sisniega, Alejandro, Zbijewski, Wojciech B., Lee, Junghoon, Jones, Craig K., Wu, Pengwei, Han, Runze, Uneri, Ali, Vagdargi, Prasad, Helm, Patrick A., Luciano, Mark, Anderson, William S., Siewerdsen, Jeffrey H.
• Format: Journal Article
• Language: English
• Published: United States 01.05.2023
• Subjects: 3D image reconstruction; aleatoric uncertainty; Algorithms; Brain - diagnostic imaging; Brain - surgery; Cone-Beam Computed Tomography - methods; cone‐beam CT; Deep Learning; epistemic uncertainty; generative adversarial network; Humans; Image Processing, Computer-Assisted - methods; image quality; image synthesis; image‐guided surgery; Pilot Projects; Uncertainty; 3D image reconstruction; generative adversarial network; image-guided surgery; deep learning; image quality; epistemic uncertainty; image synthesis; aleatoric uncertainty; cone-beam CT
• ISSN: 0094-2405; 2473-4209; 2473-4209
• DOI: 10.1002/mp.16351

Background Image‐guided neurosurgery requires high localization and registration accuracy to enable effective treatment and avoid complications. However, accurate neuronavigation based on preoperative magnetic resonance (MR) or computed tomography (CT) images is challenged by brain deformation occurring during the surgical intervention. Purpose To facilitate intraoperative visualization of brain tissues and deformable registration with preoperative images, a 3D deep learning (DL) reconstruction framework (termed DL‐Recon) was proposed for improved intraoperative cone‐beam CT (CBCT) image quality. Methods The DL‐Recon framework combines physics‐based models with deep learning CT synthesis and leverages uncertainty information to promote robustness to unseen features. A 3D generative adversarial network (GAN) with a conditional loss function modulated by aleatoric uncertainty was developed for CBCT‐to‐CT synthesis. Epistemic uncertainty of the synthesis model was estimated via Monte Carlo (MC) dropout. Using spatially varying weights derived from epistemic uncertainty, the DL‐Recon image combines the synthetic CT with an artifact‐corrected filtered back‐projection (FBP) reconstruction. In regions of high epistemic uncertainty, DL‐Recon includes greater contribution from the FBP image. Twenty paired real CT and simulated CBCT images of the head were used for network training and validation, and experiments evaluated the performance of DL‐Recon on CBCT images containing simulated and real brain lesions not present in the training data. Performance among learning‐ and physics‐based methods was quantified in terms of structural similarity (SSIM) of the resulting image to diagnostic CT and Dice similarity metric (DSC) in lesion segmentation compared to ground truth. A pilot study was conducted involving seven subjects with CBCT images acquired during neurosurgery to assess the feasibility of DL‐Recon in clinical data. Results CBCT images reconstructed via FBP with physics‐based corrections exhibited the usual challenges to soft‐tissue contrast resolution due to image non‐uniformity, noise, and residual artifacts. GAN synthesis improved image uniformity and soft‐tissue visibility but was subject to error in the shape and contrast of simulated lesions that were unseen in training. Incorporation of aleatoric uncertainty in synthesis loss improved estimation of epistemic uncertainty, with variable brain structures and unseen lesions exhibiting higher epistemic uncertainty. The DL‐Recon approach mitigated synthesis errors while maintaining improvement in image quality, yielding 15%–22% increase in SSIM (image appearance compared to diagnostic CT) and up to 25% increase in DSC in lesion segmentation compared to FBP. Clear gains in visual image quality were also observed in real brain lesions and in clinical CBCT images. Conclusions DL‐Recon leveraged uncertainty estimation to combine the strengths of DL and physics‐based reconstruction and demonstrated substantial improvements in the accuracy and quality of intraoperative CBCT. The improved soft‐tissue contrast resolution could facilitate visualization of brain structures and support deformable registration with preoperative images, further extending the utility of intraoperative CBCT in image‐guided neurosurgery.