Medical Image Synthesis with Deep Convolutional Adversarial Networks

• Published in: IEEE transactions on biomedical engineering Vol. 65; no. 12; pp. 2720 - 2730
• Main Authors: Nie, Dong, Trullo, Roger, Lian, Jun, Wang, Li, Petitjean, Caroline, Ruan, Su, Wang, Qian, Shen, Dinggang
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.12.2018; The Institute of Electrical and Electronics Engineers, Inc. (IEEE); Institute of Electrical and Electronics Engineers
• Subjects: Adversarial learning; Artificial neural networks; auto-context model; Biomedical imaging; Brain - diagnostic imaging; Computed tomography; Computer Science; Datasets; Deep Learning; Generators; Humans; Image acquisition; Image generation; Image Interpretation, Computer-Assisted - methods; image synthesis; Magnetic Resonance Imaging; Medical imaging; Pelvis - diagnostic imaging; Radiation dosage; residual learning; Signal and Image Processing; State of the art; Synthesis; Target recognition; Task analysis; Therapeutic applications; Tomography, X-Ray Computed
• ISSN: 0018-9294; 1558-2531; 1558-2531
• DOI: 10.1109/TBME.2018.2814538

Medical imaging plays a critical role in various clinical applications. However, due to multiple considerations such as cost and radiation dose, the acquisition of certain image modalities may be limited. Thus, medical image synthesis can be of great benefit by estimating a desired imaging modality without incurring an actual scan. In this paper, we propose a generative adversarial approach to address this challenging problem. Specifically, we train a fully convolutional network (FCN) to generate a target image given a source image. To better model a nonlinear mapping from source to target and to produce more realistic target images, we propose to use the adversarial learning strategy to better model the FCN. Moreover, the FCN is designed to incorporate an image-gradient-difference-based loss function to avoid generating blurry target images. Long-term residual unit is also explored to help the training of the network. We further apply Auto-Context Model to implement a context-aware deep convolutional adversarial network. Experimental results show that our method is accurate and robust for synthesizing target images from the corresponding source images. In particular, we evaluate our method on three datasets, to address the tasks of generating CT from MRI and generating 7T MRI from 3T MRI images. Our method outperforms the state-of-the-art methods under comparison in all datasets and tasks.