GP-GAN: Brain tumor growth prediction using stacked 3D generative adversarial networks from longitudinal MR Images

• Published in: Neural networks Vol. 132; pp. 321 - 332
• Main Authors: Elazab, Ahmed, Wang, Changmiao, Gardezi, Syed Jamal Safdar, Bai, Hongmin, Hu, Qingmao, Wang, Tianfu, Chang, Chunqi, Lei, Baiying
• Format: Journal Article
• Language: English
• Published: United States Elsevier Ltd 01.12.2020
• Subjects: [formula omitted] and Dice losses; Brain - diagnostic imaging; Brain Neoplasms - diagnostic imaging; Forecasting; Gliomas; Growth prediction; Humans; Image Processing, Computer-Assisted - methods; Imaging, Three-Dimensional - methods; Longitudinal MR Images; Magnetic Resonance Imaging - methods; Neural Networks, Computer; Stacked 3D generative adversarial networks; l1 and Dice losses; Growth prediction; Longitudinal MR Images; Stacked 3D generative adversarial networks; Gliomas; and Dice losses
• ISSN: 0893-6080; 1879-2782; 1879-2782
• DOI: 10.1016/j.neunet.2020.09.004

Brain tumors are one of the major common causes of cancer-related death, worldwide. Growth prediction of these tumors, particularly gliomas which are the most dominant type, can be quite useful to improve treatment planning, quantify tumor aggressiveness, and estimate patients’ survival time towards precision medicine. Studying tumor growth prediction basically requires multiple time points of single or multimodal medical images of the same patient. Recent models are based on complex mathematical formulations that basically rely on a system of partial differential equations, e.g. reaction diffusion model, to capture the diffusion and proliferation of tumor cells in the surrounding tissue. However, these models usually have small number of parameters that are insufficient to capture different patterns and other characteristics of the tumors. In addition, such models consider tumor growth independently for each subject, not being able to get benefit from possible common growth patterns existed in the whole population under study. In this paper, we propose a novel data-driven method via stacked 3D generative adversarial networks (GANs), named GP-GAN, for growth prediction of glioma. Specifically, we use stacked conditional GANs with a novel objective function that includes both l1 and Dice losses. Moreover, we use segmented feature maps to guide the generator for better generated images. Our generator is designed based on a modified 3D U-Net architecture with skip connections to combine hierarchical features and thus have a better generated image. The proposed method is trained and tested on 18 subjects with 3 time points (9 subjects from collaborative hospital and 9 subjects from BRATS 2014 dataset). Results show that our proposed GP-GAN outperforms state-of-the-art methods for glioma growth prediction and attain average Jaccard index and Dice coefficient of 78.97% and 88.26%, respectively.