A 3D multi-scale CycleGAN framework for generating synthetic PETs from MRIs for Alzheimer's disease diagnosis

• Published in: Image and vision computing Vol. 146; p. 105017
• Main Authors: Khojaste-Sarakhsi, M., Haghighi, Seyedhamidreza Shahabi, Ghomi, S.M.T. Fatemi, Marchiori, Elena
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.06.2024
• Subjects: 3D image-to-image translation; Alzheimer's Disease diagnosis; Cycle GAN; Image synthesis; Multi-scale GAN; Cycle GAN; 3D image-to-image translation; Multi-scale GAN; Image synthesis; Alzheimer's Disease diagnosis
• ISSN: 0262-8856; 1872-8138
• DOI: 10.1016/j.imavis.2024.105017

This paper proposes a novel framework for generating synthesized PET images from MRIs to fill in missing PETs and help with Alzheimer's disease (AD) diagnosis. This framework employs a 3D multi-scale image-to-image CycleGAN architecture for the end-to-end translation of MRI and PET domains together. A hybrid loss function is also proposed to enforce structural similarity while preserving voxel-wise similarity and avoiding blurry images. As shown by the quantitative and visual assessment of the synthesized PETs, this framework is superior to the state-of-the-art. Moreover, using these synthesized PETs helps improve the ternary classification of AD subjects (AD vs. MCI vs. NC). Specifically, assuming an extreme case where none of the subjects has a PET, feeding the classifier with MRIs and their corresponding synthetic PETs results in a more accurate diagnosis than feeding it with just available MRIs. Accordingly, the proposed framework can help improve AD diagnosis, which is the final goal of the current study. Ablation investigation of the proposed multi-scale framework as well as the proposed loss function, is also conducted to study their contribution to the quality of synthesized PETs. Furthermore, other factors, such as stopping criteria, the type of normalization layer, the activation function, and dropouts, are examined, concluding that the appropriate use of these factors can significantly improve the quality of synthesized PETs. •A novel 3D multi-scale CycleGAN is proposed for MRI to PET translation.•A hybrid loss is introduced to enforce structural and voxel-wise similarity.•The proposed framework outperforms state-of-the-art based on different criteria.•Using generated synthesized PETs can improve Alzheimer's disease (AD) diagnosis.•Ablation studies confirm the impact of the proposed framework and loss function.