Deep Learning-Based Feature Extraction from Whole-Body PET/CT Employing Maximum Intensity Projection Images: Preliminary Results of Lung Cancer Data

• Published in: Nuclear medicine and molecular imaging Vol. 57; no. 5; pp. 216 - 222
• Main Authors: Gil, Joonhyung, Choi, Hongyoon, Paeng, Jin Chul, Cheon, Gi Jeong, Kang, Keon Wook
• Format: Journal Article
• Language: English
• Published: Singapore Springer Nature Singapore 01.10.2023; Springer Nature B.V; 대한핵의학회
• Subjects: Artificial neural networks; Cardiology; Clustering; Computed tomography; Deep learning; Embedding; Feature extraction; Feature maps; Imaging; Lung cancer; Machine learning; Medical imaging; Medicine; Medicine & Public Health; Nuclear Medicine; Oncology; Original; Original Article; Orthopedics; Positron emission; Radiology; 방사선과학; Deep learning; FDG; Maximum intensity projection; Convolutional neural network; PET/CT
• ISSN: 1869-3474; 1869-3482
• DOI: 10.1007/s13139-023-00802-9

Purpose Deep learning (DL) has been widely used in various medical imaging analyses. Because of the difficulty in processing volume data, it is difficult to train a DL model as an end-to-end approach using PET volume as an input for various purposes including diagnostic classification. We suggest an approach employing two maximum intensity projection (MIP) images generated by whole-body FDG PET volume to employ pre-trained models based on 2-D images. Methods As a retrospective, proof-of-concept study, 562 [ 18 F]FDG PET/CT images and clinicopathological factors of lung cancer patients were collected. MIP images of anterior and lateral views were used as inputs, and image features were extracted by a pre-trained convolutional neural network (CNN) model, ResNet-50. The relationship between the images was depicted on a parametric 2-D axes map using t-distributed stochastic neighborhood embedding (t-SNE), with clinicopathological factors. Results A DL-based feature map extracted by two MIP images was embedded by t-SNE. According to the visualization of the t-SNE map, PET images were clustered by clinicopathological features. The representative difference between the clusters of PET patterns according to the posture of a patient was visually identified. This map showed a pattern of clustering according to various clinicopathological factors including sex as well as tumor staging. Conclusion A 2-D image-based pre-trained model could extract image patterns of whole-body FDG PET volume by using anterior and lateral views of MIP images bypassing the direct use of 3-D PET volume that requires large datasets and resources. We suggest that this approach could be implemented as a backbone model for various applications for whole-body PET image analyses.