The application of a workflow integrating the variable reproducibility and harmonizability of radiomic features on a phantom dataset

• Published in: PloS one Vol. 16; no. 5; p. e0251147
• Main Authors: Ibrahim, Abdalla, Refaee, Turkey, Leijenaar, Ralph T. H., Primakov, Sergey, Hustinx, Roland, Mottaghy, Felix M., Woodruff, Henry C., Maidment, Andrew D. A., Lambin, Philippe
• Format: Journal Article Web Resource
• Language: English
• Published: United States Public Library of Science 07.05.2021; Public Library of Science (PLoS)
• Subjects: Algorithms; Analysis; Artificial intelligence; Biology and Life Sciences; Biomarkers; Computer and Information Sciences; Data analysis; Deep learning; Diagnostic imaging; Drafting software; Editing; Equipment and supplies; Gene expression; Health care facilities; Health physics; Human health sciences; Humans; Image Processing, Computer-Assisted; Image Processing, Computer-Assisted/methods; Medical imaging; Medical research; Medicine; Modems; Multidisciplinary; Nuclear medicine; Oncology; Phantoms, Imaging; Physics; Precision medicine; Radiographic Image Interpretation, Computer-Assisted; Radiographic Image Interpretation, Computer-Assisted/methods; Radiologie, médecine & imagerie nucléaire; Radiology; Radiology, nuclear medicine & imaging; Radiomics; Reproducibility; Reproducibility of Results; Research and Analysis Methods; Scanners; Sciences de la santé humaine; Social Sciences; Software; Standardization; Visualization; Workflow; Netherlands; Aachen Germany; Belgium; Germany
• ISSN: 1932-6203; 1932-6203
• DOI: 10.1371/journal.pone.0251147

Radiomics–the high throughput extraction of quantitative features from medical images and their correlation with clinical and biological endpoints- is the subject of active and extensive research. Although the field shows promise, the generalizability of radiomic signatures is affected significantly by differences in scan acquisition and reconstruction settings. Previous studies reported on the sensitivity of radiomic features (RFs) to test-retest variability, inter-observer segmentation variability, and intra-scanner variability. A framework involving robust radiomics analysis and the application of a post-reconstruction feature harmonization method using ComBat was recently proposed to address these challenges. In this study, we investigated the reproducibility of RFs across different scanners and scanning parameters using this framework. We analysed thirteen scans of a ten-layer phantom that were acquired differently. Each layer was subdivided into sixteen regions of interest (ROIs), and the scans were compared in a pairwise manner, resulting in seventy-eight different scenarios. Ninety-one RFs were extracted from each ROI. As hypothesized, we demonstrate that the reproducibility of a given RF is not a constant but is dependent on the heterogeneity found in the data under analysis. The number (%) of reproducible RFs varied across the pairwise scenarios investigated, having a wide range between 8 (8.8%) and 78 (85.7%) RFs. Furthermore, in contrast to what has been previously reported, and as hypothesized in the robust radiomics analysis framework, our results demonstrate that ComBat cannot be applied to all RFs but rather on a percentage of those–the “ComBatable” RFs–which differed depending on the data being harmonized. The number (%) of reproducible RFs following ComBat harmonization varied across the pairwise scenarios investigated, ranging from 14 (15.4%) to 80 (87.9%) RFs, and was found to depend on the heterogeneity in the data. We conclude that the standardization of image acquisition protocols remains the cornerstone for improving the reproducibility of RFs, and the generalizability of the signatures developed. Our proposed approach helps identify the reproducible RFs across different datasets.