Spatial Organization and Molecular Correlation of Tumor-Infiltrating Lymphocytes Using Deep Learning on Pathology Images

• Published in: Cell reports (Cambridge) Vol. 23; no. 1; pp. 181 - 193.e7
• Main Authors: Kurc, Tahsin, Demchok, John A., Ferguson, Martin L., Tarnuzzer, Roy, Yang, Liming, DeFreitas, Timothy, Noble, Michael S., Voet, Doug, Dhankani, Varsha, Leinonen, Kalle, Ju, Zhenlin, Korkut, Anil, Liu, Wenbin, Abeshouse, Adam, Chakravarty, Debyani, Taylor, Barry S., Anur, Pavana, Wong, Christopher K., Yau, Christina, Ally, Adrian, Kasaian, Katayoon, Mayo, Michael, Moore, Richard A., Berger, Ashton C., Gao, Galen F., Meyerson, Matthew, Danilova, Ludmila, Bootwalla, Moiz S., Hoadley, Katherine A., Jones, Corbin D., Mose, Lisle E., Tan, Donghui, Hu, Jianhong, Liu, Xiuping, Sheth, Margi, Bailey, Matthew, Fronick, Catrina C., Kandoth, Cyriac, Wilson, Richard K., Lau, Kevin, Penny, Robert, Thompson, Eric, Fregnani, José H., McPherson, Christopher, Berkowitz, Ross, Barnholtz-Sloan, Jill S., Myers, Jerome, Tucker, Kelinda, Zelinka, Tomas, Sepulveda, Antonia, Desjardins, Laurence, Lipp, Eric, Sharp, Alexis, Delman, Keith, Owonikoko, Taofeek, Oosterhuis, Wolter, DiMeco, Francesco, Finocchiaro, Gaetano, Perin, Alessandro, Wiznerowicz, Maciej, Baudin, Eric, Noushmehr, Houtan, Scarpace, Lisa, Dottino, Peter R., Breggia, Anne, Bryce, Alan H., Copland, John A., Kocher, Jean-Pierre, Petersen, Gloria, Rivera, Michael, Esmaeli, Bita, Logothetis, Christopher, Sood, Anil, Kefford, Richard, Shannon, Kerwin, Ghossein, Ronald, Smith, Jennifer, Wentzensen, Nicolas, Tcaciuc, Diana, Engel, Jay, Rassl, Doris M., Urba, Walter, Merola, Roberta, Cardinale, Vincenzo, Carpino, Guido, Cottingham, Sandra, Janssen, Klaus-Peter, Davis, Amy, Lang, James, Rai, Karan, Kastl, Alison, Waldmann, Jens, Simon, Ronald, Simko, Jeffry, VandenBerg, Scott, Unterberg, Andreas, Beer, David, Peterson, Lisa, Ley, Timothy, Thorsson, Vésteinn
• Format: Journal Article
• Language: English
• Published: United States Elsevier Inc 03.04.2018; Elsevier
• Subjects: artificial intelligence; bioinformatics; computer vision; Deep Learning; digital pathology; Humans; Image Interpretation, Computer-Assisted - methods; immuno-oncology; lymphocytes; Lymphocytes, Tumor-Infiltrating - metabolism; Lymphocytes, Tumor-Infiltrating - pathology; machine learning; Neoplasms - pathology; tumor microenvironment; tumor-infiltrating lymphocytes; deep learning; lymphocytes; digital pathology; bioinformatics; computer vision; tumor-infiltrating lymphocytes; immuno-oncology; tumor microenvironment; machine learning; artificial intelligence
• ISSN: 2211-1247; 2211-1247
• DOI: 10.1016/j.celrep.2018.03.086

Beyond sample curation and basic pathologic characterization, the digitized H&E-stained images of TCGA samples remain underutilized. To highlight this resource, we present mappings of tumor-infiltrating lymphocytes (TILs) based on H&E images from 13 TCGA tumor types. These TIL maps are derived through computational staining using a convolutional neural network trained to classify patches of images. Affinity propagation revealed local spatial structure in TIL patterns and correlation with overall survival. TIL map structural patterns were grouped using standard histopathological parameters. These patterns are enriched in particular T cell subpopulations derived from molecular measures. TIL densities and spatial structure were differentially enriched among tumor types, immune subtypes, and tumor molecular subtypes, implying that spatial infiltrate state could reflect particular tumor cell aberration states. Obtaining spatial lymphocytic patterns linked to the rich genomic characterization of TCGA samples demonstrates one use for the TCGA image archives with insights into the tumor-immune microenvironment. [Display omitted] •Deep learning based computational stain for staining tumor-infiltrating lymphocytes (TILs)•TIL patterns generated from 4,759 TCGA subjects (5,202 H&E slides), 13 cancer types•Computationally stained TILs correlate with pathologist eye and molecular estimates•TIL patterns linked to tumor and immune molecular features, cancer type, and outcome Tumor-infiltrating lymphocytes (TILs) were identified from standard pathology cancer images by a deep-learning-derived “computational stain” developed by Saltz et al. They processed 5,202 digital images from 13 cancer types. Resulting TIL maps were correlated with TCGA molecular data, relating TIL content to survival, tumor subtypes, and immune profiles.