Abstract PO-076: Unsupervised learning of image embeddings enables new opportunities to extract novel information from digital pathology H&E images
Abstract Introduction: In digital pathology, large amounts of time and money are spent annotating whole slide images (WSIs). By using an AI algorithm trained on thousands of WSIs, it is possible to remove the process of annotating from the workflow whilst simultaneously finding potential imaging fea...
Saved in:
Published in | Clinical cancer research Vol. 27; no. 5_Supplement; p. PO-076 |
---|---|
Main Authors | , , , , , |
Format | Journal Article |
Language | English |
Published |
01.03.2021
|
Online Access | Get full text |
Cover
Loading…
Abstract | Abstract
Introduction: In digital pathology, large amounts of time and money are spent annotating whole slide images (WSIs). By using an AI algorithm trained on thousands of WSIs, it is possible to remove the process of annotating from the workflow whilst simultaneously finding potential imaging features that correlate with drug response, molecular phenotypes, histologic features (e.g. Tertiary Lymphoid Structures), inflammation at the tumor-stroma interface, artifacts, etc. Method: A pipeline was constructed for ingesting WSI from the set of diagnostic slides available in the cancer genome archive (TCGA). The WSIs were masked for tissue areas and divided into patches extracted at multiple magnification levels. These patches were used to train a U-Net (initialised with a pre-trained ResNet34) to complete a self-supervised in-painting task and evaluated with perceptual loss using VGG16. After training, embeddings were extracted from the bottleneck layer of the U-Net, and these were evaluated for their capacity to cluster according to tissue type and magnification level. Results: The trained embeddings showed strong clustering by tissue type, magnification level, and separation of artefacts. The results were qualitatively evaluated by pathologist with consensus and exceeded the performance of a baseline pre-trained ImageNet model. Conclusion: These results highlight a novel methodology for AI algorithm development that removes the need for numerous pathologists to annotate pathology images by hand and converts their role to reviewing and quality-checking images. AI algorithms based on this approach can have a significant impact in the pathologic evaluation of tissue for oncology clinical trials by extracting a richer set of features than glass microscopy alone would permit. Furthermore, self-supervised methods, such as in-painting allow for the training of embeddings that encapsulate a richer understanding of the data and are consequently more repurposable than supervised/labelled approaches.
Citation Format: Jason Hipp, Mona Xu, Lucas Bordeaux, Feng Gu, Carlos Pedrinaci, Khan Baykaner. Unsupervised learning of image embeddings enables new opportunities to extract novel information from digital pathology H&E images [abstract]. In: Proceedings of the AACR Virtual Special Conference on Artificial Intelligence, Diagnosis, and Imaging; 2021 Jan 13-14. Philadelphia (PA): AACR; Clin Cancer Res 2021;27(5_Suppl):Abstract nr PO-076. |
---|---|
AbstractList | Abstract
Introduction: In digital pathology, large amounts of time and money are spent annotating whole slide images (WSIs). By using an AI algorithm trained on thousands of WSIs, it is possible to remove the process of annotating from the workflow whilst simultaneously finding potential imaging features that correlate with drug response, molecular phenotypes, histologic features (e.g. Tertiary Lymphoid Structures), inflammation at the tumor-stroma interface, artifacts, etc. Method: A pipeline was constructed for ingesting WSI from the set of diagnostic slides available in the cancer genome archive (TCGA). The WSIs were masked for tissue areas and divided into patches extracted at multiple magnification levels. These patches were used to train a U-Net (initialised with a pre-trained ResNet34) to complete a self-supervised in-painting task and evaluated with perceptual loss using VGG16. After training, embeddings were extracted from the bottleneck layer of the U-Net, and these were evaluated for their capacity to cluster according to tissue type and magnification level. Results: The trained embeddings showed strong clustering by tissue type, magnification level, and separation of artefacts. The results were qualitatively evaluated by pathologist with consensus and exceeded the performance of a baseline pre-trained ImageNet model. Conclusion: These results highlight a novel methodology for AI algorithm development that removes the need for numerous pathologists to annotate pathology images by hand and converts their role to reviewing and quality-checking images. AI algorithms based on this approach can have a significant impact in the pathologic evaluation of tissue for oncology clinical trials by extracting a richer set of features than glass microscopy alone would permit. Furthermore, self-supervised methods, such as in-painting allow for the training of embeddings that encapsulate a richer understanding of the data and are consequently more repurposable than supervised/labelled approaches.
Citation Format: Jason Hipp, Mona Xu, Lucas Bordeaux, Feng Gu, Carlos Pedrinaci, Khan Baykaner. Unsupervised learning of image embeddings enables new opportunities to extract novel information from digital pathology H&E images [abstract]. In: Proceedings of the AACR Virtual Special Conference on Artificial Intelligence, Diagnosis, and Imaging; 2021 Jan 13-14. Philadelphia (PA): AACR; Clin Cancer Res 2021;27(5_Suppl):Abstract nr PO-076. |
Author | Bordeaux, Lucas Hipp, Jason Pedrinaci, Carlos Xu, Mona Gu, Feng Baykaner, Khan |
Author_xml | – sequence: 1 givenname: Jason surname: Hipp fullname: Hipp, Jason – sequence: 2 givenname: Mona surname: Xu fullname: Xu, Mona – sequence: 3 givenname: Lucas surname: Bordeaux fullname: Bordeaux, Lucas – sequence: 4 givenname: Feng surname: Gu fullname: Gu, Feng – sequence: 5 givenname: Carlos surname: Pedrinaci fullname: Pedrinaci, Carlos – sequence: 6 givenname: Khan surname: Baykaner fullname: Baykaner, Khan |
BookMark | eNqdkE1OwzAQhS1UJFrgCmhW7FLs_FbsKigqq3YBa8tpJsHI8US2W-g5uDAJQT1AVzN6M29G35uxiSWLjN0JPhciWzyILCuiJM6z-fL5NRbRdhPxIr9g09Ng0ve8WEQ8TeIrNvP-k3ORCp5O2c-y9MGpXYDR9gjv1u87dAftsQKDylltG6AadKsaBGxLrKpe8oBWlQY9WPwC6jpyYW910L0SCPB7PGvpgAa0rcm1KmiyUDtqodKNDspAp8IHGWqOsL5fjS_8DbuslfF4-1-vWf6yentaRztH3jusZef6TXeUgsshAjmQyoFU_kUgtxvZsyRnG38Bp_5rJw |
ContentType | Journal Article |
DBID | AAYXX CITATION |
DOI | 10.1158/1557-3265.ADI21-PO-076 |
DatabaseName | CrossRef |
DatabaseTitle | CrossRef |
DatabaseTitleList | CrossRef |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Medicine |
EISSN | 1557-3265 |
EndPage | PO-076 |
ExternalDocumentID | 10_1158_1557_3265_ADI21_PO_076 |
GroupedDBID | --- 18M 29B 2FS 2WC 34G 39C 476 53G 5GY 5RE 5VS 6J9 AAYXX ABOCM ACGFO ACIWK ACPRK ACSVP ADBBV ADCOW ADNWM AENEX AFHIN AFOSN AFRAH ALMA_UNASSIGNED_HOLDINGS BAWUL BR6 BTFSW CITATION CS3 DIK DU5 E3Z EBS EJD F5P FRP GX1 IH2 KQ8 L7B LSO OK1 P0W P2P QTD RCR RHF RHI RNS SJN TR2 W2D W8F WOQ YKV |
ID | FETCH-crossref_primary_10_1158_1557_3265_ADI21_PO_0763 |
ISSN | 1078-0432 |
IngestDate | Fri Aug 23 02:11:55 EDT 2024 |
IsPeerReviewed | true |
IsScholarly | true |
Issue | 5_Supplement |
Language | English |
LinkModel | OpenURL |
MergedId | FETCHMERGED-crossref_primary_10_1158_1557_3265_ADI21_PO_0763 |
ParticipantIDs | crossref_primary_10_1158_1557_3265_ADI21_PO_076 |
PublicationCentury | 2000 |
PublicationDate | 2021-03-01 |
PublicationDateYYYYMMDD | 2021-03-01 |
PublicationDate_xml | – month: 03 year: 2021 text: 2021-03-01 day: 01 |
PublicationDecade | 2020 |
PublicationTitle | Clinical cancer research |
PublicationYear | 2021 |
SSID | ssj0014104 |
Score | 4.8062954 |
Snippet | Abstract
Introduction: In digital pathology, large amounts of time and money are spent annotating whole slide images (WSIs). By using an AI algorithm trained... |
SourceID | crossref |
SourceType | Aggregation Database |
StartPage | PO-076 |
Title | Abstract PO-076: Unsupervised learning of image embeddings enables new opportunities to extract novel information from digital pathology H&E images |
Volume | 27 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV3NbtNAEF6lRap6QUCpCgU0B8QlcpvE67hwi9ogF0jbQyvlZsXedRuJ2FZtc-A1eDheh5nd9XqBCJFeLGuVzNqez7PfjueHsbe4RAiOtMBLxXjhceH7HpJY7slM-FmYLYaJynKdXYyjG_5pHsx7vZ9O1FJTJ0fp97V5JQ_RKo6hXilLdgPNWqE4gOeoXzyihvH4XzqeJOSoSOv-1aU3CMe0u7_Jq6ak979CJvm19XtQXYgVRefIVSKF-tzUlyprqqKe4v2iJBre5Kq8KtFRNNlKcF58k1SVw-Y46nwUsbylZiNUlfVOF3GKEEJTPUnlEt7TNvMyJXjd901xIeuEjpZlqaN1qy4gYN4oP22Rd94CKhG6aNRK-YX6tNm4IfVT1Mqt678YOQFcxuQOqMYvN15OacxwgKZvpLtItHZa1xAweAxi1fW0Cw_S9lc_bmc17wb-XisCyn-wUx1Nzs7x6lwJbnHuPxZNG8qoNlHBSUxyYpITKznx1WWMcrbYo1H4PqBY07Pzz_bzFh-qvpb21k3qOso5Xn89Dmty6M_1E_bY7FtgokH4lPVk_oztzExkxh770WIRtKgP4CIRWiRCkYECCXRIBINEQCTCb0iEugCDRFBIBAeJQEgEg0SwSITo3VRPUT1n44_T69PIa28qLnVtlfjfj9PfZ9t5kcsDBqmfiqHEzTJSUC4GYbLIsnCQJhlHys-FeMGONxT-cuN_HLLdDtGv2HZ938jXyFHr5I3S9y86X5S9 |
link.rule.ids | 315,786,790,27955,27956 |
linkProvider | Flying Publisher |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Abstract+PO-076%3A+Unsupervised+learning+of+image+embeddings+enables+new+opportunities+to+extract+novel+information+from+digital+pathology+H%26E+images&rft.jtitle=Clinical+cancer+research&rft.au=Hipp%2C+Jason&rft.au=Xu%2C+Mona&rft.au=Bordeaux%2C+Lucas&rft.au=Gu%2C+Feng&rft.date=2021-03-01&rft.issn=1078-0432&rft.eissn=1557-3265&rft.volume=27&rft.issue=5_Supplement&rft.spage=PO-076&rft.epage=PO-076&rft_id=info:doi/10.1158%2F1557-3265.ADI21-PO-076&rft.externalDBID=n%2Fa&rft.externalDocID=10_1158_1557_3265_ADI21_PO_076 |
thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1078-0432&client=summon |
thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1078-0432&client=summon |
thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1078-0432&client=summon |