Structured crowdsourcing enables convolutional segmentation of histology images
Abstract Motivation While deep-learning algorithms have demonstrated outstanding performance in semantic image segmentation tasks, large annotation datasets are needed to create accurate models. Annotation of histology images is challenging due to the effort and experience required to carefully deli...
Saved in:
| Published in | Bioinformatics Vol. 35; no. 18; pp. 3461 - 3467 |
|---|---|
| Main Authors | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
| Format | Journal Article |
| Language | English |
| Published |
England
Oxford University Press
15.09.2019
|
| Subjects | |
| Online Access | Get full text |
Cover
Loading…
| Abstract | Abstract
Motivation
While deep-learning algorithms have demonstrated outstanding performance in semantic image segmentation tasks, large annotation datasets are needed to create accurate models. Annotation of histology images is challenging due to the effort and experience required to carefully delineate tissue structures, and difficulties related to sharing and markup of whole-slide images.
Results
We recruited 25 participants, ranging in experience from senior pathologists to medical students, to delineate tissue regions in 151 breast cancer slides using the Digital Slide Archive. Inter-participant discordance was systematically evaluated, revealing low discordance for tumor and stroma, and higher discordance for more subjectively defined or rare tissue classes. Feedback provided by senior participants enabled the generation and curation of 20 000+ annotated tissue regions. Fully convolutional networks trained using these annotations were highly accurate (mean AUC=0.945), and the scale of annotation data provided notable improvements in image classification accuracy.
Availability and Implementation
Dataset is freely available at: https://goo.gl/cNM4EL.
Supplementary information
Supplementary data are available at Bioinformatics online. |
|---|---|
| AbstractList | While deep-learning algorithms have demonstrated outstanding performance in semantic image segmentation tasks, large annotation datasets are needed to create accurate models. Annotation of histology images is challenging due to the effort and experience required to carefully delineate tissue structures, and difficulties related to sharing and markup of whole-slide images.
We recruited 25 participants, ranging in experience from senior pathologists to medical students, to delineate tissue regions in 151 breast cancer slides using the Digital Slide Archive. Inter-participant discordance was systematically evaluated, revealing low discordance for tumor and stroma, and higher discordance for more subjectively defined or rare tissue classes. Feedback provided by senior participants enabled the generation and curation of 20,000+ annotated tissue regions. Fully convolutional networks trained using these annotations were highly accurate (Mean AUC= 0.945), and the scale of annotation data provided notable improvements in image classification accuracy.
Supplementary dataset hosted at: https://goo.gl/cNM4EL. Additional results and methods are available at Bioinformatics online. Abstract Motivation While deep-learning algorithms have demonstrated outstanding performance in semantic image segmentation tasks, large annotation datasets are needed to create accurate models. Annotation of histology images is challenging due to the effort and experience required to carefully delineate tissue structures, and difficulties related to sharing and markup of whole-slide images. Results We recruited 25 participants, ranging in experience from senior pathologists to medical students, to delineate tissue regions in 151 breast cancer slides using the Digital Slide Archive. Inter-participant discordance was systematically evaluated, revealing low discordance for tumor and stroma, and higher discordance for more subjectively defined or rare tissue classes. Feedback provided by senior participants enabled the generation and curation of 20 000+ annotated tissue regions. Fully convolutional networks trained using these annotations were highly accurate (mean AUC=0.945), and the scale of annotation data provided notable improvements in image classification accuracy. Availability and Implementation Dataset is freely available at: https://goo.gl/cNM4EL. Supplementary information Supplementary data are available at Bioinformatics online. While deep-learning algorithms have demonstrated outstanding performance in semantic image segmentation tasks, large annotation datasets are needed to create accurate models. Annotation of histology images is challenging due to the effort and experience required to carefully delineate tissue structures, and difficulties related to sharing and markup of whole-slide images.MOTIVATIONWhile deep-learning algorithms have demonstrated outstanding performance in semantic image segmentation tasks, large annotation datasets are needed to create accurate models. Annotation of histology images is challenging due to the effort and experience required to carefully delineate tissue structures, and difficulties related to sharing and markup of whole-slide images.We recruited 25 participants, ranging in experience from senior pathologists to medical students, to delineate tissue regions in 151 breast cancer slides using the Digital Slide Archive. Inter-participant discordance was systematically evaluated, revealing low discordance for tumor and stroma, and higher discordance for more subjectively defined or rare tissue classes. Feedback provided by senior participants enabled the generation and curation of 20 000+ annotated tissue regions. Fully convolutional networks trained using these annotations were highly accurate (mean AUC=0.945), and the scale of annotation data provided notable improvements in image classification accuracy.RESULTSWe recruited 25 participants, ranging in experience from senior pathologists to medical students, to delineate tissue regions in 151 breast cancer slides using the Digital Slide Archive. Inter-participant discordance was systematically evaluated, revealing low discordance for tumor and stroma, and higher discordance for more subjectively defined or rare tissue classes. Feedback provided by senior participants enabled the generation and curation of 20 000+ annotated tissue regions. Fully convolutional networks trained using these annotations were highly accurate (mean AUC=0.945), and the scale of annotation data provided notable improvements in image classification accuracy.Dataset is freely available at: https://goo.gl/cNM4EL.AVAILABILITY AND IMPLEMENTATIONDataset is freely available at: https://goo.gl/cNM4EL.Supplementary data are available at Bioinformatics online.SUPPLEMENTARY INFORMATIONSupplementary data are available at Bioinformatics online. |
| Author | Saad, Anas M Sakr, Rokia A Hussein, Hagar Khalaf, Mariam M Elkashash, Ahmad M Elsebaie, Mai A T Manthey, David Elsebaie, Maha A T Younes, Duaa M Amgad, Mohamed Chittajallu, Deepak R Gadallah, Ahmed M Zaki, Basma M Beezley, Jonathan Alagha, Yahya Osman, Mohamed H Ahmed, Joumana Salem, Hazem S E Rahman, Mustafijur Cooper, Lee A D Ismail, Ahmed F Alhusseiny, Ahmed M Abdulkarim, Ali Fala, Salma Y Younes, Abo-Alela F Elfandy, Habiba Elgazar, Nada M Atteya, Lamees A Gutman, David A Abo Elnasr, Lamia S Ruhban, Inas A |
| AuthorAffiliation | 7 Department of Pathology, Medical Research Institute, Alexandria University , Alexandria, Egypt 2 Department of Pathology, National Cancer Institute , Cairo, Egypt 11 Department of Medicine, Zagazig University , Zagazig, Egypt 12 Department of Medicine, Batterjee Medical College , Jeddah, Saudi Arabia 3 Department of Medicine, Cairo University , Cairo, Egypt 13 Department of Medicine, Suez Canal University , Ismailia, Egypt 16 Department of Biomedical Engineering, Emory University , Atlanta, GA, USA 15 Department of Neurology, Emory University School of Medicine , Atlanta, GA, USA 4 Egyptian Ministry of Health , Cairo, Egypt 1 Department of Biomedical Informatics, Emory University School of Medicine , Atlanta, GA, USA 8 Department of Medicine, Chittagong University , Chittagong, Bangladesh 6 Department of Medicine, Menoufia University , Menoufia, Egypt 14 Kitware Inc., Clifton Park , NY, USA 9 Department of Medicine, Damascus University , Damascus, Syria 10 Department of Medicine, Mansoura Universi |
| AuthorAffiliation_xml | – name: 5 Department of Medicine, Ain Shams University , Cairo, Egypt – name: 11 Department of Medicine, Zagazig University , Zagazig, Egypt – name: 12 Department of Medicine, Batterjee Medical College , Jeddah, Saudi Arabia – name: 15 Department of Neurology, Emory University School of Medicine , Atlanta, GA, USA – name: 2 Department of Pathology, National Cancer Institute , Cairo, Egypt – name: 16 Department of Biomedical Engineering, Emory University , Atlanta, GA, USA – name: 13 Department of Medicine, Suez Canal University , Ismailia, Egypt – name: 7 Department of Pathology, Medical Research Institute, Alexandria University , Alexandria, Egypt – name: 8 Department of Medicine, Chittagong University , Chittagong, Bangladesh – name: 6 Department of Medicine, Menoufia University , Menoufia, Egypt – name: 10 Department of Medicine, Mansoura University , Mansoura, Egypt – name: 14 Kitware Inc., Clifton Park , NY, USA – name: 1 Department of Biomedical Informatics, Emory University School of Medicine , Atlanta, GA, USA – name: 4 Egyptian Ministry of Health , Cairo, Egypt – name: 3 Department of Medicine, Cairo University , Cairo, Egypt – name: 9 Department of Medicine, Damascus University , Damascus, Syria |
| Author_xml | – sequence: 1 givenname: Mohamed orcidid: 0000-0001-7599-6162 surname: Amgad fullname: Amgad, Mohamed organization: Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, GA, USA – sequence: 2 givenname: Habiba surname: Elfandy fullname: Elfandy, Habiba organization: Department of Pathology, National Cancer Institute, Cairo, Egypt – sequence: 3 givenname: Hagar surname: Hussein fullname: Hussein, Hagar organization: Department of Medicine, Cairo University, Cairo, Egypt – sequence: 4 givenname: Lamees A surname: Atteya fullname: Atteya, Lamees A organization: Egyptian Ministry of Health, Cairo, Egypt – sequence: 5 givenname: Mai A T surname: Elsebaie fullname: Elsebaie, Mai A T organization: Department of Medicine, Ain Shams University, Cairo, Egypt – sequence: 6 givenname: Lamia S surname: Abo Elnasr fullname: Abo Elnasr, Lamia S organization: Department of Medicine, Menoufia University, Menoufia, Egypt – sequence: 7 givenname: Rokia A surname: Sakr fullname: Sakr, Rokia A organization: Department of Medicine, Menoufia University, Menoufia, Egypt – sequence: 8 givenname: Hazem S E surname: Salem fullname: Salem, Hazem S E organization: Department of Medicine, Ain Shams University, Cairo, Egypt – sequence: 9 givenname: Ahmed F surname: Ismail fullname: Ismail, Ahmed F organization: Department of Pathology, Medical Research Institute, Alexandria University, Alexandria, Egypt – sequence: 10 givenname: Anas M surname: Saad fullname: Saad, Anas M organization: Department of Medicine, Ain Shams University, Cairo, Egypt – sequence: 11 givenname: Joumana surname: Ahmed fullname: Ahmed, Joumana organization: Department of Medicine, Cairo University, Cairo, Egypt – sequence: 12 givenname: Maha A T surname: Elsebaie fullname: Elsebaie, Maha A T organization: Department of Medicine, Ain Shams University, Cairo, Egypt – sequence: 13 givenname: Mustafijur surname: Rahman fullname: Rahman, Mustafijur organization: Department of Medicine, Chittagong University, Chittagong, Bangladesh – sequence: 14 givenname: Inas A surname: Ruhban fullname: Ruhban, Inas A organization: Department of Medicine, Damascus University, Damascus, Syria – sequence: 15 givenname: Nada M surname: Elgazar fullname: Elgazar, Nada M organization: Department of Medicine, Mansoura University, Mansoura, Egypt – sequence: 16 givenname: Yahya surname: Alagha fullname: Alagha, Yahya organization: Department of Medicine, Cairo University, Cairo, Egypt – sequence: 17 givenname: Mohamed H surname: Osman fullname: Osman, Mohamed H organization: Department of Medicine, Zagazig University, Zagazig, Egypt – sequence: 18 givenname: Ahmed M surname: Alhusseiny fullname: Alhusseiny, Ahmed M organization: Department of Medicine, Mansoura University, Mansoura, Egypt – sequence: 19 givenname: Mariam M surname: Khalaf fullname: Khalaf, Mariam M organization: Department of Medicine, Batterjee Medical College, Jeddah, Saudi Arabia – sequence: 20 givenname: Abo-Alela F surname: Younes fullname: Younes, Abo-Alela F organization: Department of Medicine, Ain Shams University, Cairo, Egypt – sequence: 21 givenname: Ali surname: Abdulkarim fullname: Abdulkarim, Ali organization: Department of Medicine, Cairo University, Cairo, Egypt – sequence: 22 givenname: Duaa M surname: Younes fullname: Younes, Duaa M organization: Department of Medicine, Ain Shams University, Cairo, Egypt – sequence: 23 givenname: Ahmed M surname: Gadallah fullname: Gadallah, Ahmed M organization: Department of Medicine, Ain Shams University, Cairo, Egypt – sequence: 24 givenname: Ahmad M surname: Elkashash fullname: Elkashash, Ahmad M organization: Department of Medicine, Cairo University, Cairo, Egypt – sequence: 25 givenname: Salma Y surname: Fala fullname: Fala, Salma Y organization: Department of Medicine, Suez Canal University, Ismailia, Egypt – sequence: 26 givenname: Basma M surname: Zaki fullname: Zaki, Basma M organization: Department of Medicine, Suez Canal University, Ismailia, Egypt – sequence: 27 givenname: Jonathan surname: Beezley fullname: Beezley, Jonathan organization: Kitware Inc., Clifton Park, NY, USA – sequence: 28 givenname: Deepak R surname: Chittajallu fullname: Chittajallu, Deepak R organization: Kitware Inc., Clifton Park, NY, USA – sequence: 29 givenname: David surname: Manthey fullname: Manthey, David organization: Kitware Inc., Clifton Park, NY, USA – sequence: 30 givenname: David A surname: Gutman fullname: Gutman, David A organization: Department of Neurology, Emory University School of Medicine, Atlanta, GA, USA – sequence: 31 givenname: Lee A D orcidid: 0000-0002-3504-4965 surname: Cooper fullname: Cooper, Lee A D email: lee.cooper@emory.edu organization: Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, GA, USA |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/30726865$$D View this record in MEDLINE/PubMed |
| BookMark | eNqNUU1v1yAcJmbGvX4ETY9euv0olNKYmJhlOpMlO7g7AQodhsJfoDPz04-lc3Fe9ASE5y3Pc4j2QgwGobcYTjGM5Ey56IKNaZHF6Xymyi_g5BU6wJRB20E_7tU7YUNLOZB9dJjzd4AeU0rfoH0CQ8c46w_Q9beSVl3WZKZGp_hzynFN2oW5MUEqb3KjY7iLfi0uBumbbObFhCIfn020za3LJfo43zdukbPJx-i1lT6bk6fzCN18vrg5v2yvrr98Pf901eoe89JKOmjcqW6EngGfRlDUTFZxrRixtrPAO8m5YaAtpWRURHc9JnIcNRiMCTlCHzfZ3aoWM-kaKUkvdqmmSPciSide_gR3K-Z4J9hA-TCyKvD-SSDFH6vJRSwua-O9DCauWXSYE8xpT6FC3_3p9Wzyu8QK6DdALTDnZOwzBIN4HEu8HEtsY1Xeh7942m3N1sjO_5MNGzuuu_80fAB-D7dt |
| CitedBy_id | crossref_primary_10_1016_j_dsp_2021_103196 crossref_primary_10_1038_s41591_023_02643_7 crossref_primary_10_1016_j_cmpbup_2023_100104 crossref_primary_10_1016_j_ajpath_2020_03_012 crossref_primary_10_1016_j_patter_2023_100688 crossref_primary_10_1016_j_jpi_2022_100103 crossref_primary_10_1109_JBHI_2020_2991043 crossref_primary_10_1016_j_media_2023_103069 crossref_primary_10_1016_j_modpat_2023_100196 crossref_primary_10_3390_data7060075 crossref_primary_10_1200_JCO_23_02641 crossref_primary_10_1200_JCO_24_02279 crossref_primary_10_3233_ICA_220676 crossref_primary_10_23939_ujit2022_02_040 crossref_primary_10_1145_3577011 crossref_primary_10_1007_s12530_023_09491_3 crossref_primary_10_1016_j_cmpb_2021_106464 crossref_primary_10_1002_path_5331 crossref_primary_10_1109_TETCI_2023_3287978 crossref_primary_10_1016_j_media_2022_102487 crossref_primary_10_1186_s12967_020_02502_w crossref_primary_10_3390_cancers15143635 crossref_primary_10_1109_TSP_2023_3322843 crossref_primary_10_1038_s41598_025_86527_5 crossref_primary_10_1109_ACCESS_2024_3377428 crossref_primary_10_1007_s12257_023_0164_7 crossref_primary_10_34133_2022_9893639 crossref_primary_10_1038_s41523_020_0154_2 crossref_primary_10_1093_bib_bbac367 crossref_primary_10_1186_s12885_024_12962_8 crossref_primary_10_3390_s22166053 crossref_primary_10_1186_s13058_023_01726_0 crossref_primary_10_1002_path_6155 crossref_primary_10_1016_j_compbiomed_2022_106412 crossref_primary_10_1016_j_jpi_2024_100363 crossref_primary_10_3390_sym12010154 crossref_primary_10_1016_j_jpi_2022_100160 crossref_primary_10_1088_1361_6560_acaeee crossref_primary_10_1016_j_compmedimag_2024_102482 crossref_primary_10_1016_j_cmpb_2022_106828 crossref_primary_10_1016_j_media_2023_102845 crossref_primary_10_2139_ssrn_4202295 crossref_primary_10_1016_j_media_2020_101813 crossref_primary_10_1016_j_jbi_2023_104303 crossref_primary_10_1007_s00521_023_08394_3 crossref_primary_10_1016_j_cmpb_2024_108472 crossref_primary_10_1038_s41598_022_26482_7 crossref_primary_10_1038_s41523_024_00663_1 crossref_primary_10_1093_bioinformatics_btad138 crossref_primary_10_1016_j_compmedimag_2021_101974 crossref_primary_10_1109_ACCESS_2024_3470213 crossref_primary_10_1109_ACCESS_2024_3423701 crossref_primary_10_1016_j_jpi_2023_100357 crossref_primary_10_1016_j_cmpb_2024_108119 crossref_primary_10_1016_j_medengphy_2025_104284 crossref_primary_10_1109_JBHI_2022_3216293 crossref_primary_10_1038_s41523_021_00346_1 crossref_primary_10_1007_s42979_024_02855_9 crossref_primary_10_1109_TMI_2023_3274652 crossref_primary_10_1016_j_bspc_2024_106493 crossref_primary_10_1109_JBHI_2024_3383590 crossref_primary_10_3389_fdata_2020_00019 crossref_primary_10_1016_j_compmedimag_2022_102155 crossref_primary_10_1016_j_csbj_2024_12_033 crossref_primary_10_1007_s11263_024_02246_w crossref_primary_10_3389_frai_2021_754641 crossref_primary_10_1109_TMI_2022_3191398 crossref_primary_10_1111_his_15205 crossref_primary_10_1002_cyto_a_24514 crossref_primary_10_1117_1_JMI_11_4_044006 crossref_primary_10_3390_bioengineering9090475 crossref_primary_10_3788_CJL240614 crossref_primary_10_1038_s41598_021_90821_3 crossref_primary_10_1080_21681163_2023_2198037 crossref_primary_10_2196_42723 crossref_primary_10_1007_s10278_024_01043_8 crossref_primary_10_1016_j_ajpath_2022_12_011 crossref_primary_10_3390_mi13122197 crossref_primary_10_1016_j_patcog_2024_110621 crossref_primary_10_3390_s22072775 crossref_primary_10_1016_j_artmed_2023_102686 crossref_primary_10_1109_JBHI_2023_3237137 crossref_primary_10_1109_ACCESS_2023_3237990 crossref_primary_10_1016_j_media_2025_103553 crossref_primary_10_1007_s10916_021_01786_9 crossref_primary_10_1016_j_bspc_2023_104976 crossref_primary_10_1016_j_eswa_2022_117525 crossref_primary_10_1016_j_media_2022_102685 crossref_primary_10_2478_ausi_2023_0006 crossref_primary_10_1016_j_media_2023_102743 crossref_primary_10_1038_s43856_022_00138_z crossref_primary_10_1371_journal_pone_0314450 crossref_primary_10_3390_bioengineering10040396 crossref_primary_10_1002_pros_24721 crossref_primary_10_4103_jpi_jpi_83_20 crossref_primary_10_1016_j_media_2023_102997 crossref_primary_10_1007_s10278_024_01018_9 crossref_primary_10_1016_j_bspc_2023_105544 crossref_primary_10_1038_s41591_021_01343_4 crossref_primary_10_1200_CCI_19_00126 crossref_primary_10_1016_j_bspc_2022_103564 crossref_primary_10_1016_j_addr_2021_113959 crossref_primary_10_1007_s11263_025_02372_z crossref_primary_10_1016_j_neucom_2024_127834 crossref_primary_10_1016_j_eswa_2023_120128 crossref_primary_10_1038_s41591_023_02504_3 crossref_primary_10_1038_s43856_022_00186_5 crossref_primary_10_1016_j_jvcir_2023_103929 crossref_primary_10_1016_j_compbiomed_2023_107268 crossref_primary_10_1109_TCYB_2024_3403927 crossref_primary_10_1007_s10278_024_01041_w crossref_primary_10_1093_gigascience_giac037 crossref_primary_10_1016_j_imavis_2024_105371 crossref_primary_10_1111_jop_13414 crossref_primary_10_1016_j_media_2022_102594 crossref_primary_10_1007_s11517_023_02970_4 crossref_primary_10_1021_acs_chemrestox_3c00058 crossref_primary_10_1038_s41597_020_0528_1 crossref_primary_10_1016_j_compbiomed_2024_109621 crossref_primary_10_3390_cancers14092148 crossref_primary_10_4103_jpi_jpi_64_19 crossref_primary_10_1007_s00428_021_03213_3 crossref_primary_10_1038_s41551_022_00929_8 crossref_primary_10_1038_s44172_024_00246_9 crossref_primary_10_1088_1742_6596_2504_1_012047 crossref_primary_10_3390_bioengineering11050434 crossref_primary_10_1111_his_14304 crossref_primary_10_1007_s10489_023_05190_3 crossref_primary_10_1016_j_compmedimag_2024_102327 crossref_primary_10_1038_s41698_023_00352_5 crossref_primary_10_1186_s40478_023_01691_x crossref_primary_10_3390_cancers13123050 crossref_primary_10_1016_j_ebiom_2021_103492 |
| Cites_doi | 10.1038/s41592-018-0069-0 10.1097/MD.0000000000004614 10.4103/jpi.jpi_65_17 10.1038/srep43286 10.1109/38.946629 10.1561/0600000071 10.1136/amiajnl-2012-001469 10.1158/0008-5472.CAN-17-0629 |
| ContentType | Journal Article |
| Copyright | The Author(s) 2019. Published by Oxford University Press. 2019 The Author(s) 2019. Published by Oxford University Press. |
| Copyright_xml | – notice: The Author(s) 2019. Published by Oxford University Press. 2019 – notice: The Author(s) 2019. Published by Oxford University Press. |
| DBID | TOX AAYXX CITATION NPM 7X8 5PM |
| DOI | 10.1093/bioinformatics/btz083 |
| DatabaseName | Oxford Journals Open Access Collection CrossRef PubMed MEDLINE - Academic PubMed Central (Full Participant titles) |
| DatabaseTitle | CrossRef PubMed MEDLINE - Academic |
| DatabaseTitleList | PubMed MEDLINE - Academic |
| Database_xml | – sequence: 1 dbid: NPM name: PubMed url: https://proxy.k.utb.cz/login?url=http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: TOX name: Oxford Open Access Journals url: https://academic.oup.com/journals/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Biology |
| EISSN | 1460-2059 1367-4811 |
| EndPage | 3467 |
| ExternalDocumentID | PMC6748796 30726865 10_1093_bioinformatics_btz083 10.1093/bioinformatics/btz083 |
| Genre | Journal Article |
| GrantInformation_xml | – fundername: U.S. National Institutes of Health – fundername: National Cancer Institute grants grantid: U01CA220401; U24CA194362 – fundername: NCI NIH HHS grantid: U24 CA194362 – fundername: ; – fundername: ; grantid: U01CA220401; U24CA194362 |
| GroupedDBID | -~X .2P 5GY AAMVS ABJNI ABPTD ACGFS ADZXQ ALMA_UNASSIGNED_HOLDINGS F5P HW0 Q5Y RD5 TLC TN5 TOX WH7 --- -E4 .DC .I3 0R~ 23N 2WC 4.4 48X 53G 5WA 70D AAIJN AAIMJ AAJKP AAKPC AAMDB AAOGV AAPQZ AAPXW AAUQX AAVAP AAVLN AAYXX ABEJV ABEUO ABGNP ABIXL ABNKS ABPQP ABQLI ABWST ABXVV ABZBJ ACIWK ACPRK ACUFI ACUXJ ACYTK ADBBV ADEYI ADEZT ADFTL ADGKP ADGZP ADHKW ADHZD ADMLS ADOCK ADPDF ADRDM ADRTK ADVEK ADYVW ADZTZ AECKG AEGPL AEJOX AEKKA AEKSI AELWJ AEMDU AENEX AENZO AEPUE AETBJ AEWNT AFFZL AFGWE AFIYH AFOFC AFRAH AGINJ AGKEF AGQXC AGSYK AHMBA AHXPO AIJHB AJEEA AJEUX AKHUL AKWXX ALTZX ALUQC AMNDL APIBT APWMN ARIXL ASPBG AVWKF AXUDD AYOIW AZVOD BAWUL BAYMD BHONS BQDIO BQUQU BSWAC BTQHN C45 CDBKE CITATION CS3 CZ4 DAKXR DIK DILTD DU5 D~K EBD EBS EE~ EMOBN F9B FEDTE FHSFR FLIZI FLUFQ FOEOM FQBLK GAUVT GJXCC GROUPED_DOAJ GX1 H13 H5~ HAR HZ~ IOX J21 JXSIZ KAQDR KOP KQ8 KSI KSN M-Z MK~ ML0 N9A NGC NLBLG NMDNZ NOMLY NU- O9- OAWHX ODMLO OJQWA OK1 OVD OVEED P2P PAFKI PEELM PQQKQ Q1. R44 RNS ROL RPM RUSNO RW1 RXO SV3 TEORI TJP TR2 W8F WOQ X7H YAYTL YKOAZ YXANX ZKX ~91 ~KM ADRIX AFXEN BCRHZ EJD M49 NPM ROX 7X8 5PM |
| ID | FETCH-LOGICAL-c518t-a47c12b2905608d90b4edfb8cb63ff2f082a88e60cf4439b3c2513a99c0e1133 |
| IEDL.DBID | TOX |
| ISSN | 1367-4803 1367-4811 |
| IngestDate | Thu Aug 21 13:47:04 EDT 2025 Thu Jul 10 17:10:34 EDT 2025 Wed Feb 19 02:36:52 EST 2025 Tue Jul 01 02:33:48 EDT 2025 Thu Apr 24 23:02:46 EDT 2025 Wed Apr 02 07:04:42 EDT 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 18 |
| Language | English |
| License | This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited. http://creativecommons.org/licenses/by/4.0 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c518t-a47c12b2905608d90b4edfb8cb63ff2f082a88e60cf4439b3c2513a99c0e1133 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 |
| ORCID | 0000-0002-3504-4965 0000-0001-7599-6162 |
| OpenAccessLink | https://dx.doi.org/10.1093/bioinformatics/btz083 |
| PMID | 30726865 |
| PQID | 2183184540 |
| PQPubID | 23479 |
| PageCount | 7 |
| ParticipantIDs | pubmedcentral_primary_oai_pubmedcentral_nih_gov_6748796 proquest_miscellaneous_2183184540 pubmed_primary_30726865 crossref_primary_10_1093_bioinformatics_btz083 crossref_citationtrail_10_1093_bioinformatics_btz083 oup_primary_10_1093_bioinformatics_btz083 |
| ProviderPackageCode | CITATION AAYXX |
| PublicationCentury | 2000 |
| PublicationDate | 2019-09-15 |
| PublicationDateYYYYMMDD | 2019-09-15 |
| PublicationDate_xml | – month: 09 year: 2019 text: 2019-09-15 day: 15 |
| PublicationDecade | 2010 |
| PublicationPlace | England |
| PublicationPlace_xml | – name: England |
| PublicationTitle | Bioinformatics |
| PublicationTitleAlternate | Bioinformatics |
| PublicationYear | 2019 |
| Publisher | Oxford University Press |
| Publisher_xml | – name: Oxford University Press |
| References | Irshad (2023013108054510300_btz083-B8) 2017; 7 Reinhard (2023013108054510300_btz083-B12) 2001; 21 Irshad (2023013108054510300_btz083-B7) 2015 Gutman (2023013108054510300_btz083-B4) 2013; 20 Alialy (2023013108054510300_btz083-B1) 2018; 9 Kovashka (2023013108054510300_btz083-B9) 2016; 10 Su (2023013108054510300_btz083-B14) 2012 Gutman (2023013108054510300_btz083-B5) 2017; 77 Fouad (2023013108054510300_btz083-B3) 2017; 7 Hughes (2023013108054510300_btz083-B6) 2018; 15 Plasilova (2023013108054510300_btz083-B11) 2016; 95 Long (2023013108054510300_btz083-B10) 2015 |
| References_xml | – volume: 15 start-page: 587 year: 2018 ident: 2023013108054510300_btz083-B6 article-title: Quanti.us: a tool for rapid, flexible, crowd-based annotation of images publication-title: Nat. Methods doi: 10.1038/s41592-018-0069-0 – start-page: 294 year: 2015 ident: 2023013108054510300_btz083-B7 article-title: Crowdsourcing image annotation for nucleus detection and segmentation in computational pathology: evaluating experts, automated methods, and the crowd publication-title: Pac. Symp. Biocomput – volume: 95 start-page: e4614. year: 2016 ident: 2023013108054510300_btz083-B11 article-title: Features of triple-negative breast cancer: analysis of 38, 813 cases from the national cancer database publication-title: Medicine doi: 10.1097/MD.0000000000004614 – volume: 7 start-page: 1016 year: 2017 ident: 2023013108054510300_btz083-B3 article-title: Revisiting the hallmarks of cancer publication-title: Am. J. Cancer Res – volume: 9 start-page: 2. year: 2018 ident: 2023013108054510300_btz083-B1 article-title: A review on the applications of crowdsourcing in human pathology publication-title: J. Pathol. Inform doi: 10.4103/jpi.jpi_65_17 – volume: 7 start-page: 43286. year: 2017 ident: 2023013108054510300_btz083-B8 article-title: Crowdsourcing scoring of immunohistochemistry images: evaluating Performance of the Crowd and an Automated Computational Method publication-title: Sci. Rep doi: 10.1038/srep43286 – volume: 21 start-page: 34 year: 2001 ident: 2023013108054510300_btz083-B12 article-title: Color transfer between images publication-title: IEEE Comput. Graph doi: 10.1109/38.946629 – volume: 10 start-page: 177 year: 2016 ident: 2023013108054510300_btz083-B9 article-title: Crowdsourcing in computer vision publication-title: Found. Trends Comput. Graph. Vis doi: 10.1561/0600000071 – volume: 20 start-page: 1091 year: 2013 ident: 2023013108054510300_btz083-B4 article-title: Cancer Digital Slide Archive: an informatics resource to support integrated in silico analysis of TCGA pathology data publication-title: J. Am. Med. Inform. Assoc doi: 10.1136/amiajnl-2012-001469 – volume-title: Workshops at the Twenty-Sixth AAAI Conference on Artificial Intelligence year: 2012 ident: 2023013108054510300_btz083-B14 – start-page: 3431 year: 2015 ident: 2023013108054510300_btz083-B10 – volume: 77 start-page: e75 year: 2017 ident: 2023013108054510300_btz083-B5 article-title: The digital slide archive: a software platform for management, integration, and analysis of histology for cancer research publication-title: Cancer Res doi: 10.1158/0008-5472.CAN-17-0629 |
| SSID | ssj0051444 ssj0005056 |
| Score | 2.670421 |
| Snippet | Abstract
Motivation
While deep-learning algorithms have demonstrated outstanding performance in semantic image segmentation tasks, large annotation datasets... While deep-learning algorithms have demonstrated outstanding performance in semantic image segmentation tasks, large annotation datasets are needed to create... |
| SourceID | pubmedcentral proquest pubmed crossref oup |
| SourceType | Open Access Repository Aggregation Database Index Database Enrichment Source Publisher |
| StartPage | 3461 |
| SubjectTerms | Original Papers |
| Title | Structured crowdsourcing enables convolutional segmentation of histology images |
| URI | https://www.ncbi.nlm.nih.gov/pubmed/30726865 https://www.proquest.com/docview/2183184540 https://pubmed.ncbi.nlm.nih.gov/PMC6748796 |
| Volume | 35 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8NAEB5KQfAivo0vInjxELvJ5rVHEUsRtAcr9Bayj2jBJmJaRH-9s9mkNgVRjyG7kzCzu_Pt7sw3AOdUEio5444KCHF8RNhOGulEYEq8jIWKcapzh-_uw8GjfzsOxh0gTS7M6hU-oz0-KWoSUU1c3OOzT4QNuOiiI9Zk-aPh-Dumg2hmGPOASMA3JW01s3dMaJO_85PIlmdqZbstgc7V2MklZ9TfhI0aRdpXxuxb0FH5NqyZupIfOzB8qFhh529K2vixd1kd0aOTslWVKlXaOti8HnQop1RP0zoHKbeLzK5IiLUoezLF9abchVH_ZnQ9cOrKCY4I3HjmpH4kXI97DPVBYskI95XMeCx4SLPMy9Dvp3GsQiIyHxEJpwJhDk0ZE0S5uGvdg25e5OoA7EgT-EWURpJTn8mUe0qkGY1wQ44zPeUW-I3SElGziuviFi-Jud2mSVvXidG1BZeLbq-GVuO3Dhdokb-2PWvsluBk0Tcgaa6KeZloPIhbWkSpFuwbOy5E4mLnhXEYWBC1LLxooIm422_yyXNFyK0LtkQsPPzHPx7BOkKvKlrNDY6hi8NCnSC8mfHTakh_AdfKAHY |
| linkProvider | Oxford University Press |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Structured+crowdsourcing+enables+convolutional+segmentation+of+histology+images&rft.jtitle=Bioinformatics+%28Oxford%2C+England%29&rft.au=Amgad%2C+Mohamed&rft.au=Elfandy%2C+Habiba&rft.au=Hussein%2C+Hagar&rft.au=Atteya%2C+Lamees+A&rft.date=2019-09-15&rft.pub=Oxford+University+Press&rft.issn=1367-4803&rft.eissn=1367-4811&rft.volume=35&rft.issue=18&rft.spage=3461&rft.epage=3467&rft_id=info:doi/10.1093%2Fbioinformatics%2Fbtz083&rft_id=info%3Apmid%2F30726865&rft.externalDocID=PMC6748796 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1367-4803&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1367-4803&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1367-4803&client=summon |