From Shallow to Deep: Exploiting Feature-Based Classifiers for Domain Adaptation in Semantic Segmentation
The remarkable performance of Convolutional Neural Networks on image segmentation tasks comes at the cost of a large amount of pixelwise annotated images that have to be segmented for training. In contrast, feature-based learning methods, such as the Random Forest, require little training data, but...
Saved in:
Published in | Frontiers in computer science (Lausanne) Vol. 4 |
---|---|
Main Authors | , , , |
Format | Journal Article |
Language | English |
Published |
Frontiers Media S.A
03.03.2022
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | The remarkable performance of Convolutional Neural Networks on image segmentation tasks comes at the cost of a large amount of pixelwise annotated images that have to be segmented for training. In contrast, feature-based learning methods, such as the Random Forest, require little training data, but rarely reach the segmentation accuracy of CNNs. This work bridges the two approaches in a transfer learning setting. We show that a CNN can be trained to correct the errors of the Random Forest in the source domain and then be applied to correct such errors in the target domain without retraining, as the domain shift between the Random Forest predictions is much smaller than between the raw data. By leveraging a few brushstrokes as annotations in the target domain, the method can deliver segmentations that are sufficiently accurate to act as pseudo-labels for target-domain CNN training. We demonstrate the performance of the method on several datasets with the challenging tasks of mitochondria, membrane and nuclear segmentation. It yields excellent performance compared to microscopy domain adaptation baselines, especially when a significant domain shift is involved. |
---|---|
AbstractList | The remarkable performance of Convolutional Neural Networks on image segmentation tasks comes at the cost of a large amount of pixelwise annotated images that have to be segmented for training. In contrast, feature-based learning methods, such as the Random Forest, require little training data, but rarely reach the segmentation accuracy of CNNs. This work bridges the two approaches in a transfer learning setting. We show that a CNN can be trained to correct the errors of the Random Forest in the source domain and then be applied to correct such errors in the target domain without retraining, as the domain shift between the Random Forest predictions is much smaller than between the raw data. By leveraging a few brushstrokes as annotations in the target domain, the method can deliver segmentations that are sufficiently accurate to act as pseudo-labels for target-domain CNN training. We demonstrate the performance of the method on several datasets with the challenging tasks of mitochondria, membrane and nuclear segmentation. It yields excellent performance compared to microscopy domain adaptation baselines, especially when a significant domain shift is involved. |
Author | Matskevych, Alex Kreshuk, Anna Pape, Constantin Wolny, Adrian |
Author_xml | – sequence: 1 givenname: Alex surname: Matskevych fullname: Matskevych, Alex – sequence: 2 givenname: Adrian surname: Wolny fullname: Wolny, Adrian – sequence: 3 givenname: Constantin surname: Pape fullname: Pape, Constantin – sequence: 4 givenname: Anna surname: Kreshuk fullname: Kreshuk, Anna |
BookMark | eNpNkN1KwzAUgINMcM49gHd5gc40adPUu7k5HQy8mF6H0-Z0ZrRNSSrq29utIsKB8wcf53zXZNK6Fgm5jdlCCJXfVaVrugVnnC8US2MpL8iUS55EucrV5F99ReYhHBljPI3jVGVTYjfeNXT_DnXtPmnv6Bqxu6ePX13tbG_bA90g9B8eowcIaOiqhhBsZdEHWjlP164B29Klga6H3rqWDt0eG2h7Ww7FocF2XNyQywrqgPPfPCNvm8fX1XO0e3narpa7qBRp0ke5QA6yKrO4MhBzdYq85KoyElNlpAGUjJUgcyWzPEulQi4TNXwk04JlKGZkO3KNg6PuvG3Af2sHVp8Hzh80-OG4GjVwEEZwLEqFSSGZMhkmnEkOPBM5KwZWPLJK70LwWP3xYqZP6vVZvT6p16N68QNxMXnR |
CitedBy_id | crossref_primary_10_1016_j_neunet_2022_03_019 crossref_primary_10_1016_j_polymer_2023_126597 |
Cites_doi | 10.1109/TMI.2019.2946462 10.1007/978-3-540-45167-9_14 10.1023/A:1010933404324 10.1101/548081 10.1109/TMI.2020.3023466 10.1016/j.cviu.2021.103248 10.3389/fnana.2015.00142 10.1109/TMI.2019.2947628 10.1038/nmeth.4151 10.1109/TPAMI.2018.2814042 10.1038/s41592-019-0612-7 10.7554/eLife.57613 10.1109/ISBI.2019.8759383 10.1016/j.cell.2015.06.054 10.1109/CVPR46437.2021.01265 10.1371/journal.pbio.1002340 10.1093/bioinformatics/btx180 10.1038/s41592-019-0582-9 |
ContentType | Journal Article |
DBID | AAYXX CITATION DOA |
DOI | 10.3389/fcomp.2022.805166 |
DatabaseName | CrossRef Directory of Open Access Journals |
DatabaseTitle | CrossRef |
DatabaseTitleList | CrossRef |
Database_xml | – sequence: 1 dbid: DOA name: Directory of Open Access Journals url: https://www.doaj.org/ sourceTypes: Open Website |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Computer Science |
EISSN | 2624-9898 |
ExternalDocumentID | oai_doaj_org_article_a2a3d32ebc8e4b608d7e42062a27390b 10_3389_fcomp_2022_805166 |
GroupedDBID | 9T4 AAFWJ AAYXX ACXDI AFPKN ALMA_UNASSIGNED_HOLDINGS CITATION GROUPED_DOAJ M~E OK1 |
ID | FETCH-LOGICAL-c354t-93e2a6fc71fda12812819c28fd6e58d6dae600ca6986797568e264802565b07e3 |
IEDL.DBID | DOA |
ISSN | 2624-9898 |
IngestDate | Thu Jul 04 21:11:12 EDT 2024 Fri Aug 23 00:44:35 EDT 2024 |
IsDoiOpenAccess | true |
IsOpenAccess | true |
IsPeerReviewed | true |
IsScholarly | true |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-LOGICAL-c354t-93e2a6fc71fda12812819c28fd6e58d6dae600ca6986797568e264802565b07e3 |
OpenAccessLink | https://doaj.org/article/a2a3d32ebc8e4b608d7e42062a27390b |
ParticipantIDs | doaj_primary_oai_doaj_org_article_a2a3d32ebc8e4b608d7e42062a27390b crossref_primary_10_3389_fcomp_2022_805166 |
PublicationCentury | 2000 |
PublicationDate | 2022-03-03 |
PublicationDateYYYYMMDD | 2022-03-03 |
PublicationDate_xml | – month: 03 year: 2022 text: 2022-03-03 day: 03 |
PublicationDecade | 2020 |
PublicationTitle | Frontiers in computer science (Lausanne) |
PublicationYear | 2022 |
Publisher | Frontiers Media S.A |
Publisher_xml | – name: Frontiers Media S.A |
References | Beier (B3) 2017; 14 Zhao (B35) 2021 Gerhard (B13) 2013 Taha (B27) 2021 Wu (B31) 2021 Lucchi (B21) 2013 Chen (B9) 2019 Wei (B29) 2020 Xing (B32) 2019 Liu (B19) 2021 Zhang (B33) 2021 Kasthuri (B16) 2015; 162 Tarvainen (B28) 2017 Januszewski (B15) 2019; 2019 Caicedo (B8) 2019; 16 Roels (B24) 2019 Prabhu (B23) 2021 Breiman (B7) 2001; 45 Rozantsev (B26) 2018; 41 Choi (B10) 2019 Long (B20) 2015 Meilă (B22) 2003 Zhang (B34) 2018 Liu (B18) 2020; 40 Berg (B5) 2019; 16 Wolny (B30) 2020; 9 Belevich (B4) 2016; 14 Du (B11) 2021 Bermúdez-Chacón (B6) 2019; 39 El Jurdi (B12) 2021; 210 Arganda-Carreras (B1) 2017; 33 Ronneberger (B25) 2015 Arganda-Carreras (B2) 2015; 9 Han (B14) 2016 Kumar (B17) 2019; 39 |
References_xml | – year: 2013 ident: B13 article-title: Segmented anisotropic sstem dataset of neural tissue. figshare. Dataset contributor: fullname: Gerhard – volume: 39 start-page: 1256 year: 2019 ident: B6 article-title: Visual correspondences for unsupervised domain adaptation on electron microscopy images publication-title: IEEE Trans. Med. Imaging doi: 10.1109/TMI.2019.2946462 contributor: fullname: Bermúdez-Chacón – start-page: 66 volume-title: International Conference on Medical Image Computing and Computer-Assisted Intervention year: 2020 ident: B29 article-title: Mitoem dataset: large-scale 3d mitochondria instance segmentation from EM images contributor: fullname: Wei – start-page: 740 volume-title: International Conference on Medical Image Computing and Computer-Assisted Intervention year: 2019 ident: B32 article-title: Adversarial domain adaptation and pseudo-labeling for cross-modality microscopy image quantification contributor: fullname: Xing – start-page: 173 volume-title: Learning Theory and Kernel Machines year: 2003 ident: B22 article-title: Comparing clusterings by the variation of information doi: 10.1007/978-3-540-45167-9_14 contributor: fullname: Meilă – volume-title: arXiv preprint arXiv:2109.04015 year: 2021 ident: B11 article-title: Generation, augmentation, and alignment: a pseudo-source domain based method for source-free domain adaptation contributor: fullname: Du – volume-title: International Conference on Machine Learning year: 2015 ident: B20 article-title: Learning transferable features with deep adaptation networks contributor: fullname: Long – start-page: 12414 volume-title: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition year: 2021 ident: B33 article-title: Prototypical pseudo label denoising and target structure learning for domain adaptive semantic segmentation contributor: fullname: Zhang – start-page: 865 volume-title: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33 year: 2019 ident: B9 article-title: Synergistic image and feature adaptation: towards cross-modality domain adaptation for medical image segmentation contributor: fullname: Chen – volume: 45 start-page: 5 year: 2001 ident: B7 article-title: Random forests publication-title: Mach. Learn doi: 10.1023/A:1010933404324 contributor: fullname: Breiman – volume-title: arXiv preprint arXiv:1607.04381 year: 2016 ident: B14 article-title: DSD: regularizing deep neural networks with dense-sparse-dense training flow contributor: fullname: Han – start-page: 1195 volume-title: Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17 year: 2017 ident: B28 article-title: Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results contributor: fullname: Tarvainen – volume: 2019 start-page: 548081 year: 2019 ident: B15 article-title: Segmentation-enhanced cyclegan publication-title: bioRxiv doi: 10.1101/548081 contributor: fullname: Januszewski – volume: 40 start-page: 154 year: 2020 ident: B18 article-title: Pdam: a panoptic-level feature alignment framework for unsupervised domain adaptive instance segmentation in microscopy images publication-title: IEEE Trans. Med. Imaging doi: 10.1109/TMI.2020.3023466 contributor: fullname: Liu – volume-title: arXiv preprint arXiv:2106.03422 year: 2021 ident: B35 article-title: Source-free open compound domain adaptation in semantic segmentation contributor: fullname: Zhao – volume: 210 start-page: 103248 year: 2021 ident: B12 article-title: High-level prior-based loss functions for medical image segmentation: a survey publication-title: Comput. Vis. Image Understand doi: 10.1016/j.cviu.2021.103248 contributor: fullname: El Jurdi – volume-title: arXiv preprint arXiv:2107.10140 year: 2021 ident: B23 article-title: S4t: Source-free domain adaptation for semantic segmentation via self-supervised selective self-training contributor: fullname: Prabhu – start-page: 234 volume-title: International Conference on Medical Image Computing and Computer-Assisted Intervention year: 2015 ident: B25 article-title: U-net: convolutional networks for biomedical image segmentation contributor: fullname: Ronneberger – start-page: 191 volume-title: International Conference on Medical Image Computing and Computer-Assisted Intervention year: 2021 ident: B31 article-title: Uncertainty-aware label rectification for domain adaptive mitochondria segmentation contributor: fullname: Wu – volume: 9 start-page: 142 year: 2015 ident: B2 article-title: Crowdsourcing the creation of image segmentation algorithms for connectomics publication-title: Front. Neuroanat doi: 10.3389/fnana.2015.00142 contributor: fullname: Arganda-Carreras – start-page: 1987 volume-title: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition year: 2013 ident: B21 article-title: Learning for structured prediction using approximate subgradient descent with working sets contributor: fullname: Lucchi – volume: 39 start-page: 1380 year: 2019 ident: B17 article-title: A multi-organ nucleus segmentation challenge publication-title: IEEE Trans. Med. Imaging doi: 10.1109/TMI.2019.2947628 contributor: fullname: Kumar – volume: 14 start-page: 101 year: 2017 ident: B3 article-title: Multicut brings automated neurite segmentation closer to human performance publication-title: Nat. Methods doi: 10.1038/nmeth.4151 contributor: fullname: Beier – volume: 41 start-page: 801 year: 2018 ident: B26 article-title: Beyond sharing weights for deep domain adaptation publication-title: IEEE Trans. Pattern Anal. Mach. Intell doi: 10.1109/TPAMI.2018.2814042 contributor: fullname: Rozantsev – volume: 16 start-page: 1247 year: 2019 ident: B8 article-title: Nucleus segmentation across imaging experiments: the 2018 data science bowl publication-title: Nat. Methods doi: 10.1038/s41592-019-0612-7 contributor: fullname: Caicedo – volume: 9 start-page: e57613 year: 2020 ident: B30 article-title: Accurate and versatile 3D segmentation of plant tissues at cellular resolution publication-title: Elife doi: 10.7554/eLife.57613 contributor: fullname: Wolny – start-page: 1519 volume-title: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019) year: 2019 ident: B24 article-title: Domain adaptive segmentation in volume electron microscopy imaging doi: 10.1109/ISBI.2019.8759383 contributor: fullname: Roels – volume: 162 start-page: 648 year: 2015 ident: B16 article-title: Saturated reconstruction of a volume of neocortex publication-title: Cell doi: 10.1016/j.cell.2015.06.054 contributor: fullname: Kasthuri – volume-title: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) year: 2021 ident: B27 article-title: Knowledge evolution in neural networks doi: 10.1109/CVPR46437.2021.01265 contributor: fullname: Taha – volume: 14 start-page: e1002340 year: 2016 ident: B4 article-title: Microscopy image browser: a platform for segmentation and analysis of multidimensional datasets publication-title: PLoS Biol doi: 10.1371/journal.pbio.1002340 contributor: fullname: Belevich – start-page: 599 volume-title: International Conference on Medical Image Computing and Computer-Assisted Intervention year: 2018 ident: B34 article-title: Task driven generative modeling for unsupervised domain adaptation: application to x-ray image segmentation contributor: fullname: Zhang – volume: 33 start-page: 2424 year: 2017 ident: B1 article-title: Trainable Weka segmentation: a machine learning tool for microscopy pixel classification publication-title: Bioinformatics doi: 10.1093/bioinformatics/btx180 contributor: fullname: Arganda-Carreras – volume-title: arXiv preprint arXiv:1908.00262 year: 2019 ident: B10 article-title: Pseudo-labeling curriculum for unsupervised domain adaptation contributor: fullname: Choi – volume: 16 start-page: 1226 year: 2019 ident: B5 article-title: Ilastik: interactive machine learning for (bio) image analysis publication-title: Nat. Methods doi: 10.1038/s41592-019-0582-9 contributor: fullname: Berg – start-page: 1215 volume-title: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition year: 2021 ident: B19 article-title: Source-free domain adaptation for semantic segmentation contributor: fullname: Liu |
SSID | ssj0002511587 |
Score | 2.2539754 |
Snippet | The remarkable performance of Convolutional Neural Networks on image segmentation tasks comes at the cost of a large amount of pixelwise annotated images that... |
SourceID | doaj crossref |
SourceType | Open Website Aggregation Database |
SubjectTerms | biomedical segmentation deep learning domain adaptation microscopy segmentation transfer learning |
Title | From Shallow to Deep: Exploiting Feature-Based Classifiers for Domain Adaptation in Semantic Segmentation |
URI | https://doaj.org/article/a2a3d32ebc8e4b608d7e42062a27390b |
Volume | 4 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV09b4MwELWqTF36XTX9kodOlWjAYDDdkiZRVCld0kjZkMFHxQBEEVG3_vbeGdJm69IFAQJkvTO-d7rzPcYekK-loHOPghzfCTy0hRIgHEBrRxCnOdh93PO3cLYMXldytSf1RTVhbXvgFriBFto3voA0UxCkoatMBIFwQ6HR8cZualdfT-4FU7QGE3GWKmrTmBiFxYOcSrQxHhTiSeFEtF0Rfx3RXr9-61imJ-yoY4R82I7klB1AdcaOd2oLvPv5zlkx3dQlX5D4Sf3Jm5qPAdbP3BbRFVS8zInObTfgjNAzGW7lLoucpK45MlM-rktdVHxo9LpNv3O8WkCJ0BYZnnyU3Tak6oItp5P3l5nTCSU4mS-Dxol9EJo27Xi50ZQao-xYJlRuQpDKhEYD8ppMhzG114tkqIAK24juyNSNwL9kvaqu4IpxtKn28ti42oXABKnCt0IZedpqGinVZ4871JJ12w8jwTiCIE4sxAlBnLQQ99mIcP15kFpZ2xto4KQzcPKXga__4yM37JDGZYvH_FvWazZbuEM20aT3duLgcf41-QaV-sjf |
link.rule.ids | 315,786,790,870,2115,27957,27958 |
linkProvider | Directory of Open Access Journals |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=From+Shallow+to+Deep%3A+Exploiting+Feature-Based+Classifiers+for+Domain+Adaptation+in+Semantic+Segmentation&rft.jtitle=Frontiers+in+computer+science+%28Lausanne%29&rft.au=Alex+Matskevych&rft.au=Adrian+Wolny&rft.au=Constantin+Pape&rft.au=Anna+Kreshuk&rft.date=2022-03-03&rft.pub=Frontiers+Media+S.A&rft.eissn=2624-9898&rft.volume=4&rft_id=info:doi/10.3389%2Ffcomp.2022.805166&rft.externalDBID=DOA&rft.externalDocID=oai_doaj_org_article_a2a3d32ebc8e4b608d7e42062a27390b |
thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2624-9898&client=summon |
thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2624-9898&client=summon |
thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2624-9898&client=summon |