SonoSAMTrack -- Segment and Track Anything on Ultrasound Images
In this paper, we present SonoSAMTrack - that combines a promptable foundational model for segmenting objects of interest on ultrasound images called SonoSAM, with a state-of-the art contour tracking model to propagate segmentations on 2D+t and 3D ultrasound datasets. Fine-tuned and tested exclusive...
Saved in:
Main Authors | , , , , , , , |
---|---|
Format | Journal Article |
Language | English |
Published |
25.10.2023
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | In this paper, we present SonoSAMTrack - that combines a promptable
foundational model for segmenting objects of interest on ultrasound images
called SonoSAM, with a state-of-the art contour tracking model to propagate
segmentations on 2D+t and 3D ultrasound datasets. Fine-tuned and tested
exclusively on a rich, diverse set of objects from $\approx200$k ultrasound
image-mask pairs, SonoSAM demonstrates state-of-the-art performance on 7 unseen
ultrasound data-sets, outperforming competing methods by a significant margin.
We also extend SonoSAM to 2-D +t applications and demonstrate superior
performance making it a valuable tool for generating dense annotations and
segmentation of anatomical structures in clinical workflows. Further, to
increase practical utility of the work, we propose a two-step process of
fine-tuning followed by knowledge distillation to a smaller footprint model
without comprising the performance. We present detailed qualitative and
quantitative comparisons of SonoSAM with state-of-the-art methods showcasing
efficacy of the method. This is followed by demonstrating the reduction in
number of clicks in a dense video annotation problem of adult cardiac
ultrasound chamber segmentation using SonoSAMTrack. |
---|---|
AbstractList | In this paper, we present SonoSAMTrack - that combines a promptable
foundational model for segmenting objects of interest on ultrasound images
called SonoSAM, with a state-of-the art contour tracking model to propagate
segmentations on 2D+t and 3D ultrasound datasets. Fine-tuned and tested
exclusively on a rich, diverse set of objects from $\approx200$k ultrasound
image-mask pairs, SonoSAM demonstrates state-of-the-art performance on 7 unseen
ultrasound data-sets, outperforming competing methods by a significant margin.
We also extend SonoSAM to 2-D +t applications and demonstrate superior
performance making it a valuable tool for generating dense annotations and
segmentation of anatomical structures in clinical workflows. Further, to
increase practical utility of the work, we propose a two-step process of
fine-tuning followed by knowledge distillation to a smaller footprint model
without comprising the performance. We present detailed qualitative and
quantitative comparisons of SonoSAM with state-of-the-art methods showcasing
efficacy of the method. This is followed by demonstrating the reduction in
number of clicks in a dense video annotation problem of adult cardiac
ultrasound chamber segmentation using SonoSAMTrack. |
Author | Ravishankar, Hariharan Annangi, Pavan Taha, Kass-Hout Patil, Rohan Suthar, Harsh Bhatia, Parminder Anzengruber, Stephan Melapudi, Vikram |
Author_xml | – sequence: 1 givenname: Hariharan surname: Ravishankar fullname: Ravishankar, Hariharan – sequence: 2 givenname: Rohan surname: Patil fullname: Patil, Rohan – sequence: 3 givenname: Vikram surname: Melapudi fullname: Melapudi, Vikram – sequence: 4 givenname: Harsh surname: Suthar fullname: Suthar, Harsh – sequence: 5 givenname: Stephan surname: Anzengruber fullname: Anzengruber, Stephan – sequence: 6 givenname: Parminder surname: Bhatia fullname: Bhatia, Parminder – sequence: 7 givenname: Kass-Hout surname: Taha fullname: Taha, Kass-Hout – sequence: 8 givenname: Pavan surname: Annangi fullname: Annangi, Pavan |
BackLink | https://doi.org/10.48550/arXiv.2310.16872$$DView paper in arXiv |
BookMark | eNotj0FPwjAYhnvQgyI_wBP9A8W1hX7dySxEhQTjYeO8fF3bscha000j_x4ET2_yvMmTPPfkJsTgCHnk2Xyhl8vsCdNv9zMX8gy40iDuyHMZQyyL9yph80kZo6VrexdGisHSKyzCcdx3oaUx0N1hTDjE7_O56bF1wwO59XgY3PR_J6R6falWa7b9eNusii1DBYLZBi1ar0AtDJdKCJk14HgO1noJKgctQebQWPRSOW2UAq2t4cYZ763QckJmV-2loP5KXY_pWP-V1JcSeQKtAkSj |
ContentType | Journal Article |
Copyright | http://creativecommons.org/licenses/by-nc-nd/4.0 |
Copyright_xml | – notice: http://creativecommons.org/licenses/by-nc-nd/4.0 |
DBID | AKY GOX |
DOI | 10.48550/arxiv.2310.16872 |
DatabaseName | arXiv Computer Science arXiv.org |
DatabaseTitleList | |
Database_xml | – sequence: 1 dbid: GOX name: arXiv.org url: http://arxiv.org/find sourceTypes: Open Access Repository |
DeliveryMethod | fulltext_linktorsrc |
ExternalDocumentID | 2310_16872 |
GroupedDBID | AKY GOX |
ID | FETCH-LOGICAL-a672-dcadadf6764b1362230c7e197ddf37697837397cdaf36e8b66788db1bebffd283 |
IEDL.DBID | GOX |
IngestDate | Mon Jan 08 05:46:39 EST 2024 |
IsDoiOpenAccess | true |
IsOpenAccess | true |
IsPeerReviewed | false |
IsScholarly | false |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-LOGICAL-a672-dcadadf6764b1362230c7e197ddf37697837397cdaf36e8b66788db1bebffd283 |
OpenAccessLink | https://arxiv.org/abs/2310.16872 |
ParticipantIDs | arxiv_primary_2310_16872 |
PublicationCentury | 2000 |
PublicationDate | 2023-10-25 |
PublicationDateYYYYMMDD | 2023-10-25 |
PublicationDate_xml | – month: 10 year: 2023 text: 2023-10-25 day: 25 |
PublicationDecade | 2020 |
PublicationYear | 2023 |
Score | 1.9041717 |
SecondaryResourceType | preprint |
Snippet | In this paper, we present SonoSAMTrack - that combines a promptable
foundational model for segmenting objects of interest on ultrasound images
called SonoSAM,... |
SourceID | arxiv |
SourceType | Open Access Repository |
SubjectTerms | Computer Science - Computer Vision and Pattern Recognition |
Title | SonoSAMTrack -- Segment and Track Anything on Ultrasound Images |
URI | https://arxiv.org/abs/2310.16872 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwdV09T8MwELVKJxYEAlQ-5YHVECeO7YwRohSkwtBWyhb5EyEgRUmK4N9zToJgYT178bPu3vPZvkPogoa7LZ95YhJqCEsiRaR1KZE00t5GUnMZ_g7PH_hsxe6LtBgh_PMXRtWfzx99fWDdXAXxcUm5FBBkt-I4PNm6fSz6y8muFNcw_3ceaMzO9IckprtoZ1B3OO-3Yw-NXLUPenhdrRf5HGjBvGBC8MI9hZwchkM87o159dWGVBBeV3j12taqCd2O8N0beHtzgJbTm-X1jAx9C4jiIibWKKus54IzTYEfQOQb4WgmrPXgziHXIkAFGKt8wh2gAXwhrabaae8t0P0hGsPR300QZpnxxorU8CxhghkVJ9pHTmeU-VRE4ghNutWW731pijIAUXZAHP8_dIK2Q9P0EIHj9BSN23rjzoBaW33e4fsNL9F4ew |
link.rule.ids | 228,230,786,891 |
linkProvider | Cornell University |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=SonoSAMTrack+--+Segment+and+Track+Anything+on+Ultrasound+Images&rft.au=Ravishankar%2C+Hariharan&rft.au=Patil%2C+Rohan&rft.au=Melapudi%2C+Vikram&rft.au=Suthar%2C+Harsh&rft.date=2023-10-25&rft_id=info:doi/10.48550%2Farxiv.2310.16872&rft.externalDocID=2310_16872 |