Zero-shot performance of the Segment Anything Model (SAM) in 2D medical imaging: A comprehensive evaluation and practical guidelines

• Published in: arXiv.org
• Main Authors: Mattjie, Christian, de Moura, Luis Vinicius, Ravazio, Rafaela Cappelari, Lucas Silveira Kupssinskü, Parraga, Otávio, Marcelo Mussi Delucis, Rodrigo Coelho Barros
• Format: Paper Journal Article
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 05.05.2023
• Subjects: Computer Science - Artificial Intelligence; Computer Science - Computer Vision and Pattern Recognition; Datasets; Domains; Image segmentation; Medical imaging; X ray imagery
• ISSN: 2331-8422
• DOI: 10.48550/arxiv.2305.00109

Segmentation in medical imaging is a critical component for the diagnosis, monitoring, and treatment of various diseases and medical conditions. Presently, the medical segmentation landscape is dominated by numerous specialized deep learning models, each fine-tuned for specific segmentation tasks and image modalities. The recently-introduced Segment Anything Model (SAM) employs the ViT neural architecture and harnesses a massive training dataset to segment nearly any object; however, its suitability to the medical domain has not yet been investigated. In this study, we explore the zero-shot performance of SAM in medical imaging by implementing eight distinct prompt strategies across six datasets from four imaging modalities, including X-ray, ultrasound, dermatoscopy, and colonoscopy. Our findings reveal that SAM's zero-shot performance is not only comparable to, but in certain cases, surpasses the current state-of-the-art. Based on these results, we propose practical guidelines that require minimal interaction while consistently yielding robust outcomes across all assessed contexts. The source code, along with a demonstration of the recommended guidelines, can be accessed at https://github.com/Malta-Lab/SAM-zero-shot-in-Medical-Imaging.