Batch-Mask: Automated Image Segmentation for Organisms with Limbless or Non-Standard Body Forms

• Published in: Integrative and comparative biology Vol. 62; no. 4; pp. 1111 - 1120
• Main Authors: Curlis, John David, Renney, Timothy, Davis Rabosky, Alison R, Moore, Talia Y
• Format: Journal Article
• Language: English
• Published: England Oxford University Press 29.10.2022
• Subjects: Editor's Choice; Symposium; vision; image segmentation; color evolution; convolutional neural network; photography; sensory ecology
• ISSN: 1540-7063; 1557-7023; 1557-7023
• DOI: 10.1093/icb/icac036

Efficient comparisons of biological color patterns are critical for understanding the mechanisms by which organisms evolve in nature, including sexual selection, predator–prey interactions, and thermoregulation. However, limbless, elongate, or spiral-shaped organisms do not conform to the standard orientation and photographic techniques required for many automated analyses. Currently, large-scale color analysis of elongate animals requires time-consuming manual landmarking, which reduces their representation in coloration research despite their ecological importance. We present Batch-Mask: an automated, customizable workflow to automatically analyze large photographic datasets to isolate non-standard biological organisms from the background. Batch-Mask is completely open-source and does not depend on any proprietary software. We also present a user guide for fine-tuning weights to a custom dataset and incorporating existing manual visual analysis tools (e.g., micaToolbox) into a single automated workflow for comparing color patterns across images. Batch-Mask was 60x faster than manual landmarking and produced masks that correctly identified 96% of all snake pixels. To validate our approach, we used micaToolbox to compare pattern energy in a sample set of snake photographs segmented by Batch-Mask and humans and found no significant difference in the output results. The fine-tuned weights, user guide, and automated workflow substantially decrease the amount of time and attention required to quantitatively analyze non-standard biological subjects. With these tools, biologists can compare color, pattern, and shape differences in large datasets that include significant morphological variation in elongate body forms. This advance is especially valuable for comparative analyses of natural history collections across a broad range of morphologies. Through landmark-free automation, Batch-Mask can greatly expand the scale of space, time, or taxonomic breadth across which color variation can be quantitatively examined.