Deep learning aided computer vision system for automated linear type trait evaluation in dairy cows

• Published in: Smart agricultural technology Vol. 8; p. 100509
• Main Authors: Devi, Indu, Singh, Naseeb, Dudi, Kuldeep, Ranjan, Rakesh, Lathwal, Surender Singh, Tomar, Divyanshu Singh, Nagar, Harsh
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.08.2024; Elsevier
• Subjects: Automatic measurement; Computer vision; Dairy cattle; Deep learning; Precision livestock farming; Deep learning; Computer vision; Automatic measurement; Precision livestock farming; Dairy cattle
• ISSN: 2772-3755; 2772-3755
• DOI: 10.1016/j.atech.2024.100509

•Developed an automatic system for linear type trait estimation of dairy cows.•System consists of RGB-D camera and CNNs for segmentation of body parts.•CNN-based models achieved mIoU over 90.0% for body parts of cows.•Developed various algorithms to extract key points of traits.•Achieved MAPE below 15.0% in estimated traits by developed system. The assessment of traits is important in determining production potential, reproductive performance, and overall health of dairy cows. The assessment of these traits typically involves visual inspection and manual measurement, which can be time-consuming, subject to bias, and potentially distressing for the animals. To address these challenges, convolutional neural networks (CNNs)-aided non-invasive computer vision system was developed in the present study. This system consists of a depth camera to acquire the RGB images and depth information of cows. The DeepLabV3+ model, having the ResNet50 model as a backbone, was utilized to segment the body parts of cows from RGB images. Image processing-based algorithms were developed to extract key pixel locations for each trait from these segmented images. The system estimated trait dimensions utilizing 3D data of respective key points. The mean-IoU (intersection-over-union) values for the developed segmentation models were 93.46%, 91.25%, and 99.27% for side-view, back-view traits, and stature, respectively. Additionally, the vision system was able to estimate the trait dimensions with mean absolute percentage error (MAPE) below 6.0%. For a few traits, MAPE, however, exceeded 10.0%, indicating higher error. Inaccurate segmentation, imprecise key point extraction, visual overlaps of specific body parts, and variations in cow postures contribute to such errors. The developed system attained a Ratio of Performance to Deviation (RPD) above 1.2 for all traits, indicating its ability to estimate the dimensions of traits efficaciously. Thus, the present study demonstrated the potential of a CNN-based computer vision-based system for automating the trait measurement process in cows.