In-Field Automatic Identification of Pomegranates Using a Farmer Robot

• Published in: Sensors (Basel, Switzerland) Vol. 22; no. 15; p. 5821
• Main Authors: Devanna, Rosa Pia, Milella, Annalisa, Marani, Roberto, Garofalo, Simone Pietro, Vivaldi, Gaetano Alessandro, Pascuzzi, Simone, Galati, Rocco, Reina, Giulio
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 04.08.2022; MDPI
• Subjects: Accuracy; agricultural robotics; Automatic vehicle identification systems; Automation; Datasets; Deep learning; fruit detection; Fruits; Image annotation; Image contrast; Labeling; multi-stage transfer learning; Orchards; Pomegranates; precision farming; Robotics
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s22155821

Ground vehicles equipped with vision-based perception systems can provide a rich source of information for precision agriculture tasks in orchards, including fruit detection and counting, phenotyping, plant growth and health monitoring. This paper presents a semi-supervised deep learning framework for automatic pomegranate detection using a farmer robot equipped with a consumer-grade camera. In contrast to standard deep-learning methods that require time-consuming and labor-intensive image labeling, the proposed system relies on a novel multi-stage transfer learning approach, whereby a pre-trained network is fine-tuned for the target task using images of fruits in controlled conditions, and then it is progressively extended to more complex scenarios towards accurate and efficient segmentation of field images. Results of experimental tests, performed in a commercial pomegranate orchard in southern Italy, are presented using the DeepLabv3+ (Resnet18) architecture, and they are compared with those that were obtained based on conventional manual image annotation. The proposed framework allows for accurate segmentation results, achieving an F1-score of 86.42% and IoU of 97.94%, while relieving the burden of manual labeling.