Fruit growing direction recognition and nesting grasping strategies for tomato harvesting robots

• Published in: Journal of field robotics Vol. 41; no. 2; pp. 300 - 313
• Main Authors: Zheng, Xiajun, Rong, Jiacheng, Zhang, Zhiqin, Yang, Yan, Li, Wei, Yuan, Ting
• Format: Journal Article
• Language: English
• Published: Hoboken Wiley Subscription Services, Inc 01.03.2024
• Subjects: Computer vision; deep learning; Field tests; fruit growing direction recognition; Grasping (robotics); Harvesting; inspection robot; Nesting; nesting grasping strategies; object detection; Robots; tomato harvesting; tomato yield estimation; Tomatoes; Vision systems
• ISSN: 1556-4959; 1556-4967
• DOI: 10.1002/rob.22263

In recent years, the potential of robotic harvesting in greenhouse tomato production has garnered significant attention within the tomato industry. However, there is a lack of sufficient research on the complete replacement of manual harvesting with this technology. In this paper, we propose a tomato harvesting method that utilizes a nesting approach to simplify the process and minimize damage. The paper describes the tomato harvesting robot prototype, the visual system equipped with three vision‐based tomato detectors: YOLOv5_CBAM, which incorporates a convolutional block attention module; YOLOv5_SE, enhanced with a squeeze‐and‐excitation block; and a standard YOLOv5s model. Additionally, a novel shear gripping method for fruit bunches is presented, utilizing a bottom‐up snapping technique during harvesting. Point cloud data are utilized to determine the position of the tomato's main stem and bunch. The paper includes field tests and experimental findings, which indicate that the YOLOv5_CBAM model achieves the highest precision (82.62%) and recall (82.57%), outperforming YOLOv5_SE and standard YOLOv5s. Field experiments demonstrate that the improved end‐effector and vision system have significantly enhanced the robot's performance, achieving a 57.5% harvesting success rate in just 14.9 s.