Recognition of sweet peppers and planning the robotic picking sequence in high-density orchards

• Published in: Computers and electronics in agriculture Vol. 196; p. 106878
• Main Authors: Ning, Zhengtong, Luo, Lufeng, Ding, XinMing, Dong, Zhiqiang, Yang, Bofeng, Cai, Jinghui, Chen, Weilin, Lu, Qinghua
• Format: Journal Article
• Language: English
• Published: Amsterdam Elsevier B.V 01.05.2022; Elsevier BV
• Subjects: Algorithms; Anti-collision picking; Clustering; Collision avoidance; Damage prevention; Density; Inflection points; Localization; Multi-target picking; Object recognition; Orchards; Peppers; Picking; Planning; Robot arms; Robotic picking; Sequence planning; Sweet pepper; Technical services; Sweet pepper; Multi-target picking; Anti-collision picking; Sequence planning; Robotic picking
• ISSN: 0168-1699; 1872-7107
• DOI: 10.1016/j.compag.2022.106878

•A YOLO-V4-CBAM method for the accurate recognition of green pepper in a dense orchard environment was developed.•A DPC algorithm for automatic classification of picking clusters is proposed.•A principle of anti-collision picking within picking clusters is proposed based on the "empirical expert method", Gaussian distance weighting, and "winner-take-all" optical neural filter competition method. To improve the operational efficiency of and to prevent possible collision damage in the near-neighbor multi-target picking of sweet peppers by robots in densely planted complex orchards, this study proposes an algorithm for recognizing sweet peppers and planning a picking sequence called AYDY. First, the convolutional block attention module is embedded into the you only look once model (YOLO-V4), and this combined model is used to recognize and localize sweet peppers. Then, the clustering algorithm for the fast search-and-find of density peaks is improved based on the inflection points and gaps of a decision graph. Sweet peppers with multiple near-neighbor targets are automatically partitioned into picking clusters. An anti-collision picking sequence for a picking cluster is determined based on the experience of experts. The algorithm combines Gaussian distance weights with the winner-takes-all approach as an optic neural filter. In tests, the F1-score of this method for sweet peppers in a densely planted environment was 91.84%, which is a 9.14% improvement compared to YOLO-V4. The average localization accuracy and collision-free harvesting success rate were 89.55% and 90.04%, respectively. The recognition and localization time for a single image was 0.3033 s. The time to plan a picking sequence for a single image was 0.283 s. When the robotic arm harvested 22 and 24 sweet peppers, compared to sequential and stochastic planning, the proposed method had higher collision-free picking rates by 18.18, 18.18, 16.67, and 25 percentage points, respectively. This method can accurately detect sweet peppers, reduce collision damage, and improve picking efficiency in high-density orchard environments. This study may provide technical support for anti-collision picking of sweet peppers by robots.