Detection and counting of banana bunches by integrating deep learning and classic image-processing algorithms

• Published in: Computers and electronics in agriculture Vol. 209; p. 107827
• Main Authors: Wu, Fengyun, Yang, Zhou, Mo, Xingkang, Wu, Zihao, Tang, Wei, Duan, Jieli, Zou, Xiangjun
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.06.2023
• Subjects: agriculture; algorithms; Banana fruit cluster; bananas; computer software; decision making; Edge detection; electronics; fruit weight; fruits; harvest date; Image segmentation; Machine vision; neural networks; Recognition of number of fruit bunches; Sterile bud removal; Sterile bud removal; Image segmentation; Machine vision; Banana fruit cluster; Recognition of number of fruit bunches; Edge detection
• ISSN: 0168-1699; 1872-7107
• DOI: 10.1016/j.compag.2023.107827

•A method for Segmenting and counting banana bunches for debudding.•Fusion algorithm-based detection and counting for banana bunches in harvest.•An estimation model for overall bunches counting based on growth pattern.•Counting accuracy were 86% for debudding and 93.2% for harvesting. Robots must first detect the number of banana bunches when making judgements on sterile bud removal and estimating weight for harvest in the field environment. Banana bunches are complex in shape, arranged in a nonlinear helical curve along the stalk, and have different growth states in different periods, with bunches widely spaced in the early period and densely arranged in the harvest period. Deep learning nor classical image-processing algorithms alone can detect and count bunches in both periods. Therefore, these algorithms were combined to calculate the number of bunches in the two periods. For counting bunches in the debudding period, the convolutional neural network Deeplab V3 + model and classic image-processing algorithm were combined to finely segment bunches and calculate bunch numbers, providing intelligent decision-making for judgment on the timing for debudding. To count bunches during harvest, based on deep learning to identify the overall banana fruit cluster, the edge detection algorithm was employed to extract the centroid points of fruit fingers, and the clustering algorithm was used to determine the optimal number of bunches on the visual detection surface. An estimation model for the total number of bunches, including hidden ones, was created based on their helical curve arrangement. The results indicated a target segmentation MIoU of 0.878 during the debudding period, a mean pixel precision of 0.936, and a final bunch detection accuracy rate of 86%. Bunch detection was highly challenging during the harvest period, with a detection accuracy rate of 76% and a final overall bunch counting accuracy rate of 93.2%. Software was designed to estimate banana fruit weight during the harvest period. This research method provided a theoretical basis and experimental data support for automatic sterile bud removal and weight estimation for bananas.