Path detection for autonomous traveling in orchards using patch-based CNN

• Published in: Computers and electronics in agriculture Vol. 175; p. 105620
• Main Authors: Kim, Wan-Soo, Lee, Dae-Hyun, Kim, Yong-Joo, Kim, Taehyeong, Hwang, Rok-Yeun, Lee, Hyo-Jai
• Format: Journal Article
• Language: English
• Published: Amsterdam Elsevier B.V 01.08.2020; Elsevier BV
• Subjects: Area segmentation; Artificial neural networks; Autonomous traveling; Citrus orchard; Farms; Image classification; Image segmentation; Machine vision; Orchards; Patch-based CNN; Performance evaluation; Sliding; Target detection; Target recognition; Patch-based CNN; Citrus orchard; Area segmentation; Machine vision; Autonomous traveling
• ISSN: 0168-1699; 1872-7107
• DOI: 10.1016/j.compag.2020.105620

•First study to use deep learning in image-patch-based path detection in agriculture.•Orchard paths can be segmented with CNN trained with small-scale datasets.•Applicability to various farm machines or robotic systems with simple configuration. This paper proposes a novel and efficient patch-based approach for autonomous path detection in semi-structured environments such as orchards. The proposed approach can segment a perspective path area in the frontal scene and is expected to be applicable to various types of area detection tasks in agriculture. In the proposed approach, a patch-based convolutional neural network (CNN) is used for image classification to achieve path area segmentation, which involves cropping image patches from an input image by using a sliding window, generating a path score map by using a four-layer CNN for tree and path classification, path area segmentation, and target path detection by using boundary line determination. Results show that the maximum intersection over union (IoU) is approximately 0.81 for path area localization and the average lateral and angular errors are 0.051 and 7.8°, respectively. The performance of patch-based path detection depends on the patch size of the sliding window. Hence, the path detection performance is evaluated in terms of the patch size, and a patch size of 96 × 96 shows the best performance, with a classification accuracy of 0.93, IoU of 0.75, and processing time of 0.111 s. In addition, the proposed approach was verified by applying the approach to various images including curved paths. The results indicate that the performance of the proposed patch-based approach for path detection is comparable to that of previous approaches. Moreover, an autonomous farm robot can be easily developed using the proposed technique with a simple hardware configuration.