Automatic greenhouse insect pest detection and recognition based on a cascaded deep learning classification method

• Published in: Journal of applied entomology (1986) Vol. 145; no. 3; pp. 206 - 222
• Main Authors: Rustia, Dan Jeric Arcega, Chao, Jun‐Jee, Chiu, Lin‐Ya, Wu, Ya‐Fang, Chung, Jui‐Yung, Hsu, Ju‐Chun, Lin, Ta‐Te
• Format: Journal Article
• Language: English
• Published: Berlin Wiley Subscription Services, Inc 01.04.2021
• Subjects: Agricultural practices; Aleyrodidae; Algorithms; Artificial neural networks; automation; Classification; Classifiers; convolutional neural network; data collection; Datasets; Deep learning; Diptera; Drosophilidae; entomology; Farm buildings; Greenhouses; Herbivores; Image acquisition; Image classification; insect pests; insect recognition; Insects; Inspection; Integrated pest management; Neural networks; object detection; Object recognition; Pest control; Pests; Sciaridae; sticky paper trap; sticky traps; Thripidae; Traps
• ISSN: 0931-2048; 1439-0418
• DOI: 10.1111/jen.12834

Inspection of insect sticky paper traps is an essential task for an effective integrated pest management (IPM) programme. However, identification and counting of the insect pests stuck on the traps is a very cumbersome task. Therefore, an efficient approach is needed to alleviate the problem and to provide timely information on insect pests. In this research, an automatic method for the multi‐class recognition of small‐size greenhouse insect pests on sticky paper trap images acquired by wireless imaging devices is proposed. The developed algorithm features a cascaded approach that uses a convolutional neural network (CNN) object detector and CNN image classifiers, separately. The object detector was trained for detecting objects in an image, and a CNN classifier was applied to further filter out non‐insect objects from the detected objects in the first stage. The obtained insect objects were then further classified into flies (Diptera: Drosophilidae), gnats (Diptera: Sciaridae), thrips (Thysanoptera: Thripidae) and whiteflies (Hemiptera: Aleyrodidae), using a multi‐class CNN classifier in the second stage. Advantages of this approach include flexibility in adding more classes to the multi‐class insect classifier and sample control strategies to improve classification performance. The algorithm was developed and tested for images taken by multiple wireless imaging devices installed in several greenhouses under natural and variable lighting environments. Based on the testing results from long‐term experiments in greenhouses, it was found that the algorithm could achieve average F1‐scores of 0.92 and 0.90 and mean counting accuracies of 0.91 and 0.90, as tested on a separate 6‐month image data set and on an image data set from a different greenhouse, respectively. The proposed method in this research resolves important problems for the automated recognition of insect pests and provides instantaneous information of insect pest occurrences in greenhouses, which offers vast potential for developing more efficient IPM strategies in agriculture.