Estimation of Greenhouse Lettuce Growth Indices Based on a Two-Stage CNN Using RGB-D Images

• Published in: Sensors (Basel, Switzerland) Vol. 22; no. 15; p. 5499
• Main Authors: Gang, Min-Seok, Kim, Hak-Jin, Kim, Dong-Wook
• Format: Journal Article
• Language: English
• Published: Switzerland MDPI AG 23.07.2022; MDPI
• Subjects: Cameras; convolutional neural network (CNN); Crops; Datasets; Deep learning; greenhouse; Greenhouses; growth estimation; growth index; growth monitoring; Humans; Lactuca - classification; Lactuca - growth & development; Lettuce; Neural networks; Neural Networks, Computer; Performance evaluation; Plant Leaves - growth & development; Plant Roots - growth & development; Processing speed; Regression analysis; United States > US; growth estimation; stereo camera; convolutional neural network (CNN); growth index; greenhouse; lettuce; red, green, blue, and depth (RGB-D); growth monitoring
• DOI: 10.3390/s22155499
• ISSN: 1424-8220

Growth indices can quantify crop productivity and establish optimal environmental, nutritional, and irrigation control strategies. A convolutional neural network (CNN)-based model is presented for estimating various growth indices (i.e., fresh weight, dry weight, height, leaf area, and diameter) of four varieties of greenhouse lettuce using red, green, blue, and depth (RGB-D) data obtained using a stereo camera. Data from an online autonomous greenhouse challenge (Wageningen University, June 2021) were employed in this study. The data were collected using an Intel RealSense D415 camera. The developed model has a two-stage CNN architecture based on ResNet50V2 layers. The developed model provided coefficients of determination from 0.88 to 0.95, with normalized root mean square errors of 6.09%, 6.30%, 7.65%, 7.92%, and 5.62% for fresh weight, dry weight, height, diameter, and leaf area, respectively, on unknown lettuce images. Using red, green, blue (RGB) and depth data employed in the CNN improved the determination accuracy for all five lettuce growth indices due to the ability of the stereo camera to extract height information on lettuce. The average time for processing each lettuce image using the developed CNN model run on a Jetson SUB mini-PC with a Jetson Xavier NX was 0.83 s, indicating the potential for the model in fast real-time sensing of lettuce growth indices.