Deep convolutional neural networks for rice grain yield estimation at the ripening stage using UAV-based remotely sensed images

• Published in: Field crops research Vol. 235; pp. 142 - 153
• Main Authors: Yang, Qi, Shi, Liangsheng, Han, Jinye, Zha, Yuanyuan, Zhu, Penghui
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.04.2019
• Subjects: CNN; Deep learning; Rice crop; UAV; Yield estimation; Deep learning; CNN; UAV; Yield estimation; Rice crop
• ISSN: 0378-4290; 1872-6852
• DOI: 10.1016/j.fcr.2019.02.022

•The ability of CNN to estimate rice grain yield using UAV images is investigated.•The correlation between VI and rice grain yield is low at the ripening stage.•The proposed CNN provides robust yield forecast throughout the ripening stage.•RGB images dominate the network training at the ripening stage of paddy rice.•A more robust network can be trained by RGB data from late stage. Forecasting rice grain yield prior to harvest is essential for crop management, food security evaluation, food trade, and policy-making. Many successful applications have been made in crop yield estimation using remotely sensed products, such as vegetation index (VI) from multispectral imagery. However, VI-based approaches are only suitable for estimating rice grain yield at the middle stage of growth but have limited capability at the ripening stage. In this study, an efficient convolutional neural network (CNN) architecture was proposed to learn the important features related to rice grain yield from low-altitude remotely sensed imagery. In one major region for rice cultivation of Southern China, a 160-hectare site with over 800 management units was chosen to investigate the ability of CNN in rice grain yield estimation. The datasets of RGB and multispectral images were obtained by a fixed-wing, unmanned aerial vehicle (UAV), which was mounted with a digital camera and multispectral sensors. The network was trained with different datasets and compared against the traditional vegetation index-based method. In addition, the temporal and spatial generality of the trained network was investigated. The results showed that the CNNs trained by RGB and multispectral datasets perform much better than VIs-based regression model for rice grain yield estimation at the ripening stage. The RGB imagery of very high spatial resolution contains important spatial features with respect to grain yield distribution, which can be learned by deep CNN. The results highlight the promising potential of deep convolutional neural networks for rice grain yield estimation with excellent spatial and temporal generality, and a wider time window of yield forecasting.