A near real-time deep learning approach for detecting rice phenology based on UAV images

• Published in: Agricultural and forest meteorology Vol. 287; p. 107938
• Main Authors: Yang, Qi, Shi, Liangsheng, Han, Jingye, Yu, Jin, Huang, Kai
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 15.06.2020
• Subjects: China; Convolutional neural network; crop management; cultivation area; Deep learning; forests; harvest date; meteorology; neural networks; Oryza sativa; phenology; rice; Rice phenology detection; Shape model fitting; time series analysis; UAV; unmanned aerial vehicles; vegetation index; China; Deep learning; UAV; Convolutional neural network; Shape model fitting; Rice phenology detection
• DOI: 10.1016/j.agrformet.2020.107938
• ISSN: 0168-1923

•Using CNN to detect rice phenology by a mono-temporal imagery of UAV was investigated.•Shape-model-fitting method underperformed with short length of time-series data.•Integrating regional mean thermal time into CNN improve the detection accuracy.•The estimated harvest dates was in close agreement with the observation.•The proposed approach provides near real-time estimation of the principal phenological stages of rice. Near real-time crop phenology detection is essential for crop management, estimation of harvest time and yield estimation. Previous approaches to crop phenology detection have relied on time-series (multi-temporal) vegetation index (VI) data, and have included threshold-based, phenometrics-based and shape-model-fitting-based (SMF) methods. However, the performance of these methods depends on the duration and temporal resolution of the time-series data. In this study, we propose a new approach which identifies the principal growth stages of rice (Oryza sativa L.) directly from RGB images. Only a mono-temporal unmanned aerial vehicle (UAV) imagery was required for a large-area phenology detection via the well-trained network. An efficient convolutional neural network (CNN) architecture was designed to estimate rice phenology. The CNN incorporated spatial pyramid pooling (SPP), transfer learning and an auxiliary branch with external data. A total of 82 plots across a 160-hectare rice cultivation area of Southern China were selected to evaluate the proposed network. CNN predictions were ground truthed using rice phenology measurements taken from each plot throughout the growing season. Aerial data were collected using a fixed-wing UAV equipped with multispectral and RGB cameras. The performance of traditional SMF methods deteriorated when time-series VI data were of short duration. In contrast, the phenological stage estimated by the proposed network showed good agreement with ground observations, with a top-1 accuracy rate of 83.9% and mean absolute error (MAE) of 0.18. The spatial distribution of harvest dates for 627 plots in the study area were computed from the phenological stage estimates. The estimates matched well with the observed harvest dates. The results demonstrated the excellent performance of the proposed deep learning approach in near real-time phenology detection and harvest time estimation.