Deep learning based multi-temporal crop classification

• Published in: Remote sensing of environment Vol. 221; pp. 430 - 443
• Main Authors: Zhong, Liheng, Hu, Lina, Zhou, Hang
• Format: Journal Article
• Language: English
• Published: New York Elsevier Inc 01.02.2019; Elsevier BV
• Subjects: Accuracy; Agronomy; Artificial neural network; automation; California; Classification; Classifiers; Convolutional neural network; Crop classification; Crops; Data processing; Deep learning; Dependence; equations; Farming systems; Feature extraction; irrigation; Irrigation systems; Landsat; Landsat satellites; Long short-term memory; Multi-temporal classification; Remote sensing; Representations; seasonal variation; summer; Support vector machines; Temporal variations; Time series; time series analysis; Vegetation index; California; Deep learning; Crop classification; Multi-temporal classification; Artificial neural network; Convolutional neural network; Landsat
• ISSN: 0034-4257; 1879-0704
• DOI: 10.1016/j.rse.2018.11.032

This study aims to develop a deep learning based classification framework for remotely sensed time series. The experiment was carried out in Yolo County, California, which has a very diverse irrigated agricultural system dominated by economic crops. For the challenging task of classifying summer crops using Landsat Enhanced Vegetation Index (EVI) time series, two types of deep learning models were designed: one is based on Long Short-Term Memory (LSTM), and the other is based on one-dimensional convolutional (Conv1D) layers. Three widely-used classifiers were also tested for comparison, including a gradient boosting machine called XGBoost, Random Forest, and Support Vector Machine. Although LSTM is widely used for sequential data representation, in this study its accuracy (82.41%) and F1 score (0.67) were the lowest among all the classifiers. Among non-deep-learning classifiers, XGBoost achieved the best result with 84.17% accuracy and an F1 score of 0.69. The highest accuracy (85.54%) and F1 score (0.73) were achieved by the Conv1D-based model, which mainly consists of a stack of Conv1D layers and an inception module. The behavior of the Conv1D-based model was inspected by visualizing the activation on different layers. The model employs EVI time series by examining shapes at various scales in a hierarchical manner. Lower Conv1D layers of the optimized model capture small scale temporal variations, while upper layers focus on overall seasonal patterns. Conv1D layers were used as an embedded multi-level feature extractor in the classification model which automatically extracts features from input time series during training. The automated feature extraction reduces the dependency on manual feature engineering and pre-defined equations of crop growing cycles. This study shows that the Conv1D-based deep learning framework provides an effective and efficient method of time series representation in multi-temporal classification tasks. •Deep neural networks were developed for crop classification.•Deep neural network achieved 85.54% accuracy and an F1 score of 0.73.•The best non-deep-learning classifier achieved 84.17% accuracy and an F1 score of 0.69.•One-dimensional convolutional neural network was used as automated temporal feature extractor.•One-dimensional convolutional neural network identifies complex seasonal dynamics of economic crops.