Learning Spectral-Spatial-Temporal Features via a Recurrent Convolutional Neural Network for Change Detection in Multispectral Imagery

• Published in: IEEE transactions on geoscience and remote sensing Vol. 57; no. 2; pp. 924 - 935
• Main Authors: Mou, Lichao, Bruzzone, Lorenzo, Zhu, Xiao Xiang
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.02.2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; Change detection; Computer architecture; Convolutional neural networks; Data mining; Dependence; Detection; Earth; Feature extraction; Frameworks; Harnesses; Image analysis; Image detection; Image processing; Imagery; long short-term memory (LSTM); Machine learning; multitemporal image analysis; Neural networks; recurrent convolutional neural network (ReCNN); Recurrent neural networks; Remote sensing; Representations; Spatial data; Spatial discrimination learning; Spectra; Task analysis; Temporal variations
• ISSN: 0196-2892; 1558-0644
• DOI: 10.1109/TGRS.2018.2863224

Change detection is one of the central problems in earth observation and was extensively investigated over recent decades. In this paper, we propose a novel recurrent convolutional neural network (ReCNN) architecture, which is trained to learn a joint spectral-spatial-temporal feature representation in a unified framework for change detection in multispectral images. To this end, we bring together a convolutional neural network and a recurrent neural network into one end-to-end network. The former is able to generate rich spectral-spatial feature representations, while the latter effectively analyzes temporal dependence in bitemporal images. In comparison with previous approaches to change detection, the proposed network architecture possesses three distinctive properties: 1) it is end-to-end trainable, in contrast to most existing methods whose components are separately trained or computed; 2) it naturally harnesses spatial information that has been proven to be beneficial to change detection task; and 3) it is capable of adaptively learning the temporal dependence between multitemporal images, unlike most of the algorithms that use fairly simple operation like image differencing or stacking. As far as we know, this is the first time that a recurrent convolutional network architecture has been proposed for multitemporal remote sensing image analysis. The proposed network is validated on real multispectral data sets. Both visual and quantitative analyses of the experimental results demonstrate competitive performance in the proposed mode.