Skip-Connected Covariance Network for Remote Sensing Scene Classification

• Published in: IEEE transaction on neural networks and learning systems Vol. 31; no. 5; pp. 1461 - 1474
• Main Authors: He, Nanjun, Fang, Leyuan, Li, Shutao, Plaza, Javier, Plaza, Antonio
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.05.2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Aggregates; Artificial neural networks; Classification; Computational modeling; Covariance; Covariance pooling; Datasets; deep neural network; Feature extraction; Feature maps; Learning; Learning systems; Mathematical models; multi-layer feature; Neural networks; Parameters; Remote sensing; scene recognition; Training
• ISSN: 2162-237X; 2162-2388
• DOI: 10.1109/TNNLS.2019.2920374

This paper proposes a novel end-to-end learning model, called skip-connected covariance (SCCov) network, for remote sensing scene classification (RSSC). The innovative contribution of this paper is to embed two novel modules into the traditional convolutional neural network (CNN) model, i.e., skip connections and covariance pooling. The advantages of newly developed SCCov are twofold. First, by means of the skip connections, the multi-resolution feature maps produced by the CNN are combined together, which provides important benefits to address the presence of large-scale variance in RSSC data sets. Second, by using covariance pooling, we can fully exploit the second-order information contained in such multi-resolution feature maps. This allows the CNN to achieve more representative feature learning when dealing with RSSC problems. Experimental results, conducted using three large-scale benchmark data sets, demonstrate that our newly proposed SCCov network exhibits very competitive or superior classification performance when compared with the current state-of-the-art RSSC techniques, using a much lower amount of parameters. Specifically, our SCCov only needs 10% of the parameters used by its counterparts.