Multi-Layers Feature Fusion of Convolutional Neural Network for Scene Classification of Remote Sensing

• Published in: IEEE access Vol. 7; pp. 121685 - 121694
• Main Authors: Ma, Chenhui, Mu, Xiaodong, Sha, Dexuan
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; Classification; Computer vision; Convolution; convolutional neural network; Data mining; Datasets; Encoding; Feature extraction; Feature maps; Modules; Multi-layer feature fusion; Multilayers; Neural networks; Remote sensing; remote sensing image; scene classification; Semantics; Shape
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2019.2936215

Remote sensing scene classification is still a challenging task in remote sensing applications. How to effectively extract features from a dataset with limited scale is crucial for improvement of scene classification. Recently, convolutional neural network (CNN) performs impressively in different fields of computer vision and has been used for remote sensing. However, most works focus on the feature maps of the last convolution layer and pay little attention to the benefits of additional layers. In fact, the feature information hidden in different layers has potential for feature discrimination capacity. The most attention of this work is how to explore the potential of multiple layers from a CNN model. Therefore, this paper proposes multi-layers feature fusion based on CNN and designs a fusion module to solve relevant issues of fusion. In this module, firstly, all the feature maps are transformed to match sizes mutually due to infeasible fusion of feature maps with different scales; then, two fusion methods are introduced to integrate feature maps from different layers instead of the last convolution layer only; finally, the fusion of features are delivered to the next layer or classifier as the routine CNN does. The experimental results show that the suggested methods achieve promising performance on public datasets.