Remote Sensing Image Scene Classification Using Multiscale Feature Fusion Covariance Network With Octave Convolution

• Published in: IEEE transactions on geoscience and remote sensing Vol. 60; pp. 1 - 14
• Main Authors: Bai, Lin, Liu, Qingxin, Li, Cuiling, Ye, Zhen, Hui, Meng, Jia, Xiuping
• Format: Journal Article
• Language: English
• Published: New York IEEE 2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Classification; Convolution; Convolutional neural networks (CNNs); Covariance; covariance pooling; Data mining; Feature extraction; Frequency dependence; Fuses; Image classification; Information processing; Modules; multiscale feature fusion (MF²); octave convolution (Oct Conv); Remote sensing; scene classification; Semantics; Statistical methods; Task analysis
• ISSN: 0196-2892; 1558-0644
• DOI: 10.1109/TGRS.2022.3160492

In remote sensing scene classification (RSSC), features can be extracted with different spatial frequencies where high-frequency features usually represent detailed information and low-frequency features usually represent global structures. However, it is challenging to extract meaningful semantic information for RSSC tasks by just utilizing high- or low-frequency features. The spatial composition of remote sensing images (RSIs) is more complex than that of natural images, and the scales of objects vary significantly. In this article, a multiscale feature fusion covariance network (MF 2 CNet) with octave convolution (Oct Conv) is proposed, which can extract multifrequency and multiscale features from RSIs. First, the multifrequency feature extraction (MFE) module is used to obtain fine-grained frequency features by Oct Conv. Then, the features of different layers in MF 2 CNet are fused by the multiscale feature fusion (MF 2 ) module. Finally, instead of using global average pooling (GAP), global covariance pooling (GCP) extracts high-order information from RSIs to capture richer statistics of deep features. In the proposed MF 2 CNet, the obtained multifrequency and multiscale features can effectively improve the performance of CNNs. Experimental results on four public RSI datasets show that MF 2 CNet has advantages in RSSC over current state-of-the-art methods. The source codes of this method can be found at https://github.com/liuqingxin-chd/MF2CNet .