PiCovS: Pixel-Level With Covariance Pooling Feature and Superpixel-Level Feature Fusion for Hyperspectral Image Classification

• Published in: IEEE transactions on geoscience and remote sensing Vol. 61; pp. 1 - 20
• Main Authors: Nartey, Obed Tettey, Sarpong, Kwabena, Addo, Daniel, Rao, Yunbo, Qin, Zhiguang
• Format: Journal Article
• Language: English
• Published: New York IEEE 2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accuracy; Algorithms; Artificial neural networks; Classification; Clustering; Computational efficiency; Computational modeling; Convolutional neural network (CNN); Convolutional neural networks; Covariance; covariance pooling; Data models; Feature extraction; feature fusion; graph convolutional network (GCN); Harnesses; hyperspectral image (HSI) classification; Hyperspectral imaging; Image classification; Iterative methods; Neural networks; Nodes; Pixels; Receptive field; Representations
• ISSN: 0196-2892; 1558-0644
• DOI: 10.1109/TGRS.2023.3322641

In hyperspectral image (HSI) classification, convolutional neural networks (CNNs) have exhibited exceptional performance, owing to their hierarchical nonlinear modeling. However, their fixed square receptive field constrains their ability to effectively handle irregular image regions. Graph convolutional networks (GCNs) have been introduced to learn irregular regions through correlations between adjacent pixels modeled as superpixel-based nodes, yet they lack pixel-level information. We propose a novel approach Pixel-level with Covariance Pooling feature and Superpixel-level feature Fusion for HSI Classification (PiCovS). Our method harnesses complementary spectral-spatial features at both pixel-level and superpixel-level to capture characteristics of both small-scale regular and large-scale irregular regions. We introduce a hybrid network that integrates and propagates features between image-level pixels and graph-level nodes using a graph encoder-decoder, effectively reconciling the differences between regular CNN and irregular GCN data representations. To enhance superpixel boundary learning, we modify the manifold simple linear iterative clustering (M-SLIC) algorithm by incorporating texture feature information, resulting in refined superpixel representations. In addition, we propose a novel covariance pooling mechanism with an attention mechanism within the CNN branch, enabling the capturing and utilization of holistic HSI information along spectral and spatial dimensions by exploiting second-order statistics throughout the network. Our comprehensive experiments showcase the efficiency and robustness of the proposed framework, achieving an impressive overall accuracy of 99.84%, 99.97%, 99.98%, and 81.96% on the Indian Pines, University of Pavia, Salinas, and Houston University datasets, respectively. Remarkably, PiCovS excels even with limited training samples, outperforming other state-of-the-art methods in accuracy.