Spatial-Spectral Involution MLP Network for Hyperspectral Image Classification

• Published in: IEEE journal of selected topics in applied earth observations and remote sensing Vol. 15; pp. 9293 - 9310
• Main Authors: Shao, Yihao, Liu, Jianjun, Yang, Jinlong, Wu, Zebin
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Classification; Convolution; Data mining; Distribution; Feature extraction; Feature maps; Hyperspectral image (HSI) classification; Hyperspectral imaging; Image classification; Information retrieval; involution; Kernel; multilayer perceptron (MLP) like model; Multilayer perceptrons; Receptive field; Spatial data; Spatial distribution; Task analysis; Transformers
• ISSN: 1939-1404; 2151-1535
• DOI: 10.1109/JSTARS.2022.3216590

Recently, more and more multilayer perceptron (MLP) like models have been proposed. Among them, CycleMLP is good at dense feature prediction tasks, which is potentially useful for hyperspectral image (HSI) classification. However, the receptive field of CycleMLP tends to be cross-shaped, which will lead to insufficient spatial information extraction. Additionally, most of the HSI classification methods only use information from single HSI data. Lack of diversity in the features of a single modality limits classification performance. To address these issues, a novel spatial-spectral involution MLP network (SSIN) is proposed for HSI classification. SSIN contains two paths for extracting different kinds of information, namely the image path and the coordinate path. In the image path, we combine the MLP structure with the involution operation and propose involution MLP (InvoMLP). It obtains the spatial kernel weights corresponding to each pixel individually, thus improving the spatial interaction capability. At the same time, InvoMLP has the same receptive field range as conventional convolution, i.e., a rectangular receptive field. In the coordinate path, we build a lightweight module for extracting information. Unlike the information of images, the coordinates are intuitive information about the location distribution. Considering that the coordinate information contains the global spatial distribution of HSI, fusing it with the image information could improve long-distance dependencies of feature maps. Experimental results on four HSI datasets illustrate that SSIN can outperform some state-of-the-art methods.