Topological Structure and Semantic Information Transfer Network for Cross-Scene Hyperspectral Image Classification

• Published in: IEEE transaction on neural networks and learning systems Vol. 34; no. 6; pp. 2817 - 2830
• Main Authors: Zhang, Yuxiang, Li, Wei, Zhang, Mengmeng, Qu, Ying, Tao, Ran, Qi, Hairong
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.06.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptation models; Alignment; Artificial neural networks; Classification; Convolutional neural networks; Cross-scene; Data mining; Data structures; distribution alignment; domain adaption; Domains; Feature extraction; graph alignment; graph convolutional network (GCN); Graph theory; hyperspectral image (HSI) classification; Hyperspectral imaging; Image classification; Information transfer; Land cover; Land use; Neural networks; Semantics; Task analysis; Topology; Training
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2021.3109872

Domain adaptation techniques have been widely applied to the problem of cross-scene hyperspectral image (HSI) classification. Most existing methods use convolutional neural networks (CNNs) to extract statistical features from data and often neglect the potential topological structure information between different land cover classes. CNN-based approaches generally only model the local spatial relationships of the samples, which largely limits their ability to capture the nonlocal topological relationship that would better represent the underlying data structure of HSI. In order to make up for the above shortcomings, a Topological structure and Semantic information Transfer network (TSTnet) is developed. The method employs the graph structure to characterize topological relationships and the graph convolutional network (GCN) that is good at processing for cross-scene HSI classification. In the proposed TSTnet, graph optimal transmission (GOT) is used to align topological relationships to assist distribution alignment between the source domain and the target domain based on the maximum mean difference (MMD). Furthermore, subgraphs from the source domain and the target domain are dynamically constructed based on CNN features to take advantage of the discriminative capacity of CNN models that, in turn, improve the robustness of classification. In addition, to better characterize the correlation between distribution alignment and topological relationship alignment, a consistency constraint is enforced to integrate the output of CNN and GCN. Experimental results on three cross-scene HSI datasets demonstrate that the proposed TSTnet performs significantly better than some state-of-the-art domain-adaptive approaches. The codes will be available from the website: https://github.com/YuxiangZhang-BIT/IEEE_TNNLS_TSTnet .