Collaborative Classification of Hyperspectral and LiDAR Date Based on Dynamic Multiple Fractional Fourier Domains Fusion

• Published in: IEEE transactions on geoscience and remote sensing Vol. 63; pp. 1 - 16
• Main Authors: Qin, Boao, Shou Feng, Zhao, Chunhui, Li, Wei, Tao, Ran
• Format: Journal Article
• Language: English
• Published: New York The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 01.01.2025
• Subjects: Aggregation; Algorithms; Classification; Coders; Frequency dependence; Hyperspectral imaging; Information processing; Land cover; Lidar; Machine learning; Upper bounds
• ISSN: 0196-2892; 1558-0644
• DOI: 10.1109/TGRS.2025.3579433

Collaboratively utilizing the complementary information provided by hyperspectral imagery and light detection and ranging (LiDAR) data will extend the applications associated with land cover recognition and mapping. Existing joint classification algorithms mainly focus on learning complementary patterns in the pure spatial domain, while paying little attention to complementary cues in the spatial-frequency domain. The model’s expressive capability of these methods may be limited by an upper bound subject to the spatial domain. To fill this gap, a dynamic multiple fractional Fourier domains fusion (DMFraF) is proposed for the joint classification of hyperspectral and LiDAR data. First, to comprehensively learn the complementary patterns between hyperspectral image (HSI) and LiDAR data, we transform the features of two modalities into multiple fractional domains containing different spatial-frequency components for multimodal fusion. Second, to obtain the optimal representation from the multimodal features of multiple fractional domains, we propose a dynamic fusion scheme guided by the optimal transport (OT) technique, which can dynamically adjust the contributions from different fractional domains. Finally, to extract purer modality-specific features, we propose a channel aggregation Transformer encoder with channel aggregation Transformer ([Formula Omitted]AT encoder), to aggregate channel-wise features of central pixels into the spatial branch and compress interference from noisy surroundings. Extensive experiments and analysis on three hyperspectral and LiDAR datasets suggest the superiority of the proposed method.