Multimodal heterogeneous graph attention network

• Published in: Neural computing & applications Vol. 35; no. 4; pp. 3357 - 3372
• Main Authors: Jia, Xiangen, Jiang, Min, Dong, Yihong, Zhu, Feng, Lin, Haocai, Xin, Yu, Chen, Huahui
• Format: Journal Article
• Language: English
• Published: London Springer London 01.02.2023; Springer Nature B.V
• Subjects: Actors; Apexes; Artificial Intelligence; Classification; Clustering; Computational Biology/Bioinformatics; Computational Science and Engineering; Computer Science; Data Mining and Knowledge Discovery; Datasets; Deep learning; Graph representations; Graphs; Heterogeneity; Image Processing and Computer Vision; Neighborhoods; Networks; Neural networks; Original Article; Probability and Statistics in Computer Science; Semantics; Subject specialists; Visualization; Network representation learning; Heterogeneous networks; Graph convolutional networks; Multimodal; Clustering
• ISSN: 0941-0643; 1433-3058
• DOI: 10.1007/s00521-022-07862-6

The real world involves many graphs and networks that are essentially heterogeneous, in which various types of relations connect multiple types of vertices. With the development of information networks, node features can be described by data of different modalities, resulting in multimodal heterogeneous graphs. However, most existed methods can only handle unimodal heterogeneous graphs. Moreover, most existing heterogeneous graph mining methods are based on meta-paths that depend on domain experts for modeling. In this paper, we propose a novel multimodal heterogeneous graph attention network (MHGAT) to address these problems. Specifically, we exploit edge-level aggregation to capture graph heterogeneity information to achieve more informative representations adaptively. Further, we use the modality-level attention mechanism to obtain multimodal fusion information. Because plain graph convolutional networks can not capture higher-order neighborhood information, we utilize the residual connection and the dense connection access to obtain it. Extensive experimental results show that the MHGAT outperforms state-of-the-art baselines on three datasets for node classification, clustering, and visualization tasks.