Structured self-attention architecture for graph-level representation learning

• Published in: Pattern recognition Vol. 100; p. 107084
• Main Authors: Fan, Xiaolong, Gong, Maoguo, Xie, Yu, Jiang, Fenlong, Li, Hao
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.04.2020
• Subjects: Graph classification; Graph neural networks; Neural self-attention mechanism; Neural self-attention mechanism; Graph neural networks; Graph classification
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2019.107084

•We develop a Structured Self-attention Architecture for graph-level representation. Compared with previous GNN variants, the architecture proposed in this paper can focus more effectively on the influential part of the input graph.•The proposed architecture’s readout can be incorporated into any existing node-level GNNs and provide effective features for graph-level representation. Compared with pooling readout, the proposed architecture shows its superior performance.•Extensive experiments on two types of graph datasets illustrate the effectiveness of our proposed architecture. Combining our architecture’s readout with popular graph convolutional networks have validated the feasibility of structured self-attention. Recently, graph neural networks (GNNs) have shown to be effective in learning representative graph features. However, current pooling-based strategies for graph classification lack efficient utilization of graph representation information in which each node and layer have the same contribution to the output of graph-level representation. In this paper, we develop a novel architecture for extracting an effective graph representation by introducing structured multi-head self-attention in which the attention mechanism consists of three different forms, i.e., node-focused, layer-focused and graph-focused. In order to make full use of the information of graphs, the node-focused self-attention firstly aggregates neighbor node features with a scaled dot-product manner, and then the layer-focused and graph-focused self-attention serve as readout module to measure the importance of different nodes and layers to the model’s output. Moreover, it is able to improve the performance on graph classification tasks by combining these two self-attention mechanisms with base node-level GNNs. The proposed Structured Self-attention Architecture is evaluated on two kinds of graph benchmarks: bioinformatics datasets and social network datasets. Extensive experiments have demonstrated superior performance improvement to existing methods on predictive accuracy.