Subgraph-Based Attention Network for Multi-Hop Question Answering

• Published in: 2024 International Joint Conference on Neural Networks (IJCNN) pp. 1 - 9
• Main Authors: Gong, Chuanyang, Wei, Zhihua, Wang, Xinpeng, Wang, Rui, Li, Yu, Zhu, Ping
• Format: Conference Proceeding
• Language: English
• Published: IEEE 30.06.2024
• Subjects: Cognition; Constrained Reasoning; Graph Neural Network; Message passing; Multi-hop question answering; Process control; Question answering (information retrieval); Reinforcement learning; Semantics; Spread spectrum communication; Subgraph Decomposition
• ISSN: 2161-4407
• DOI: 10.1109/IJCNN60899.2024.10650851

Multi-hop question answering (QA) requires the model to integrate multiple evidential fragments scattered in the document to infer the answer. At present, while graph neural network-based (GNN-based) approaches have exhibited promising performance in multi-hop QA tasks, some underlying issues have also emerged. The first is that GNN is insensitive to crucial dependencies between "levels" in the graph. The second is that reasoning messages on the graph require multiple hops to reach the answer entity, and this jumping between nodes induced by reasoning significantly disperses attention. To address these issues, we propose a novel model of Subgraph-based Attention Network (SuAN) for multi-hop QA, which achieves controlled message passing in GNN through subgraph decomposition and constrained reasoning order, more in line with the multi-hop reasoning process. In SuAN, we decompose the graph into a series of subgraphs composed of first-order neighboring nodes based on semantics. Subsequently, we update node representations on these subgraphs according to a pre-defined reasoning order. This approach not only narrows the gap between the human reasoning process and the message passing mechanism of GNN but also enables the full exploitation of the implicit or explicit hierarchical property inherent in the graph structure. The experimental results on HotpotQA demonstrate the effectiveness of the proposed approach and help the baseline model achieve significant improvements.