Multi-Hop Question Generation Using Hierarchical Encoding-Decoding and Context Switch Mechanism

• Published in: Entropy (Basel, Switzerland) Vol. 23; no. 11; p. 1449
• Main Authors: Ji, Tianbo, Lyu, Chenyang, Cao, Zhichao, Cheng, Peng
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 31.10.2021; MDPI
• Subjects: Autoregressive models; Coders; Context; Datasets; Encoding-Decoding; hierarchical encoding-decoding; Labeling; Linguistics; Middle Ages; multi-hop question generation; Natural language processing; Neural networks; Newton, Isaac (1642-1727); Questions; Representations; Semantics; syntactic knowledge
• ISSN: 1099-4300; 1099-4300
• DOI: 10.3390/e23111449

Neural auto-regressive sequence-to-sequence models have been dominant in text generation tasks, especially the question generation task. However, neural generation models suffer from the global and local semantic semantic drift problems. Hence, we propose the hierarchical encoding–decoding mechanism that aims at encoding rich structure information of the input passages and reducing the variance in the decoding phase. In the encoder, we hierarchically encode the input passages according to its structure at four granularity-levels: [word, chunk, sentence, document]-level. Second, we progressively select the context vector from the document-level representations to the word-level representations at each decoding time step. At each time-step in the decoding phase, we progressively select the context vector from the document-level representations to word-level. We also propose the context switch mechanism that enables the decoder to use the context vector from the last step when generating the current word at each time-step.It provides a means of improving the stability of the text generation process during the decoding phase when generating a set of consecutive words. Additionally, we inject syntactic parsing knowledge to enrich the word representations. Experimental results show that our proposed model substantially improves the performance and outperforms previous baselines according to both automatic and human evaluation. Besides, we implement a deep and comprehensive analysis of generated questions based on their types.