A Memory-Based Sentence Split and Rephrase Model with Multi-task Training

• Published in: Neural Information Processing Vol. 12532; pp. 643 - 654
• Main Authors: Fan, Xiaoning, Liu, Yiding, Liu, Gongshen, Su, Bo
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2020; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Memory gate; Multi-task; Split and rephrase; VAE
• ISBN: 9783030638290; 3030638294
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-030-63830-6_54

The task of sentence split and rephrase refers to breaking down a complex sentence into some simple sentences with the same semantic information, which is a basic preprocess method for simplification in many natural language processing (NLP) fields. Previous works mainly focus on applying conventional sequence-to-sequence models into this task, which fails to capture relations between entities and lacks memory of the decoded parts, and thus causes duplication of generated subsequences and confuses the relationship between subjects and objects. In this paper, we introduce a memory-based Transformer model with multi-task training to improve the accuracy of the sentence information obtained by the encoder. To enrich the semantic representation of the model, we further incorporated a conditional Variational Autoencoder (VAE) component to our model. Through experiments on the WebSplit-v1.0 benchmark dataset, results show that our proposed model outperforms other state-of-the-art baselines from both BLEU and human evaluations.