Reinforcement learning for few-shot text generation adaptation

This paper proposes a novel method based on reinforcement learning (RL) to control the generation model in adapting to new domains with limited samples. To avoid the problem of overfitting, the method combines maximum likelihood estimation (MLE) with RL process to improve the sample utilization rate...

Full description

Saved in:
Bibliographic Details
Published inNeurocomputing (Amsterdam) Vol. 558; p. 126689
Main Authors Cheng, Pengsen, Dai, Jinqiao, Liu, Jiamiao, Liu, Jiayong, Jia, Peng
Format Journal Article
LanguageEnglish
Published Elsevier B.V 14.11.2023
Subjects
Domain adaption
Few-shot learning
Reinforcement learning
Text generation
Text generation
Few-shot learning
Reinforcement learning
Domain adaption
Online AccessGet full text

Cover

More Information
Summary:This paper proposes a novel method based on reinforcement learning (RL) to control the generation model in adapting to new domains with limited samples. To avoid the problem of overfitting, the method combines maximum likelihood estimation (MLE) with RL process to improve the sample utilization rate and reduce the sample requirement of RL. The training process is divided into two parts: pre-training and fine-tuning, to effectively express the semantic of the target domain. In order to ensure the robustness of the reward function, adversarial training is introduced. A new measurement called “Net Accuracy” is proposed to better evaluate the domain relevance of the generated text and eliminate the problem of inaccurate domain relevance measurement caused by overfitting and generating a large amount of duplicate text. Finally, experimental results show the effectiveness and superiority of the proposed method in five target domains.
ISSN:0925-2312
1872-8286
DOI:10.1016/j.neucom.2023.126689