A Knowledge-Grounded Dialog System Based on Pre-Trained Language Models

We present a knowledge-grounded dialog system developed for the ninth Dialog System Technology Challenge (DSTC9) Track 1 - Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access. We leverage transfer learning with existing language models to accomplish the tasks...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Zhang, Weijie, Chen, Jiaoxuan, Wu, Haipang, Wan, Sanhui, Li, Gongfeng
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 28.06.2021
Subjects
Knowledge management
Online AccessGet full text

Cover

More Information
Summary:We present a knowledge-grounded dialog system developed for the ninth Dialog System Technology Challenge (DSTC9) Track 1 - Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access. We leverage transfer learning with existing language models to accomplish the tasks in this challenge track. Specifically, we divided the task into four sub-tasks and fine-tuned several Transformer models on each of the sub-tasks. We made additional changes that yielded gains in both performance and efficiency, including the combination of the model with traditional entity-matching techniques, and the addition of a pointer network to the output layer of the language model.
ISSN:2331-8422