A Deep Reinforcement Learning Chatbot (Short Version)

• Published in: arXiv.org
• Main Authors: Serban, Iulian V, Chinnadhurai Sankar, Germain, Mathieu, Zhang, Saizheng, Lin, Zhouhan, Subramanian, Sandeep, Kim, Taesup, Pieper, Michael, Chandar, Sarath, Ke, Nan Rosemary, Rajeswar, Sai, de Brebisson, Alexandre, Sotelo, Jose M R, Suhubdy, Dendi, Michalski, Vincent, Nguyen, Alexandre, Pineau, Joelle, Bengio, Yoshua
• Format: Paper Journal Article
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 20.01.2018
• Subjects: Algorithms; Computer Science - Artificial Intelligence; Computer Science - Computation and Language; Computer Science - Learning; Computer Science - Neural and Evolutionary Computing; Machine learning; Neural networks; Speech recognition; Statistics - Machine Learning
• ISSN: 2331-8422
• DOI: 10.48550/arxiv.1801.06700

We present MILABOT: a deep reinforcement learning chatbot developed by the Montreal Institute for Learning Algorithms (MILA) for the Amazon Alexa Prize competition. MILABOT is capable of conversing with humans on popular small talk topics through both speech and text. The system consists of an ensemble of natural language generation and retrieval models, including neural network and template-based models. By applying reinforcement learning to crowdsourced data and real-world user interactions, the system has been trained to select an appropriate response from the models in its ensemble. The system has been evaluated through A/B testing with real-world users, where it performed significantly better than other systems. The results highlight the potential of coupling ensemble systems with deep reinforcement learning as a fruitful path for developing real-world, open-domain conversational agents.