Teaching Robots Generalizable Hierarchical Tasks Through Natural Language Instruction

• Published in: IEEE robotics and automation letters Vol. 2; no. 1; pp. 201 - 208
• Main Authors: Suddrey, Gavin, Lehnert, Christopher, Eich, Markus, Maire, Frederic, Roberts, Jonathan
• Format: Journal Article
• Language: English
• Published: IEEE 01.01.2017
• Subjects: Autonomous agents; Grounding; Natural languages; Planning; Robot kinematics; scheduling and coordination; social human-robot interaction; Training
• ISSN: 2377-3766
• DOI: 10.1109/LRA.2016.2588584

Natural language provides a convenient means of communicating information, and as such, is an ideal medium for enabling nonexpert users to teach robots novel tasks. However, in order to take advantage of natural language, a series of challenges must first be overcome. These challenges include the need to a) generalize learnt tasks to novel scenarios without retraining, b) resolve problems encountered during task execution, and c) derive implicit information from knowledge about the domain. To solve these challenges, this paper presents a novel approach to learning complex hierarchical tasks through natural language instruction, which not only allows learnt tasks to be generalized to novel situations without the need for retraining, but also enables an agent to derive implicit information from domain knowledge. Additionally, the approach presented in this paper enables the agent to infer task properties, such as preconditions and effects, directly from the explanation of the task flow. The authors validate the approach by demonstrating an implementation of the algorithms both on a simulated agent, as well as a Baxter robot. In each case, the agent is provided with a small set of primitive tasks for manipulating its workspace. From these primitives, the authors demonstrate the ability to teach the agent increasing complex tasks, including tasks of table cleaning, solely through natural language instructions.