Learning High-Level Navigation Strategies via Inverse Reinforcement Learning: A Comparative Analysis

• Published in: AI 2016: Advances in Artificial Intelligence Vol. 9992; pp. 525 - 534
• Main Authors: Herman, Michael, Gindele, Tobias, Wagner, Jörg, Schmitt, Felix, Quignon, Christophe, Burgard, Wolfram
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2016; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Inverse Reinforcement Learning; Learning from demonstration; Reinforcement Learning; Simultaneous Estimation of Rewards and Dynamics
• ISBN: 3319501267; 9783319501260
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-319-50127-7_45

With an increasing number of robots acting in populated environments, there is an emerging necessity for programming techniques that allow for efficient adjustment of the robot’s behavior to new environments or tasks. A promising approach for teaching robots a certain behavior is Inverse Reinforcement Learning (IRL), which estimates the underlying reward function of a Markov Decision Process (MDP) from observed behavior of an expert. Recently, an approach called Simultaneous Estimation of Rewards and Dynamics (SERD) has been proposed, which extends IRL by simultaneously estimating the dynamics. The objective of this work is to compare classical IRL algorithms with SERD for learning high level navigation strategies in a realistic hallway navigation scenario solely from human expert demonstrations. We show that the theoretical advantages of SERD also pay off in practice by estimating better models of the dynamics and explaining the expert’s demonstrations more accurately.