A Sample Efficiency Improved Method via Hierarchical Reinforcement Learning Networks

• Published in: 2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN) pp. 1498 - 1505
• Main Authors: Chen, Qinghua, Dallas, Evan, Shahverdi, Pourya, Korneder, Jessica, Rawashdeh, Osamah A., Geoffrey Louie, Wing-Yue
• Format: Conference Proceeding
• Language: English
• Published: IEEE 29.08.2022
• Subjects: Education; Medical services; Reinforcement learning; Social robots; Task analysis
• ISSN: 1944-9437
• DOI: 10.1109/RO-MAN53752.2022.9900738

Learning from demonstration (LfD) approaches have garnered significant interest for teaching social robots a variety of tasks in healthcare, educational, and service domains after they have been deployed. These LfD approaches often require a significant number of demonstrations for a robot to learn a performant model from task demonstrations. However, requiring non-experts to provide numerous demonstrations for a social robot to learn a task is impractical in real-world applications. In this paper, we propose a method to improve the sample efficiency of existing learning from demonstration approaches via data augmentation, dynamic experience replay sizes, and hierarchical Deep Q-Networks (DQN). After validating our methods on two different datasets, results suggest that our proposed hierarchical DQN is effective for improving sample efficiency when learning tasks from demonstration. In the future, such a sample-efficient approach has the potential to improve our ability to apply LfD approaches for social robots to learn tasks in domains where demonstration data is limited, sparse, and imbalanced.