Learning Functionally Decomposed Hierarchies for Continuous Control Tasks with Path Planning

We present HiDe, a novel hierarchical reinforcement learning architecture that successfully solves long horizon control tasks and generalizes to unseen test scenarios. Functional decomposition between planning and low-level control is achieved by explicitly separating the state-action spaces across...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Christen, Sammy, Jendele, Lukas, Aksan, Emre, Hilliges, Otmar
Format Paper Journal Article
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 06.10.2021
Subjects
Computer Science - Artificial Intelligence
Computer Science - Learning
Computer Science - Robotics
Control tasks
Decision making
Decomposition
Hierarchies
Humanoid
Learning
Statistics - Machine Learning
Task complexity
Online AccessGet full text

Cover

More Information
Summary:We present HiDe, a novel hierarchical reinforcement learning architecture that successfully solves long horizon control tasks and generalizes to unseen test scenarios. Functional decomposition between planning and low-level control is achieved by explicitly separating the state-action spaces across the hierarchy, which allows the integration of task-relevant knowledge per layer. We propose an RL-based planner to efficiently leverage the information in the planning layer of the hierarchy, while the control layer learns a goal-conditioned control policy. The hierarchy is trained jointly but allows for the modular transfer of policy layers across hierarchies of different agents. We experimentally show that our method generalizes across unseen test environments and can scale to 3x horizon length compared to both learning and non-learning based methods. We evaluate on complex continuous control tasks with sparse rewards, including navigation and robot manipulation.
ISSN:2331-8422
DOI:10.48550/arxiv.2002.05954