Hierarchical learning of robotic contact policies

• Published in: Robotics and computer-integrated manufacturing Vol. 86; p. 102657
• Main Authors: Simonič, Mihael, Ude, Aleš, Nemec, Bojan
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.04.2024
• Subjects: Autonomous robot learning; Compliance and impedance control; Compliant assembly; Learning from experience; Perception-action coupling; Perception-action coupling; Compliance and impedance control; Autonomous robot learning; Learning from experience; Compliant assembly
• ISSN: 0736-5845; 1879-2537
• DOI: 10.1016/j.rcim.2023.102657

The paper addresses the issue of learning tasks where a robot maintains permanent contact with the environment. We propose a new methodology based on a hierarchical learning scheme coupled with task representation through directed graphs. These graphs are constituted of nodes and branches that correspond to the states and robotic actions, respectively. The upper level of the hierarchy essentially operates as a decision-making algorithm. It leverages reinforcement learning (RL) techniques to facilitate optimal decision-making. The actions are generated by a constraint-space following (CSF) controller that autonomously identifies feasible directions for motion. The controller generates robot motion by adjusting its stiffness in the direction defined by the Frenet–Serret frame, which is aligned with the robot path. The proposed framework was experimentally verified through a series of challenging robotic tasks such as maze learning, door opening, learning to shift the manual car gear, and learning car license plate light assembly by disassembly. •A novel framework for autonomous learning of tasks where the robot maintains contact with the environment.•A graph-based task representation to decouple learning at different levels.•A novel algorithm to autonomously discover both the graph topology and the control policies for transitioning between the nodes of the learnt graph.•Fast learning rate, comparable to human learning.