A Framework for Few-Shot Policy Transfer through Observation Mapping and Behavior Cloning

• Main Authors: Shukla, Yash, Kesari, Bharat, Goel, Shivam, Wright, Robert, Sinapov, Jivko
• Format: Journal Article
• Language: English
• Published: 12.10.2023
• Subjects: Computer Science - Artificial Intelligence; Computer Science - Learning; Computer Science - Robotics
• DOI: 10.48550/arxiv.2310.08836

Despite recent progress in Reinforcement Learning for robotics applications, many tasks remain prohibitively difficult to solve because of the expensive interaction cost. Transfer learning helps reduce the training time in the target domain by transferring knowledge learned in a source domain. Sim2Real transfer helps transfer knowledge from a simulated robotic domain to a physical target domain. Knowledge transfer reduces the time required to train a task in the physical world, where the cost of interactions is high. However, most existing approaches assume exact correspondence in the task structure and the physical properties of the two domains. This work proposes a framework for Few-Shot Policy Transfer between two domains through Observation Mapping and Behavior Cloning. We use Generative Adversarial Networks (GANs) along with a cycle-consistency loss to map the observations between the source and target domains and later use this learned mapping to clone the successful source task behavior policy to the target domain. We observe successful behavior policy transfer with limited target task interactions and in cases where the source and target task are semantically dissimilar.