Transfer Learning in Deep Reinforcement Learning: A Survey

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 45; no. 11; pp. 1 - 20
• Main Authors: Zhu, Zhuangdi, Lin, Kaixiang, Jain, Anil K., Zhou, Jiayu
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.11.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; Deep learning; Machine learning; Markov processes; Neural networks; Reinforcement learning; Robotics; survey; Surveys; Task analysis; Transfer learning
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2023.3292075

Reinforcement learning is a learning paradigm for solving sequential decision-making problems. Recent years have witnessed remarkable progress in reinforcement learning upon the fast development of deep neural networks. Along with the promising prospects of reinforcement learning in numerous domains such as robotics and game-playing, transfer learning has arisen to tackle various challenges faced by reinforcement learning, by transferring knowledge from external expertise to facilitate the efficiency and effectiveness of the learning process. In this survey, we systematically investigate the recent progress of transfer learning approaches in the context of deep reinforcement learning. Specifically, we provide a framework for categorizing the state-of-the-art transfer learning approaches, under which we analyze their goals, methodologies, compatible reinforcement learning backbones, and practical applications. We also draw connections between transfer learning and other relevant topics from the reinforcement learning perspective and explore their potential challenges that await future research progress.