Deep Reinforcement Learning for Multiagent Systems: A Review of Challenges, Solutions, and Applications

• Published in: IEEE transactions on cybernetics Vol. 50; no. 9; pp. 3826 - 3839
• Main Authors: Nguyen, Thanh Thi, Nguyen, Ngoc Duy, Nahavandi, Saeid
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.09.2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Continuous action space; Decision making; Deep learning; deep reinforcement learning (RL); Dynamic programming; Games; Machine learning; Mathematical model; multiagent; Multiagent systems; nonstationary; Observability; partial observability; Reinforcement learning; review; robotics; Robots; survey; Task complexity
• ISSN: 2168-2267; 2168-2275; 2168-2275
• DOI: 10.1109/TCYB.2020.2977374

Reinforcement learning (RL) algorithms have been around for decades and employed to solve various sequential decision-making problems. These algorithms, however, have faced great challenges when dealing with high-dimensional environments. The recent development of deep learning has enabled RL methods to drive optimal policies for sophisticated and capable agents, which can perform efficiently in these challenging environments. This article addresses an important aspect of deep RL related to situations that require multiple agents to communicate and cooperate to solve complex tasks. A survey of different approaches to problems related to multiagent deep RL (MADRL) is presented, including nonstationarity, partial observability, continuous state and action spaces, multiagent training schemes, and multiagent transfer learning. The merits and demerits of the reviewed methods will be analyzed and discussed with their corresponding applications explored. It is envisaged that this review provides insights about various MADRL methods and can lead to the future development of more robust and highly useful multiagent learning methods for solving real-world problems.