A survey and critique of multiagent deep reinforcement learning

• Published in: Autonomous agents and multi-agent systems Vol. 33; no. 6; pp. 750 - 797
• Main Authors: Hernandez-Leal, Pablo, Kartal, Bilal, Taylor, Matthew E.
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.11.2019; Springer Nature B.V
• Subjects: Artificial Intelligence; Computer Science; Computer Systems Organization and Communication Networks; Domains; Machine learning; Multiagent systems; New Horizons in Multiagent Learning; Software Engineering/Programming and Operating Systems; User Interfaces and Human Computer Interaction; Multiagent learning; Deep reinforcement learning; Multiagent systems; Survey; Multiagent reinforcement learning
• ISSN: 1387-2532; 1573-7454
• DOI: 10.1007/s10458-019-09421-1

Deep reinforcement learning (RL) has achieved outstanding results in recent years. This has led to a dramatic increase in the number of applications and methods. Recent works have explored learning beyond single-agent scenarios and have considered multiagent learning (MAL) scenarios. Initial results report successes in complex multiagent domains, although there are several challenges to be addressed. The primary goal of this article is to provide a clear overview of current multiagent deep reinforcement learning (MDRL) literature. Additionally, we complement the overview with a broader analysis: (i) we revisit previous key components, originally presented in MAL and RL, and highlight how they have been adapted to multiagent deep reinforcement learning settings. (ii) We provide general guidelines to new practitioners in the area: describing lessons learned from MDRL works, pointing to recent benchmarks, and outlining open avenues of research. (iii) We take a more critical tone raising practical challenges of MDRL (e.g., implementation and computational demands). We expect this article will help unify and motivate future research to take advantage of the abundant literature that exists (e.g., RL and MAL) in a joint effort to promote fruitful research in the multiagent community.