Neural Population Learning beyond Symmetric Zero-sum Games

• Main Authors: Liu, Siqi, Marris, Luke, Lanctot, Marc, Piliouras, Georgios, Leibo, Joel Z, Heess, Nicolas
• Format: Journal Article
• Language: English
• Published: 10.01.2024
• Subjects: Computer Science - Artificial Intelligence; Computer Science - Multiagent Systems
• DOI: 10.48550/arxiv.2401.05133

We study computationally efficient methods for finding equilibria in n-player general-sum games, specifically ones that afford complex visuomotor skills. We show how existing methods would struggle in this setting, either computationally or in theory. We then introduce NeuPL-JPSRO, a neural population learning algorithm that benefits from transfer learning of skills and converges to a Coarse Correlated Equilibrium (CCE) of the game. We show empirical convergence in a suite of OpenSpiel games, validated rigorously by exact game solvers. We then deploy NeuPL-JPSRO to complex domains, where our approach enables adaptive coordination in a MuJoCo control domain and skill transfer in capture-the-flag. Our work shows that equilibrium convergent population learning can be implemented at scale and in generality, paving the way towards solving real-world games between heterogeneous players with mixed motives.