DeepStack Expert-level artificial intelligence in heads-up no-limit poker

• Published in: Science (American Association for the Advancement of Science) Vol. 356; no. 6337; pp. 508 - 513
• Main Authors: Moravčík, Matej, Schmid, Martin, Burch, Neil, Lisý, Viliam, Morrill, Dustin, Bard, Nolan, Davis, Trevor, Waugh, Kevin, Johanson, Michael, Bowling, Michael
• Format: Journal Article
• Language: English
• Published: United States American Association for the Advancement of Science 05.05.2017; The American Association for the Advancement of Science
• Subjects: Algorithms; Artificial Intelligence; Asymmetry; Boards; Building codes; Cards; Chess; Computers; Expert systems; Games; Head; Machine learning; Players; Statistical Significance; Strategy; World Problems; United States > US; Texas
• ISSN: 0036-8075; 1095-9203
• DOI: 10.1126/science.aam6960

Artificial intelligence has seen several breakthroughs in recent years, with games often serving as milestones. A common feature of these games is that players have perfect information. Poker, the quintessential game of imperfect information, is a long-standing challenge problem in artificial intelligence. We introduce DeepStack, an algorithm for imperfect-information settings. It combines recursive reasoning to handle information asymmetry, decomposition to focus computation on the relevant decision, and a form of intuition that is automatically learned from self-play using deep learning. In a study involving 44,000 hands of poker, DeepStack defeated, with statistical significance, professional poker players in heads-up no-limit Texas hold’em. The approach is theoretically sound and is shown to produce strategies that are more difficult to exploit than prior approaches.