Information capture and reuse strategies in Monte Carlo Tree Search, with applications to games of hidden information

• Published in: Artificial intelligence Vol. 217; pp. 92 - 116
• Main Authors: Powley, Edward J., Cowling, Peter I., Whitehouse, Daniel
• Format: Journal Article
• Language: English
• Published: Oxford Elsevier B.V 01.12.2014; Elsevier
• Subjects: Applied sciences; Artificial intelligence; Asymmetry; Computer science; control theory; systems; Decision making; Decision theory. Utility theory; Exact sciences and technology; Game tree search; Games; Hidden information; Information reuse; Machine learning; Monte Carlo methods; Monte Carlo Tree Search (MCTS); Operational research and scientific management; Operational research. Management science; Reuse; Searching; Strategy; Trees; Uncertainty; Monte Carlo Tree Search (MCTS); Information reuse; Uncertainty; Hidden information; Game tree search; Machine learning; Monte Carlo method; Decision making; Reuse; Graph theory; Random distribution; Modeling; Game theory; Uncertain system; Asymmetry; Information communication; Efficiency; Learning (artificial intelligence); Randomness; Search tree; Artificial intelligence; Frame of reference
• ISSN: 0004-3702; 1872-7921
• DOI: 10.1016/j.artint.2014.08.002

Monte Carlo Tree Search (MCTS) has produced many breakthroughs in search-based decision-making in games and other domains. There exist many general-purpose enhancements for MCTS, which improve its efficiency and effectiveness by learning information from one part of the search space and using it to guide the search in other parts. We introduce the Information Capture And ReUse Strategy (ICARUS) framework for describing and combining such enhancements. We demonstrate the ICARUS framework's usefulness as a frame of reference for understanding existing enhancements, combining them, and designing new ones. We also use ICARUS to adapt some well-known MCTS enhancements (originally designed for games of perfect information) to handle information asymmetry between players and randomness, features which can make decision-making much more difficult. We also introduce a new enhancement designed within the ICARUS framework, EPisodic Information Capture and reuse (EPIC), designed to exploit the episodic nature of many games. Empirically we demonstrate that EPIC is stronger and more robust than existing enhancements in a variety of game domains, thus validating ICARUS as a powerful tool for enhancement design within MCTS.