Evolving action pre-selection parameters for MCTS in real-time strategy games

• Published in: Entertainment computing Vol. 42; p. 100493
• Main Authors: Ouessai, Abdessamed, Salem, Mohammed, Mora, Antonio M.
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.05.2022
• Subjects: [formula omitted]RTS; Game AI; Genetic Algorithms; MCTS; Parameter Optimization; Real-Time Strategy; Game AI; MCTS; Real-Time Strategy; Genetic Algorithms; μRTS; Parameter Optimization
• ISSN: 1875-9521; 1875-953X
• DOI: 10.1016/j.entcom.2022.100493

•The massive number of possible actions in an RTS game is a challenge for AI.•Using domain knowledge can lead an AI towards promising states.•Small-scale heuristics could help in shaping the space of actions.•Heuristics may be combined to form a wide array of strategies.•Genetic algorithms are able to automatically configure heuristics for optimal play. Real-Time Strategy (RTS) games are well-known for their substantially large combinatorial decision and state spaces, responsible for creating significant challenges for search and machine learning techniques. Exploiting domain knowledge to assist in navigating the expansive decision and state spaces could facilitate the emergence of competitive RTS game-playing agents. Usually, domain knowledge can take the form of expert traces or expert-authored scripts. A script encodes a strategy conceived by a human expert and can be used to steer a search algorithm, such as Monte Carlo Tree Search (MCTS), towards high-value states. However, a script is coarse by nature, meaning that it could be subject to exploitation and poor low-level tactical performance. We propose to perceive scripts as a collection of heuristics that can be parameterized and combined to form a wide array of strategies. The parameterized heuristics mold and filter the decision space in favor of a strategy expressed in terms of parameters. The proposed agent, ParaMCTS, implements several common heuristics and uses NaïveMCTS to search the downsized decision space; however, it requires a preceding manual parameterization step. A genetic algorithm is proposed for use in an optimization phase that aims to replace manual tuning and find an optimal set of parameters for use by EvoPMCTS, the evolutionary counterpart of ParaMCTS. Experimentation results using the μRTS testbed show that EvoPMCTS outperforms several state-of-the-art agents across multiple maps of distinct layouts.