Modelling Stock Markets by Multi-agent Reinforcement Learning

• Published in: Computational economics Vol. 57; no. 1; pp. 113 - 147
• Main Authors: Lussange, Johann, Lazarevich, Ivan, Bourgeois-Gironde, Sacha, Palminteri, Stefano, Gutkin, Boris
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.01.2021; Springer Nature B.V; Springer Verlag
• Subjects: Agents; Behavioral/Experimental Economics; Cognitive science; Complex systems; Computer Appl. in Social and Behavioral Sciences; Economic Theory/Quantitative Economics/Mathematical Methods; Economics; Economics and Finance; Finance; Intelligence; Learning; Math Applications in Computer Science; Microstructure; Modelling; Multiagent systems; Operations Research/Decision Theory; Reinforcement; Scalars; Securities markets; Statistical inference; Stock exchanges; Trading; Multi-agent system; Agent-based; Stock markets; Reinforcement learning
• DOI: 10.1007/s10614-020-10038-w
• ISSN: 0927-7099

Quantitative finance has had a long tradition of a bottom-up approach to complex systems inference via multi-agent systems (MAS). These statistical tools are based on modelling agents trading via a centralised order book, in order to emulate complex and diverse market phenomena. These past financial models have all relied on so-called zero-intelligence agents, so that the crucial issues of agent information and learning, central to price formation and hence to all market activity, could not be properly assessed. In order to address this, we designed a next-generation MAS stock market simulator, in which each agent learns to trade autonomously via reinforcement learning. We calibrate the model to real market data from the London Stock Exchange over the years 2007 to 2018, and show that it can faithfully reproduce key market microstructure metrics, such as various price autocorrelation scalars over multiple time intervals. Agent learning thus enables accurate emulation of the market microstructure as an emergent property of the MAS.