Stock market microstructure inference via multi-agent reinforcement learning

TL;DR

This paper introduces a multi-agent reinforcement learning-based stock market simulator that learns trading behaviors autonomously, accurately reproducing real market microstructure features from historical data.

Contribution

It presents a novel multi-agent system where agents learn trading strategies via reinforcement learning, improving realism in market microstructure modeling compared to zero-intelligence models.

Findings

Successfully calibrated to London Stock Exchange data (2007-2018)

Reproduces key microstructure metrics such as price autocorrelations

Demonstrates the importance of agent learning in market simulation

Abstract

Quantitative finance has had a long tradition of a bottom-up approach to complex systems inference via multi-agent systems (MAS). These statistical tools are based on modelling agents trading via a centralised order book, in order to emulate complex and diverse market phenomena. These past financial models have all relied on so-called zero-intelligence agents, so that the crucial issues of agent information and learning, central to price formation and hence to all market activity, could not be properly assessed. In order to address this, we designed a next-generation MAS stock market simulator, in which each agent learns to trade autonomously via model-free reinforcement learning. We calibrate the model to real market data from the London Stock Exchange over the years $2007$ to $2018$, and show that it can faithfully reproduce key market microstructure metrics, such as various price autocorrelation scalars over multiple time intervals. Agent learning thus enables model emulation of the microstructure with greater realism.