Modelling crypto markets by multi-agent reinforcement learning

TL;DR

This paper presents a multi-agent reinforcement learning model for crypto markets, calibrated with Binance data from 2018-2022, capturing market microstructure and behaviors across different market regimes.

Contribution

It introduces a novel MARL approach with agents that perform asset valuation using market prices and fundamental estimates, improving robustness over previous models.

Findings

Accurately emulates crypto market microstructure.

Captures market behaviors during bullish and bearish regimes.

Demonstrates robustness in volatile market conditions.

Abstract

Building on a previous foundation work (Lussange et al. 2020), this study introduces a multi-agent reinforcement learning (MARL) model simulating crypto markets, which is calibrated to the Binance's daily closing prices of $153$ cryptocurrencies that were continuously traded between 2018 and 2022. Unlike previous agent-based models (ABM) or multi-agent systems (MAS) which relied on zero-intelligence agents or single autonomous agent methodologies, our approach relies on endowing agents with reinforcement learning (RL) techniques in order to model crypto markets. This integration is designed to emulate, with a bottom-up approach to complexity inference, both individual and collective agents, ensuring robustness in the recent volatile conditions of such markets and during the COVID-19 era. A key feature of our model also lies in the fact that its autonomous agents perform asset price valuation based on two sources of information: the market prices themselves, and the approximation of the crypto assets fundamental values beyond what those market prices are. Our MAS calibration against real market data allows for an accurate emulation of crypto markets microstructure and probing key market behaviors, in both the bearish and bullish regimes of that particular time period.