Market Making via Reinforcement Learning

TL;DR

This paper presents a reinforcement learning-based market making agent that uses a high-fidelity simulation and custom reward functions to outperform benchmark strategies in limit order book markets.

Contribution

It introduces a novel RL approach with a custom reward for inventory risk management in market making, validated through high-fidelity simulations.

Findings

The RL agent outperforms benchmark strategies.

The approach effectively manages inventory risk.

The method surpasses recent online learning approaches.

Abstract

Market making is a fundamental trading problem in which an agent provides liquidity by continually offering to buy and sell a security. The problem is challenging due to inventory risk, the risk of accumulating an unfavourable position and ultimately losing money. In this paper, we develop a high-fidelity simulation of limit order book markets, and use it to design a market making agent using temporal-difference reinforcement learning. We use a linear combination of tile codings as a value function approximator, and design a custom reward function that controls inventory risk. We demonstrate the effectiveness of our approach by showing that our agent outperforms both simple benchmark strategies and a recent online learning approach from the literature.