Optimal Market Making by Reinforcement Learning

TL;DR

This paper demonstrates how reinforcement learning, specifically Deep Q-Learning, can effectively solve the market making problem by balancing inventory risk and profit, achieving near-optimal strategies.

Contribution

It introduces a reinforcement learning framework for market making that aligns with the utility maximization goal and shows that Deep Q-Learning can recover optimal policies.

Findings

Deep Q-Learning recovers the optimal market making strategy.

The environment's reward function aligns with the utility function.

Reinforcement learning outperforms benchmark agents.

Abstract

We apply Reinforcement Learning algorithms to solve the classic quantitative finance Market Making problem, in which an agent provides liquidity to the market by placing buy and sell orders while maximizing a utility function. The optimal agent has to find a delicate balance between the price risk of her inventory and the profits obtained by capturing the bid-ask spread. We design an environment with a reward function that determines an order relation between policies equivalent to the original utility function. When comparing our agents with the optimal solution and a benchmark symmetric agent, we find that the Deep Q-Learning algorithm manages to recover the optimal agent.