Deep Reinforcement Learning for Market Making Under a Hawkes Process-Based Limit Order Book Model

TL;DR

This paper develops a deep reinforcement learning controller for market making using a Hawkes process-based limit order book model, demonstrating superior performance over benchmarks in simulated trading scenarios.

Contribution

It introduces a novel RL-based market making controller trained on a Hawkes process model, advancing the application of deep learning in complex financial market simulations.

Findings

RL controller outperforms benchmarks in risk-reward metrics

Controller remains effective under high transaction costs

Method leverages Monte Carlo backtesting for validation

Abstract

The stochastic control problem of optimal market making is among the central problems in quantitative finance. In this paper, a deep reinforcement learning-based controller is trained on a weakly consistent, multivariate Hawkes process-based limit order book simulator to obtain market making controls. The proposed approach leverages the advantages of Monte Carlo backtesting and contributes to the line of research on market making under weakly consistent limit order book models. The ensuing deep reinforcement learning controller is compared to multiple market making benchmarks, with the results indicating its superior performance with respect to various risk-reward metrics, even under significant transaction costs.