Resolving Latency and Inventory Risk in Market Making with Reinforcement Learning

TL;DR

This paper introduces Relaver, an RL-based market making method that effectively manages latency and inventory risks by incorporating realistic exchange delays, batch matching, and market trend prediction, improving performance over existing strategies.

Contribution

The paper presents a novel RL framework, Relaver, that accounts for exchange latency and inventory risk through augmented state spaces, dynamic programming guidance, and market trend prediction.

Findings

Relaver outperforms existing RL-based market making strategies on real-world datasets.

Incorporating latency and inventory considerations improves trading performance.

Dynamic programming enhances RL training efficiency and policy quality.

Abstract

The latency of the exchanges in Market Making (MM) is inevitable due to hardware limitations, system processing times, delays in receiving data from exchanges, the time required for order transmission to reach the market, etc. Existing reinforcement learning (RL) methods for Market Making (MM) overlook the impact of these latency, which can lead to unintended order cancellations due to price discrepancies between decision and execution times and result in undesired inventory accumulation, exposing MM traders to increased market risk. Therefore, these methods cannot be applied in real MM scenarios. To address these issues, we first build a realistic MM environment with random delays of 30-100 milliseconds for order placement and market information reception, and implement a batch matching mechanism that collects orders within every 500 milliseconds before matching them all at once, simulating the batch auction mechanisms adopted by some exchanges. Then, we propose Relaver, an RL-based method for MM to tackle the latency and inventory risk issues. The three main contributions of Relaver are: i) we introduce an augmented state-action space that incorporates order hold time alongside price and volume, enabling Relaver to optimize execution strategies under latency constraints and time-priority matching mechanisms, ii) we leverage dynamic programming (DP) to guide the exploration of RL training for better policies, iii) we train a market trend predictor, which can guide the agent to intelligently adjust the inventory to reduce the risk. Extensive experiments and ablation studies on four real-world datasets demonstrate that \textsc{Relaver} significantly improves the performance of state-of-the-art RL-based MM strategies across multiple metrics.