Over-the-Counter Market Making via Reinforcement Learning

TL;DR

This paper introduces a reinforcement learning approach to optimize market-making strategies in OTC markets, accounting for complex bid-ask spread adjustments based on order sizes, and demonstrates the optimal policy under certain assumptions.

Contribution

It proposes a novel RL-based framework for OTC market-making, modeling the problem as a high-dimensional stochastic control task with a Gaussian spread distribution.

Findings

Optimal bid-ask spread policy follows a Gaussian distribution.

Reinforcement learning algorithms effectively analyze return distributions.

The approach adapts to different time and inventory levels.

Abstract

The over-the-counter (OTC) market is characterized by a unique feature that allows market makers to adjust bid-ask spreads based on order size. However, this flexibility introduces complexity, transforming the market-making problem into a high-dimensional stochastic control problem that presents significant challenges. To address this, this paper proposes an innovative solution utilizing reinforcement learning techniques to tackle the OTC market-making problem. By assuming a linear inverse relationship between market order arrival intensity and bid-ask spreads, we demonstrate the optimal policy for bid-ask spreads follows a Gaussian distribution. We apply two reinforcement learning algorithms to conduct a numerical analysis, revealing the resulting return distribution and bid-ask spreads under different time and inventory levels.