Select and Trade: Towards Unified Pair Trading with Hierarchical Reinforcement Learning

TL;DR

This paper introduces a hierarchical reinforcement learning framework that unifies pair selection and trading in a single model, improving performance in statistical arbitrage strategies.

Contribution

It proposes a novel hierarchical RL approach that jointly optimizes pair selection and trading, overcoming limitations of traditional two-step methods.

Findings

Outperforms existing pair trading methods on real stock data

Effectively unifies pair selection and trading into a single framework

Demonstrates improved profit and robustness over traditional methods

Abstract

Pair trading is one of the most effective statistical arbitrage strategies which seeks a neutral profit by hedging a pair of selected assets. Existing methods generally decompose the task into two separate steps: pair selection and trading. However, the decoupling of two closely related subtasks can block information propagation and lead to limited overall performance. For pair selection, ignoring the trading performance results in the wrong assets being selected with irrelevant price movements, while the agent trained for trading can overfit to the selected assets without any historical information of other assets. To address it, in this paper, we propose a paradigm for automatic pair trading as a unified task rather than a two-step pipeline. We design a hierarchical reinforcement learning framework to jointly learn and optimize two subtasks. A high-level policy would select two assets from all possible combinations and a low-level policy would then perform a series of trading actions. Experimental results on real-world stock data demonstrate the effectiveness of our method on pair trading compared with both existing pair selection and trading methods.