Reinforcement Learning Pair Trading: A Dynamic Scaling approach

TL;DR

This paper explores how reinforcement learning can improve cryptocurrency pair trading by outperforming traditional methods, demonstrating significant profit increases in highly volatile markets.

Contribution

It introduces a novel RL-based pair trading approach with customized reward shaping and environment design, tailored for volatile cryptocurrency markets.

Findings

RL-based pair trading achieved up to 31.53% annualized profit

Traditional pair trading achieved 8.33% annualized profit

RL significantly outperforms traditional methods in volatile markets

Abstract

Cryptocurrency is a cryptography-based digital asset with extremely volatile prices. Around USD 70 billion worth of cryptocurrency is traded daily on exchanges. Trading cryptocurrency is difficult due to the inherent volatility of the crypto market. This study investigates whether Reinforcement Learning (RL) can enhance decision-making in cryptocurrency algorithmic trading compared to traditional methods. In order to address this question, we combined reinforcement learning with a statistical arbitrage trading technique, pair trading, which exploits the price difference between statistically correlated assets. We constructed RL environments and trained RL agents to determine when and how to trade pairs of cryptocurrencies. We developed new reward shaping and observation/action spaces for reinforcement learning. We performed experiments with the developed reinforcement learner on pairs of BTC-GBP and BTC-EUR data separated by 1 min intervals (n=263,520). The traditional non-RL pair trading technique achieved an annualized profit of 8.33%, while the proposed RL-based pair trading technique achieved annualized profits from 9.94% to 31.53%, depending upon the RL learner. Our results show that RL can significantly outperform manual and traditional pair trading techniques when applied to volatile markets such as~cryptocurrencies.