Deep Reinforcement Learning for Foreign Exchange Trading

TL;DR

This paper applies deep reinforcement learning algorithms, specifically DQN and PPO, to foreign exchange trading, demonstrating their potential to handle complex market dynamics and improve trading strategies.

Contribution

It introduces a novel system that encodes forex prices into heat-maps and compares reinforcement learning algorithms for trading, validating their effectiveness.

Findings

Reinforcement learning models achieved favorable trading performance.

Encoding prices as heat-maps aids in capturing market trends.

Models successfully handled complex, random forex market processes.

Abstract

Reinforcement learning can interact with the environment and is suitable for applications in decision control systems. Therefore, we used the reinforcement learning method to establish a foreign exchange transaction, avoiding the long-standing problem of unstable trends in deep learning predictions. In the system design, we optimized the Sure-Fire statistical arbitrage policy, set three different actions, encoded the continuous price over a period of time into a heat-map view of the Gramian Angular Field (GAF) and compared the Deep Q Learning (DQN) and Proximal Policy Optimization (PPO) algorithms. To test feasibility, we analyzed three currency pairs, namely EUR/USD, GBP/USD, and AUD/USD. We trained the data in units of four hours from 1 August 2018 to 30 November 2018 and tested model performance using data between 1 December 2018 and 31 December 2018. The test results of the various models indicated that favorable investment performance was achieved as long as the model was able to handle complex and random processes and the state was able to describe the environment, validating the feasibility of reinforcement learning in the development of trading strategies.