Deep reinforcement learning on a multi-asset environment for trading

TL;DR

This paper explores the application of deep reinforcement learning, specifically deep Q-networks, to develop and test trading strategies for futures contracts, demonstrating potential benefits and challenges in real market conditions.

Contribution

It introduces a DRL-based trading approach using DQN for futures, combining artificial and real data for training, and evaluates its performance on the E-mini S&P 500 futures.

Findings

Preliminary positive results in trading performance.

Successful deployment of combined artificial and real data training.

Potential for DRL methods in financial trading, requiring further refinement.

Abstract

Financial trading has been widely analyzed for decades with market participants and academics always looking for advanced methods to improve trading performance. Deep reinforcement learning (DRL), a recently reinvigorated method with significant success in multiple domains, still has to show its benefit in the financial markets. We use a deep Q-network (DQN) to design long-short trading strategies for futures contracts. The state space consists of volatility-normalized daily returns, with buying or selling being the reinforcement learning action and the total reward defined as the cumulative profits from our actions. Our trading strategy is trained and tested both on real and simulated price series and we compare the results with an index benchmark. We analyze how training based on a combination of artificial data and actual price series can be successfully deployed in real markets. The trained reinforcement learning agent is applied to trading the E-mini S&P 500 continuous futures contract. Our results in this study are preliminary and need further improvement.