Deep Reinforcement Learning Approach for Trading Automation in The Stock Market

TL;DR

This paper presents a Deep Reinforcement Learning model using TD3 for stock trading automation, demonstrating superior decision-making and profitability over traditional methods, with a reported Sharpe Ratio of 2.68 on unseen data.

Contribution

It formulates stock trading as a POMDP and applies DRL with TD3, showcasing improved autonomous trading strategies over supervised learning approaches.

Findings

Achieved a Sharpe Ratio of 2.68 on test data.

Demonstrated DRL's superiority over other machine learning methods.

Validated the effectiveness of the POMDP formulation for trading.

Abstract

Deep Reinforcement Learning (DRL) algorithms can scale to previously intractable problems. The automation of profit generation in the stock market is possible using DRL, by combining the financial assets price "prediction" step and the "allocation" step of the portfolio in one unified process to produce fully autonomous systems capable of interacting with their environment to make optimal decisions through trial and error. This work represents a DRL model to generate profitable trades in the stock market, effectively overcoming the limitations of supervised learning approaches. We formulate the trading problem as a Partially Observed Markov Decision Process (POMDP) model, considering the constraints imposed by the stock market, such as liquidity and transaction costs. We then solve the formulated POMDP problem using the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm reporting a 2.68 Sharpe Ratio on unseen data set (test data). From the point of view of stock market forecasting and the intelligent decision-making mechanism, this paper demonstrates the superiority of DRL in financial markets over other types of machine learning and proves its credibility and advantages of strategic decision-making.