Improving Deep Reinforcement Learning Agent Trading Performance in Forex using Auxiliary Task

TL;DR

This paper enhances Deep Reinforcement Learning for Forex trading by integrating an auxiliary task with PPO, significantly improving profitability and risk management based on simulation and backtesting results.

Contribution

Introduces an auxiliary task to PPO in DRL, improving reward modeling and trading performance in Forex markets.

Findings

Increased overall return from -25.25% to 14.86% in dataset 1

Improved Sharpe ratio from -2.61 to 0.24 in dataset 1

Achieved higher returns and better risk metrics in dataset 2

Abstract

Advanced algorithms based on Deep Reinforcement Learning (DRL) have been able to become a reliable tool for the Forex market traders and provide a suitable strategy for maximizing profit and reducing trading risk. These tools try to find the most profitable strategy in this market by examining past market data. Artificial intelligent agents based on the Proximal Policy Optimization (PPO) algorithm, one of the DRL algorithms, have shown a special ability to determine a profitable strategy. In this research, to increase profitability and determine the optimal strategy for the PPO, an auxiliary task has been used. The auxiliary function helps the PPO to model the reward function in a better way by recognizing and classifying patterns to obtain additional information from the problem's inputs and data. The results of simulation and backtesting on the EUR/USD currency pair have shown that the new proposed model has been able to increase the overall return from -25.25% to 14.86% and the value of sharp ratio from -2.61 to 0.24 in dataset 1 and increased overall return from 2.12% to 42.22% and the value of sharp ratio from -2.93 to 0.47 in dataset 2. These results indicate better identification of trends, timely trading, and great improvement in risk reduction while increasing the profit. It has also reduced the transaction risk and can be used as an effective tool for automatic trading in the complex forex market