An intelligent algorithmic trading based on a risk-return reinforcement learning algorithm

TL;DR

This paper introduces a novel deep reinforcement learning algorithm for portfolio optimization that incorporates risk measures like VaR, uses an actor-critic architecture with quantile regression, and demonstrates superior backtest performance.

Contribution

It presents an improved deep reinforcement learning model with a new objective function and multi-process training for portfolio optimization, including asset short selling capabilities.

Findings

The proposed model outperforms benchmark strategies in backtesting.

Incorporating VaR improves risk-adjusted returns.

Multi-process training accelerates convergence.

Abstract

This scientific paper propose a novel portfolio optimization model using an improved deep reinforcement learning algorithm. The objective function of the optimization model is the weighted sum of the expectation and value at risk(VaR) of portfolio cumulative return. The proposed algorithm is based on actor-critic architecture, in which the main task of critical network is to learn the distribution of portfolio cumulative return using quantile regression, and actor network outputs the optimal portfolio weight by maximizing the objective function mentioned above. Meanwhile, we exploit a linear transformation function to realize asset short selling. Finally, A multi-process method is used, called Ape-x, to accelerate the speed of deep reinforcement learning training. To validate our proposed approach, we conduct backtesting for two representative portfolios and observe that the proposed model in this work is superior to the benchmark strategies.