A Novel Experts Advice Aggregation Framework Using Deep Reinforcement Learning for Portfolio Management

TL;DR

This paper introduces a novel deep reinforcement learning framework that combines expert signals and historical data for portfolio management, achieving significant profit approximation to top experts.

Contribution

It is the first to integrate expert signals with deep RL in a portfolio management framework, using convolutional networks and PPO for decision-making.

Findings

Framework can achieve 90% of the profit of the best expert.

Uses convolutional networks for signal and price data aggregation.

Employs PPO algorithm for reinforcement learning in finance.

Abstract

Solving portfolio management problems using deep reinforcement learning has been getting much attention in finance for a few years. We have proposed a new method using experts signals and historical price data to feed into our reinforcement learning framework. Although experts signals have been used in previous works in the field of finance, as far as we know, it is the first time this method, in tandem with deep RL, is used to solve the financial portfolio management problem. Our proposed framework consists of a convolutional network for aggregating signals, another convolutional network for historical price data, and a vanilla network. We used the Proximal Policy Optimization algorithm as the agent to process the reward and take action in the environment. The results suggested that, on average, our framework could gain 90 percent of the profit earned by the best expert.