Deep Graph Convolutional Reinforcement Learning for Financial Portfolio Management -- DeepPocket

TL;DR

DeepPocket is a novel graph convolutional reinforcement learning framework that models and exploits the dynamic correlations between financial assets to optimize portfolio management, demonstrating superior performance on real-world data.

Contribution

It introduces a graph-based reinforcement learning approach with an autoencoder and actor-critic architecture for adaptive portfolio management.

Findings

Outperforms market indexes on five real-life datasets.

Effectively handles concept drift with online training.

Maintains robust performance during volatile periods like Covid-19.

Abstract

Portfolio management aims at maximizing the return on investment while minimizing risk by continuously reallocating the assets forming the portfolio. These assets are not independent but correlated during a short time period. A graph convolutional reinforcement learning framework called DeepPocket is proposed whose objective is to exploit the time-varying interrelations between financial instruments. These interrelations are represented by a graph whose nodes correspond to the financial instruments while the edges correspond to a pair-wise correlation function in between assets. DeepPocket consists of a restricted, stacked autoencoder for feature extraction, a convolutional network to collect underlying local information shared among financial instruments, and an actor-critic reinforcement learning agent. The actor-critic structure contains two convolutional networks in which the actor learns and enforces an investment policy which is, in turn, evaluated by the critic in order to determine the best course of action by constantly reallocating the various portfolio assets to optimize the expected return on investment. The agent is initially trained offline with online stochastic batching on historical data. As new data become available, it is trained online with a passive concept drift approach to handle unexpected changes in their distributions. DeepPocket is evaluated against five real-life datasets over three distinct investment periods, including during the Covid-19 crisis, and clearly outperformed market indexes.